A customer maintenance form in Uniface 10, part 12 - 252 tests in one run, a query that hung, and a form painted in the IDE

Parts 9 to 11 added about twenty services. Services without a user interface are only useful to other code, and services without tests are only useful until someone changes them. This part closes both gaps, as far as they are closed right now:

• a test runner that runs all six test services in one go - 252 tests at the time of writing,
• a bulk clean-up service (phone numbers, name case, archiving old customers) that always runs as a dry run first,
• a tools form that makes reports, exports, backup and clean-ups reachable from the main menu.

And in between, the one real incident of the whole series: a test run that stopped halfway and left the database locked.

One runner for six test services

Each test service follows the pattern from part 3: an exec operation that runs its tests, logs PASS/FAIL lines via putmess, prints a summary, rolls back, and returns a negative status if anything failed. So a runner only needs to call exec on each of them and look at the status:

public operation RUN_ALL
variables
    string vSuites, vSuite, vFailed
    numeric vCount, vFailedCount, vStatus
endvariables
    vSuites = ""
    putitem vSuites, -1, "CUSTOMER_TST_SVC"
    putitem vSuites, -1, "ADDRESS_TST_SVC"
    putitem vSuites, -1, "EXPORT_TST_SVC"
    putitem vSuites, -1, "EXT_TST_SVC"
    putitem vSuites, -1, "FEAT_TST_SVC"
    putitem vSuites, -1, "FEAT2_TST_SVC"
    vCount = 0
    vFailedCount = 0
    vFailed = ""
    putmess "===== All test services started ====="
    forlist vSuite in vSuites
        vCount = vCount + 1
        activate vSuite.exec()
        vStatus = $status
        if (vStatus < 0)
            vFailedCount = vFailedCount + 1
            ...
            putmess "===== %%(vSuite): FAILED (status %%(vStatus)) ====="
        else
            putmess "===== %%(vSuite): OK ====="
        endif
    endfor
    rollback
    ...

The line worth noting is activate vSuite.exec(). The component name comes from a variable. Uniface treats it as an instance name and creates the instance on first use. That makes the runner a plain list; adding a suite is one putitem.

The summary of the last run:

===== All test services started =====
Customer service: 35 tests, 0 failures
===== CUSTOMER_TST_SVC: OK =====
Address service: 21 tests, 0 failures
===== ADDRESS_TST_SVC: OK =====
Export service: 10 tests, 0 failures
===== EXPORT_TST_SVC: OK =====
Extension tests: 97 tests, 0 failures
===== EXT_TST_SVC: OK =====
Feature tests: 36 tests, 0 failures
===== FEAT_TST_SVC: OK =====
Feature 2 tests: 53 tests, 0 failures
===== FEAT2_TST_SVC: OK =====
===== All test services: 6 suites, all passed =====

252 tests, run with "Compile & Test" on ALL_TST_SVC in the IDE. Every suite rolls back, and the runner rolls back once more at the end; afterwards the customer tables were empty and the number-range counters unchanged - checked with a query, not assumed.

Where does the output go? putmess writes into the IDE's message frame and into a log file under the user's Uniface folder (log/ide_<pid>.log). The results were read from that file, not from a screenshot. For anyone automating Uniface tests from outside the IDE, that log file is the interface.

The day a test hung

The first run of FEAT_TST_SVC stopped after 32 passed tests. No error, no summary - the IDE simply did not come back. The customers.db-journal file next to the database showed that a write transaction was open, and every other access to the database now waited for the lock.

The statement at that point was a check of the shape

SELECT (SELECT COUNT(*) FROM ... WHERE ...)
     + (SELECT COUNT(*) FROM ... WHERE ...)

• a SELECT without FROM, whose value is the sum of two scalar subqueries.

To be precise about what I know and what I don't:

• Known: the test stopped at that statement and the process held the write lock until it was ended. After ending it, the open transaction was rolled back via the journal and the database was consistent (PRAGMA integrity_check = ok). After replacing the statement with two separate COUNT queries, the suite passed and never hung again.
• Not known: why. The syntax is valid SQLite, so "SQLite can't do this" is not the explanation. The cause is somewhere between Uniface's sql statement, the SLE driver and this query shape - or something else in that test that I have not identified. I cannot prove more than that, and I would not call it a bug report.

So the project rule is purely empirical: in sql statements, no scalar subqueries without a FROM. Scalar subqueries inside a normal SELECT ... FROM ... WHERE are used in several places and work.

Two practical lessons from that day:

1. A test that holds a lock blocks everything, including the investigation: no schema changes, no further test runs. The waiting time went into work that did not need the database: a static check of all service sources, see below.
2. Ending the process is a deliberate decision. First write down the state (process ID, journal file, what would be lost: nothing committed), then end the IDE and continue.

Static checks while waiting

With the database locked, a small script (outside Uniface) parsed all .proc files and checked:

• block structure (if/endif, while/endwhile, forlist/endfor, params/endparams, ...),
• every $concat has at most five arguments (the limit from part 8),
• every activate "X".OP(...) and every call ENTRY(...) targets something that exists, with the right number of parameters,
• all 214 SQL literals in the services, with %%(...) substitutions replaced by sample values, run through EXPLAIN against a copy of the database with the current schema.

It found nothing. That is a good result, but not a surprising one - the compiler would have caught most of it. The value was in the last point: the Uniface compiler does not look inside SQL strings, and EXPLAIN against the real schema does.

Bulk clean-ups: dry run, then ask, then do

CLEANUP_SVC has three operations that change many customers at once:

• NORMALIZE_PHONES - brings every phone number into the format CUSTOMER_SVC.NORMALIZE_PHONE produces (+49/0049 to 0, separators removed),
• CAPITALIZE_NAMES - fixes names typed in all upper or all lower case,
• ARCHIVE_INACTIVE - sets customers to inactive that have not been changed for n months.

All three have the same signature shape: (..., pDryRun, pUser, pCount, pError). With pDryRun = 1 they only count. The form uses that for a two-step dialog:

entry DO_PHONES
variables
    string vUser, vError
    numeric vCount
endvariables
    call CURRENT_USER(vUser)
    activate "CLEANUP_SVC".NORMALIZE_PHONES(1, vUser, vCount, vError)
    if ($status < 0)
        message/error vError
        return -1
    endif
    if (vCount = 0)
        message/info "All phone numbers are already normalized."
        return 0
    endif
    askmess/question "%%(vCount) phone number(s) would be changed. Change them now?", "Yes,No"
    if ($status != 1)
        return 0
    endif
    activate "CLEANUP_SVC".NORMALIZE_PHONES(0, vUser, vCount, vError)
    call FINISH(vCount, vError, "phone number(s) normalized")
    return $status
end

The user never gets "done, 312 records changed" as the first message. They get "312 would be changed - now?". Every real change writes a history entry (PHONE, NAME, STATUS) with old and new value, so a wrong clean-up is at least traceable. FINISH commits on success and rolls back on the first error.

Unlike the import in part 10, this dry run really is a separate code path (if (!pDryRun) around the UPDATE). It is simpler here because nothing depends on earlier rows.

Name case - and where it goes wrong

entry FIX_CASE
params
    string pText : IN
    string pOut : OUT
endparams
...
    vLower = $lowercase(pText)
    vUpper = $uppercase(pText)
    if (vLower = vUpper)
        return 0
    endif
    if (pText != vLower & pText != vUpper)
        return 0
    endif
    pOut = ""
    vPrev = " "
    vLen = $length(pText)
    vPos = 1
    while (vPos <= vLen)
        vCh = vLower[vPos:1]
        if (vPrev = " " | vPrev = "-")
            vCh = $uppercase(vCh)
        endif
        pOut = $concat(pOut, vCh)
        vPrev = vCh
        vPos = vPos + 1
    endwhile
    return 0
end

Only names that are entirely upper or lower case are touched. McDonald or van der Berg typed correctly stay as they are. MÜLLER-LÜDENSCHEIDT becomes Müller-Lüdenscheidt, which the tests check.

What it gets wrong, and why the dry run and the confirmation matter:

• VAN DER BERG becomes Van Der Berg.
• O'NEIL becomes O'neil.
• MCDONALD becomes Mcdonald.

There is no rule that gets all names right. The operation fixes the common case (all caps from an old system or a form someone filled in with caps lock) and relies on the user to look at the count before saying yes. A better version would show the list of proposed changes, not just the number.

The tools form

All of this needed a place in the UI. TOOLS_FRM is a modal form opened from a new "Tools..." button in the main menu. Every button calls one entry in the form's script; every entry calls one service and writes the result into a status line:

entry DO_DBREPORT
variables
    string vDir, vFile, vError
endvariables
    call OUT_DIR(vDir)
    vFile = $concat(vDir, "database.html")
    activate "MAINT_SVC".DB_REPORT(vFile, vError)
    if ($status < 0)
        message/error vError
        return -1
    endif
    call SHOW_FILE(vFile, "Database report:")
    return 0
end

entry SHOW_FILE
params
    string pFile : IN
    string pText : IN
endparams
variables
    string vWin
endvariables
    STATUS_INFO.TOOLS_DMY = $concat(pText, " ", pFile)
    vWin = $replace(pFile, 1, "/", "\", -1)
    spawn $concat("explorer.exe ", vWin)
    return 0
end

spawn "explorer.exe <file>" opens an HTML report in the default browser and a CSV in whatever is registered for it. The path is converted to backslashes first, because Explorer is a Windows program and should get a Windows path; the rest of the app uses forward slashes, which Uniface accepts. The output folder comes from the EXPORT_DIR setting (part 10).

The test: open "Tools..." from the main menu, the fields show 7 and 24 (days for follow-ups, months for archiving), "Database report" writes the file, opens it in the browser and shows the path in the status line, "Close" returns to the menu. The buttons that change data were deliberately not clicked in that session - the database had no test data, and a clean-up test belongs in a test service, where it already is.

Lessons from painting in the IDE

Writing ProcScript is text. Painting a form in the Uniface 10 IDE is mouse work, and it was by far the slowest part of the project. The things that cost the most time, in case you automate the IDE or just want to paint faster yourself:

• Drag and drop from the palette does not create a frame on an empty form. What works: click the template ("Non-DBMS Entity: Occurrence List"), then draw a rectangle on the canvas. The same works for fields inside an existing frame.
• Dropping a field template on the entity row in the structure tree opens an "Insert Frames" dialog and appends the field below the last one. Useful when the frame is full.
• A frame cannot be made taller afterwards - neither with the resize handle nor through the properties. Plan the height when you draw it, or use free rows inside.
• Top, Left, Width and Height of a painted field are read-only in the property grid. Move the field on the canvas (rows are about 13 pixels at this zoom) and resize with the bottom-right handle, slowly.
• The palette scrolls to a different position after each new field. Clicking "where the Command Button was" produced a "Map" widget once. Scroll to a known position and check before every click.
• Name and Initial Value: click the value cell (the text is then selected), type, click on an empty spot. Ctrl+A is ^ACCEPT in the IDE and closes dialogs.
• Never click the same tree row twice in quick succession. It opens the rename box. Earlier in the project that renamed the main menu component to the name of one of its buttons for a moment.
• Appending to a script: Ctrl+End puts the cursor on the last line; selecting that line and deleting it removes the end of the last entry. That happened once in this session and was fixed before compiling - but it is the kind of mistake that a compile error would otherwise report three screens later.

Both forms compiled with 0 errors and 1 warning (1016, the non-modeled dummy entity every form and service in this project has).

Where the series stands

Implemented and tested at service level:

Area	Services
Customers, addresses, export	CUSTOMER_SVC, ADDRESS_SVC, EXPORT_SVC
Users, history, login log	USER_SVC, HISTORY_SVC, AUDIT_SVC
Contacts, notes, categories	CONTACT_SVC, NOTE_SVC, CATEGORY_SVC
Import, backup, settings	IMPORT_SVC, BACKUP_SVC, SETTINGS_SVC
Quality, merge, GDPR	QUALITY_SVC, MERGE_SVC, PRIVACY_SVC
Mailing, follow-ups, saved filters	MAILING_SVC, FOLLOWUP_SVC, FILTER_SVC
Clean-up, maintenance, reports	CLEANUP_SVC, MAINT_SVC, REPORT_SVC

With a user interface: the customer form, the address form, the lookup, the main menu, and now the tools form. Without one: login, user administration, contacts, notes, follow-ups, saved searches, merge, settings and the GDPR functions. The gap between "tested service" and "usable feature" is, in this IDE, mostly painting time.

Takeaways

A test runner can be a list of names. activate vName.exec() with the component name in a variable is all it takes, if every test service follows the same contract: log, roll back, return a negative status on failure.

Write down what you know and what you don't. "This statement hung, the rewrite didn't" is useful. "SQLite can't do scalar subqueries" would be wrong.

Bulk changes get a count first and a question second. And a history entry per change, because a count is not a list.

When the database is locked, check what doesn't need it. EXPLAIN on every SQL string against the real schema is a check the Uniface compiler cannot do.

Budget for the UI. In this project, painting one form with thirteen buttons took longer than writing several of the services it calls.