It Compiles. Does It Do the Same Thing?

A practical way to prove a converted CA 2E program behaves like the one it replaces: two data copies, the real entry point, and SQL that shows every difference

---

Every CA 2E conversion estimate I've seen has a column that says "testing," and in most of them the number is blank. Ours said "TBD" after the conversion and compile-fix columns had real hours in them.

That blank is the biggest risk in the whole project. A clean compile proves the generated source is valid RPG. It doesn't prove the program writes the same rows, updates the same fields or rejects the same bad input as the COBOL it replaces. Article 10 is about a converted program that ran clean and still wrote nothing. That's the dramatic version. The quiet version is a program that writes almost the same thing.

This post is the method I'd use to fill that column: run both versions against identical copies of the data, through the same entry point, and let SQL report every difference.

---

Pick programs you can test, not programs that are easy to convert

The single most useful rule from our pilots was about selection. After the first program, we picked the next one because it was easy to test, and we knew how to get its data. Testing effort varies far more between programs than conversion effort does, so this choice decides how believable your first numbers are.

A good first program to test:

• Is batch, not interactive. Its inputs are parameters and files, and its outputs are files. Screen programs come later, once the method works.
• Has inputs you can reproduce. You know which parameter values produce a normal run, and where to find real data that exercises them.
• Writes to files you can name. You can list every file it creates, updates or deletes rows in. If you can't name them, you can't compare them.

Our first pilot was chosen for no particular reason, which is common. Afterwards, the engineer who converted it thought a screen program might have been the better pick. That instinct, test what users actually touch, is right eventually. It's the wrong place to start, because a screen program's cost lies in the testing, and you don't yet have a way to test.

---

Find every file the program touches

"Writes to files you can name" is easier to say than to check by reading generated code. Let the system tell you. DSPPGMREF lists every object a program references, and it can write the list to a file you can query:

DSPPGMREF PGM(ORIGPGMLIB/PGMNAME) OUTPUT(*OUTFILE) +
          OUTFILE(QTEMP/PGMREFS)
DSPPGMREF PGM(ORIGPGMLIB/WRAPPERCL) OUTPUT(*OUTFILE) +
          OUTFILE(QTEMP/PGMREFS) OUTMBR(*FIRST *ADD)

Run it on the wrapper too, with *ADD, because the wrapper often opens or overrides files the program itself never names. Then put the list in the order you'll need it:

SELECT DISTINCT
       r.whfnam     AS file_name,
       t.table_type AS file_type,      -- P / T = physical, L = logical
       r.whfusg     AS usage           -- how the program opens it
  FROM qtemp.pgmrefs r
  LEFT JOIN qsys2.systables t
         ON t.system_table_schema = 'DATALIB'
        AND t.system_table_name   = r.whfnam
 WHERE r.whotyp = '*FILE'
 ORDER BY CASE WHEN t.table_type IN ('P', 'T') THEN 0 ELSE 1 END,
          r.whfnam;

The usage column separates files the program only reads from files it writes or updates. The writes are what you'll compare. The reads go on the "should not change" list further down.

Physical files come first in that list for a reason. If you duplicate a logical file into a test library before its physical file is there, the copy can still point at the physical file in the original library, and your "test" run quietly updates real data. Copy the physicals first, or copy the whole set in one CRTDUPOBJ OBJ(*ALL), which handles the order for you.

---

Two copies of the same data

Running the old and new program against the same live files proves nothing: the second run starts from the first run's results. You need two copies that start out identical:

CRTDUPOBJ OBJ(*ALL) FROMLIB(DATALIB) OBJTYPE(*FILE) TOLIB(TESTA) DATA(*YES)
CRTDUPOBJ OBJ(*ALL) FROMLIB(DATALIB) OBJTYPE(*FILE) TOLIB(TESTB) DATA(*YES)

TESTA is for the original program, TESTB for the converted one. For a big library, copy only the files the program touches plus everything those files depend on. The list of files you built to choose the program is the list you copy here.

Two practical notes:

• Copy once, run both. Take both copies from the same source in the same window. A copy taken an hour later can differ because production kept running.
• Reset by copying again. After every test case, delete and re-copy rather than trying to undo the run. Undoing writes by hand is how test data gets quietly corrupted.

---

Run through the real entry point

Call each version the way production calls it: through its CL wrapper or the menu option's command, with the same parameters, and with the library list arranged so the only difference is which program library comes first.

/* Original */
CHGLIBL LIBL(TESTA ORIGPGMLIB COMMONLIB)
CALL PGM(WRAPPERCL) PARM('CODE1' 'CODE2' 'VALUE1')

/* Converted */
CHGLIBL LIBL(TESTB CONVPGMLIB COMMONLIB)
CALL PGM(WRAPPERCL) PARM('CODE1' 'CODE2' 'VALUE1')

Calling the generated program directly with hand-typed parameters is tempting, and it's how our first converted program was tried. It isn't the same test. The wrapper sets up overrides, commitment control and parameter formats the program depends on. Article 10 walks through what goes wrong when you skip it. If both versions run through the same wrapper, any difference in their results comes from the programs themselves.

Write each test case down as you go: the parameters, which data it exercises, and what a correct run should do. That list becomes the regression suite for every later program that touches the same files.

---

Let SQL find the differences

For each file the program writes to, compare the two copies in both directions:

SELECT 'ORIGINAL ONLY' AS side, d.*
  FROM (SELECT * FROM testa.outfile
        EXCEPT
        SELECT * FROM testb.outfile) d
UNION ALL
SELECT 'CONVERTED ONLY' AS side, d.*
  FROM (SELECT * FROM testb.outfile
        EXCEPT
        SELECT * FROM testa.outfile) d
ORDER BY 2, 3;

An empty result means the two files ended with the same rows. Anything that comes back is a difference, with a label saying which side has it.

Three refinements make this usable on real files:

Exclude columns that are supposed to differ. Timestamps, job numbers, user IDs and generated sequence numbers will never match between two runs. List the columns explicitly instead of SELECT *, and leave those out. Write down which ones you excluded and why, because it's part of the test.

Check counts too. EXCEPT compares distinct rows, so a program that writes a row twice can look identical to one that writes it once. Add a row count per file for both libraries next to the comparison.

Compare what shouldn't change. Run the same comparison on files the program only reads. If one version changed a file it was never supposed to touch, that's the most important difference you'll find.

---

Doing it for every file, every case

Typing that query per file gets old by the third test case. This small SQLRPGLE program does one file at a time. It builds the column list from the catalog, leaves out the columns you tell it to skip, counts rows on both sides and the differences in each direction, and writes one result row:

**free
// CMPFILE - compare one file between TESTA (original run)
//           and TESTB (converted run), log the result
ctl-opt dftactgrp(*no) actgrp(*new) option(*srcstmt);

dcl-pi *n;
  pFile char(10) const;          // file to compare
  pSkip char(200) const;         // columns to ignore: 'CHGTS,CHGJOB,CHGUSR'
end-pi;

dcl-s file   varchar(10);
dcl-s skip   varchar(202);
dcl-s cols   varchar(8000);
dcl-s rowsA  int(10);
dcl-s rowsB  int(10);
dcl-s onlyA  int(10);
dcl-s onlyB  int(10);

file = %trim(pFile);
skip = ',' + %scanrpl(' ' : '' : %trim(pSkip)) + ',';

// Column list from the catalog, in file order, minus the skipped ones
exec sql
  select listagg(cast(system_column_name as varchar(10)), ', ')
           within group (order by ordinal_position)
    into :cols
    from qsys2.syscolumns
   where system_table_schema = 'TESTA'
     and system_table_name   = :file
     and locate(',' concat trim(system_column_name) concat ',', :skip) = 0;

if sqlcode <> 0 or cols = '';
  dsply ('CMPFILE: no columns found for ' + file);
  *inlr = *on;
  return;
endif;

rowsA = countOf('select count(*) from testa.' + file);
rowsB = countOf('select count(*) from testb.' + file);
onlyA = countOf('select count(*) from (select ' + cols + ' from testa.' + file
              + ' except select ' + cols + ' from testb.' + file + ') x');
onlyB = countOf('select count(*) from (select ' + cols + ' from testb.' + file
              + ' except select ' + cols + ' from testa.' + file + ') x');

exec sql
  insert into convlib.cmp_result
         (file_name, rows_orig, rows_conv, only_orig, only_conv, skipped_cols)
  values (:file, :rowsA, :rowsB, :onlyA, :onlyB, :pSkip);

*inlr = *on;
return;

// Run a SELECT COUNT(*) built at run time and return the number
dcl-proc countOf;
  dcl-pi *n int(10);
    sqlText varchar(10000) const;
  end-pi;
  dcl-s stmt varchar(10000);
  dcl-s n    int(10) inz(-1);

  stmt = sqlText;
  exec sql prepare cntStmt from :stmt;
  exec sql declare cntCur cursor for cntStmt;
  exec sql open cntCur;
  exec sql fetch cntCur into :n;
  exec sql close cntCur;
  return n;
end-proc;

A few choices in there are deliberate:

• The column list comes from QSYS2.SYSCOLUMNS, so nobody types forty field names per file, and a field added to the file later is compared automatically.
• The skip list is padded with commas on both ends before the LOCATE, so skipping CHGTS doesn't also skip a column called CHGTSX.
• Counts and differences are stored separately. A file where only_orig and only_conv are both zero but rows_orig and rows_conv differ has a duplicate-row problem that EXCEPT alone would hide.
• countOf returns -1 if anything fails, so a broken statement shows up as an impossible count instead of a reassuring zero.

LISTAGG needs a reasonably current release (7.3 or later). The result table is plain:

CREATE TABLE convlib.cmp_result (
  run_ts        TIMESTAMP     NOT NULL DEFAULT CURRENT TIMESTAMP,
  file_name     VARCHAR(10)   NOT NULL,
  rows_orig     INTEGER,
  rows_conv     INTEGER,
  only_orig     INTEGER,
  only_conv     INTEGER,
  skipped_cols  VARCHAR(200)
);

And a CL program drives it for each file in a test case:

PGM
  DCL VAR(&SKIP) TYPE(*CHAR) LEN(200) +
        VALUE('CHGTS,CHGJOB,CHGUSR')

  CALL PGM(CONVLIB/CMPFILE) PARM('OUTFILE1' &SKIP)
  CALL PGM(CONVLIB/CMPFILE) PARM('OUTFILE2' &SKIP)
  CALL PGM(CONVLIB/CMPFILE) PARM('READFILE1' &SKIP)  /* should not change */
ENDPGM

Note &SKIP is a declared 200-byte variable, not a literal. CL passes a character literal shorter than 32 bytes padded to only 32, so a program expecting 200 bytes would read whatever happens to sit in memory after that. It's a classic IBM i trap, and in a test harness it shows up as random columns being skipped.

After each case, one query gives the verdict:

SELECT file_name, rows_orig, rows_conv, only_orig, only_conv,
       CASE WHEN only_orig = 0 AND only_conv = 0
             AND rows_orig = rows_conv THEN 'SAME'
            ELSE 'DIFFERENT'
       END AS result
  FROM convlib.cmp_result
 WHERE run_ts > CURRENT TIMESTAMP - 1 HOUR
 ORDER BY result, file_name;

When a file comes back DIFFERENT, go back to the hand-written EXCEPT query above for that one file. It shows you the actual rows.

---

Cases worth running

Normal runs prove the common path. Conversions usually break on the uncommon ones:

• Not found. A key that doesn't exist. Generated code leans heavily on indicators after a keyed read, and a key list built differently in the conversion shows up here first.
• More than one match. A key that matches several records, where the order they're read in matters.
• Boundary values. Zero quantities, maximum field lengths, blank codes, the first and last day of a period.
• The error path. Input the original rejects. The converted version should reject it the same way and leave the files the same way.
• Anything you changed by hand. Every entry in the change ledger from the previous post gets a case. A field reference you removed because "the value comes from another file" is exactly the kind of fix to prove with data.

---

Screen programs, later

Interactive programs are harder because their inputs are keystrokes. Two approaches work, in increasing order of effort:

1. Compare file effects only. Script the same sequence of actions by hand in both versions against the two data copies, then run the same SQL comparison. It doesn't check what the screen shows, but it checks what the program did, which is where most conversion bugs land.
2. Record and replay. Use an emulator macro or a 5250 automation tool to play identical keystrokes into both versions, and capture the screens for comparison. It's worth setting up only once there are enough screen programs to reuse it on.

Either way, by the time you get to screens you should already have a working batch method, a set of data copies you trust and a comparison query you can reuse. That's why they come second.

---

Filling in the blank column

Do this on the pilots and the "testing" column stops being a guess. You'll know how long setting up data copies took, how many cases a program of that size needed, and how long it took to explain each difference the SQL found. Put those hours in the estimate next to conversion and compile fixes, as the inventory post recommends, and the estimate starts describing the real job.

A clean compile says the program is valid. A clean comparison says it's the same program. Only the second one is a reason to switch.

---

Jaya Krushna Mohapatra is a Warehouse Management Systems Architect focused on enterprise integrations, IBM i modernization, and scalable backend systems.