Regression test coverage analysis in TCL

#tcl #tdd #testcoverage

If you’re like me, you like having lots of regression tests to keep you covered from making stupid mistakes when hacking up some complicated piece of code. Whilst code coverage tools exist for the major development enviroments, one major blind spot is how do do coverage analysis of TCL, which becomes a problem when your application (eg Minsky) starts to sport a significant amount of application logic in TCL.

A quick Google indicated that you could buy into the Active TCL way of doing things (not so useful for me), or use and application called Nagelfar. Unfortunately, Nagelfar really assumes you are coding in a standard environment, such as wish or tclsh, not in an application scripting environment such as Minsky or EcoLab. Then the realisation I could do it fairly simply in TCL. Well I did have a few false turns which took its time, but I found I could attach a command to fire on every step executed in TCL. Then I peek into the immediately enclosing stack frame to look at details such as which line I’m executing, and save these to a database. Since I’m doing this in the EcoLab environment, I make use of the cachedDBM class to accumulate executaion counts as their found. Finally, I write a C++ program that reads in a TCL file, identifies which proc I’m in and checks whether an entry for the proc, or for the file line number is in the database, and produces output not unlike gcov, with ### indicating a line that wasn’t executed.

The C++ code is called tcl-cov, and is currently located in Minsky’s test directory, although I’m considering moving it to the ecolab utilities directory.

The TCL code to be added to the main application? Here is is:

proc attachTraceProc {namesp} {
    foreach p [info commands $namesp*] {
        if {$p ne "::traceProc"} {
            trace add execution $p enterstep traceProc
        }
    }
    # recursively process child namespaces
    foreach n [namespace children $namesp] {
        attachTraceProc ${n}::
    }
}`

# check whether coverage analysis is required  
if [info exists env(MINSKY\_COV)] {  
# trace add execution proc leave enableTraceProc  
 proc traceProc {args} {  
 array set frameInfo [info frame -2]  
 if {$frameInfo(type)=="proc"} {  
 minsky.cov.add $frameInfo(proc) $frameInfo(line)  
 }  
 if {$frameInfo(type)=="source"} {  
 minsky.cov.add $frameInfo(file) $frameInfo(line)  
 }  
 }  
# open coverage database, and set cache size  
 minsky.cov.init $env(MINSKY\_COV) w  
 minsky.cov.max\_elem 10000  
# attach trace execuation to all created procs  
 attachTraceProc ::  
}

The name of the coverage database is passed in via the MINSKY_COV environment variable. minsky.cov.add is a command for adding 1 to the counter for file/line, or proc/line as appropriate. The traceProc command is attached to all defined procs, which requires walking through all namespaces, hence the recursive call into attachTraceProc (which starts in global namespace ::).

That’s it! Enjoy.