KVCODERS: The subtle bug in comparing Python output as text

kvcoders — Thu, 24 Sep 2026 17:08:35 +0000

KVCoders has an "Output Practice" mode where students see a Python (or SQL) snippet and have to predict its exact printed output — a real, common CBSE Class 11–12 Computer Science exam question type. Grading it sounds trivial: run the reference code once, store the expected output, compare it to what the student typed. It isn't.

The first version did exactly that — a plain string comparison — and it produced false negatives constantly. A student who correctly predicted [1, 2, 3] but typed [1,2,3] (no space after the commas) was marked wrong, even though Python's own print() would show the spaced version and the student clearly understood the actual value. The same problem showed up around dictionary colons ({'a':1} vs {'a': 1}) and generally anywhere CBSE's own textbook examples are inconsistent about spacing that Python itself normalizes on output.

The fix isn't "trim whitespace" — that's too blunt, because whitespace is sometimes the actual answer. A question testing string formatting might have "R*A*D*A*R*" as a genuinely correct output where every character matters, and a multi-line print output's internal line breaks are meaningful, not incidental. So the real requirement was narrower: normalize spacing specifically around Python's own container punctuation (, and : inside [...], (...), {...}), and leave everything else — including anything inside a quoted string literal, at any nesting depth — completely untouched.

KVCoders' actual normalizer (normalize_output_answer() in examiner/includes/output_scoring.php) is a small character-by-character parser that tracks two pieces of state as it walks the string: bracket depth (how many [/(/{ are currently open) and whether it's currently inside a quoted string (and if so, which quote character, so it can handle an escaped quote correctly and not exit early). Outside of quotes, at any depth greater than zero, a comma or colon gets its trailing whitespace collapsed to exactly one space (matching how Python itself prints a list or dict) — but only if what follows isn't a closing bracket, so [1, 2] normalizes consistently without producing a stray trailing space before the ]. Inside quotes, every character — including whitespace — passes through byte-for-byte, so string content can never be silently rewritten by a grading rule.

This function is called from three separate places — the web submit endpoint, the Android app's API submit endpoint, and (identically) the client-side preview — specifically so none of them can quietly drift into disagreeing about what counts as a correct answer, which used to be a real, separate bug class before it was consolidated into one shared implementation.

The broader lesson wasn't really about string parsing. It's that "auto-graded" for a board-exam audience has to tolerate exactly the kind of formatting variance a human teacher would silently forgive — without becoming so lenient that it stops catching genuinely wrong answers. Getting that boundary right mattered more than the parsing logic itself.

This runs in production at kvcoders.in for CBSE Class 11–12 CS/IP output-prediction practice. Happy to answer questions in the comments.

DEV Community: kvcoders

KVCODERS: The subtle bug in comparing Python output as text