The Generated Helper Passed Locally, Then Died in a Clean Container Because It Read a Global I Never Passed

I almost shipped a small feature-flag helper that worked on my laptop and crashed in the clean Docker image. The generated source looked self-contained, and my local smoke test gave the exact boolean I expected, so I assumed the problem was the deployment environment. It was not.

The generated function was silently reading module-level variables that my interactive shell still had from an older script. The clean container did not have those globals, so the same code raised NameError at the first call. I rebuilt the failure in a tiny harness and then added a pre-execution check that catches this class of problem before the code leaves the test directory.

The generated code looked self-contained

The helper was supposed to tell me whether a feature flag was enabled for a given name. The generated source was short enough to trust:

generated_source = '''
def is_enabled(feature):
    if DEBUG:
        return feature in FEATURE_STORE
    return feature in DEFAULT_FEATURES
'''

The function has three names it expects to find somewhere else: DEBUG, FEATURE_STORE, and DEFAULT_FEATURES. None of them is a parameter, a return value, or an import. They are free globals, but the source does not say that, and a quick glance at the function body can miss it.

Why my laptop passed while the container failed

My local Python session still had the leftovers from a previous experiment. I executed the generated snippet in a namespace that happened to contain exactly the names it needed, so every branch ran without incident:

globals_here = {
    'DEBUG': False,
    'FEATURE_STORE': {'dark': False},
    'DEFAULT_FEATURES': {'dark', 'beta'},
}

exec(generated_source, globals_here)
print(globals_here['is_enabled']('dark'))

That printed False, which matched my expectation. The same code, however, behaved very differently in an empty namespace that simulated a clean container:

clean_ns = {}
exec(generated_source, clean_ns)

try:
    print(clean_ns['is_enabled']('dark'))
except NameError as exc:
    print(f"clean execution failed: {exc}")

The output made the hidden dependency visible:

False
clean execution failed: name 'DEBUG' is not defined

Only the second run told the truth. The first run passed because my environment had already provided the missing pieces, not because the generated code was complete.

Reproduce the silent global capture

I first captured this bug while generating a small helper with MonkeyCode's free model access, then I re-ran the clean-namespace check using its free server option so the reproduction would not depend on whatever globals happened to be in my shell. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The problem is provider-neutral, so the fix belongs in the code path that turns model output into an artifact for evaluation. A model can produce clean syntax and still create a function that depends on state the caller never declared.

The most useful check is to find every name the generated code loads but neither assigns nor receives as a parameter. I use an AST walk for that, because a regex that scans for identifiers is too easy to fool with nested functions, comprehensions, and string annotations.

Find free globals before execution

This small analyzer collects locally bound names, then subtracts them from all loaded names:

import ast, builtins

def free_globals(source):
    tree = ast.parse(source)
    local_names = set(dir(builtins))

    for node in ast.walk(tree):
        if isinstance(node, ast.Name) and isinstance(node.ctx, ast.Store):
            local_names.add(node.id)
        elif isinstance(node, ast.arg):
            local_names.add(node.arg)
        elif isinstance(node, (ast.FunctionDef, ast.ClassDef)):
            local_names.add(node.name)
        elif isinstance(node, ast.Import):
            for alias in node.names:
                local_names.add(alias.asname or alias.name.split('.')[0])
        elif isinstance(node, ast.ImportFrom):
            for alias in node.names:
                local_names.add(alias.asname or alias.name)

    referenced = set()
    for node in ast.walk(tree):
        if isinstance(node, ast.Name) and isinstance(node.ctx, ast.Load):
            referenced.add(node.id)

    return sorted(referenced - local_names)

print(free_globals(generated_source))

Running that script prints:

['DEFAULT_FEATURES', 'DEBUG', 'FEATURE_STORE']

The list is the missing contract. Whenever free_globals returns a non-empty list, the generated artifact is not self-contained and should not be treated as a finished unit until the caller either passes those names explicitly or marks them as an intentional runtime dependency.

Run generated code with an empty namespace

The analyzer tells you what is missing, but it does not prove the function works. A second guard is to execute the generated source in a namespace that contains only builtins and the values you deliberately inject. That is roughly what the earlier clean_ns block did, and it is cheap enough to run before every merge.

I keep these two checks in separate stages because they catch different mistakes:

1. Static check: walk the AST and fail fast when free globals exist.
2. Runtime check: execute with an empty namespace and assert that the result matches a few expected values.

If the generated function expects a global by design, I pass it through an allowlist instead of letting the local shell supply it accidentally. That way the environment is as deliberate as the generated code itself.

When a missing global is a real feature

Not every free variable is a bug. A generated data-analysis function may be allowed to rely on pandas or numpy as an injected dependency, and a generated SQL helper may be allowed to read a connection object from an application context.

The rule I follow is not to ban globals outright. It is to make them visible:

• Add the name to an explicit allowlist before execution.
• Pass the dependency through a parameter or a documented context object when possible.
• Never let a local shell or a previous script provide the missing name by accident.
• Fail the evaluation if a free global appears outside the allowlist.

That distinction is what prevents a function from being portable on my laptop and broken everywhere else.

What this check does not catch

A clean namespace run can still produce a result that is logically wrong even when the function is self-contained. If the generated code returns True for a feature that should be disabled, the AST check will not notice, and a shallow smoke test may miss it. This approach only removes the false confidence that comes from an ambient global accidentally completing the program.

The static analyzer also needs to be updated when the target language changes. The code above is for Python; a JavaScript or TypeScript pipeline needs a different walk, and a shell-script generator needs its own parser. The principle stays the same: before execution, make the undeclared external references visible.

Before the next generated helper gets merged, run it in a namespace that has nothing except builtins. The missing names will tell you more than your laptop's green check ever could.