Debugging an AWS SAM Local 502 That Wasn't a Spring Boot Bug

The local API started cleanly.

Running on http://127.0.0.1:3001
Press CTRL+C to quit

Then the dashboard called /jobs, and every request came back as 502.

At first glance, that looks like the application is broken. Maybe the controller failed. Maybe the database configuration is wrong. Maybe the Lambda bridge is not passing the request into Spring Boot correctly.

But the stack trace told a different story. The request was not failing inside the /jobs endpoint. It was failing inside AWS SAM CLI before the application had a clean chance to handle it:

TypeError: unhashable type: 'collections.OrderedDict'

That changed the debugging question. The problem was no longer “why is my API endpoint failing?” It became “why is local infrastructure handing SAM a value it cannot use as a function name?”

The useful clue was buried in the middle of the Python stack trace:

File ".../samcli/commands/local/lib/local_lambda.py", line 393, in _make_env_vars
  or self.env_vars_values.get(function_name, None)
TypeError: unhashable type: 'collections.OrderedDict'

That line matters because dict.get() expects a hashable key. A normal Lambda function name would be a string, and a string is hashable. SAM was not holding a string there. It was holding a parsed CloudFormation object.

That ruled out the first few obvious suspects. The database was not the immediate failure. The Spring controller was not the immediate failure. The Java handler was not throwing the exception shown in the terminal. The failure happened while SAM local was trying to assemble the Lambda invocation environment.

The source of that object was the SAM template.

The deployed template allowed the Lambda function name to be optional:

FunctionName:
  Fn::If:
    - HasLambdaFunctionName
    - Ref: LambdaFunctionName
    - Ref: AWS::NoValue

That is a reasonable deployment pattern. In CloudFormation, Fn::If can decide whether to use a fixed Lambda function name or let AWS generate one.

But local execution is a different environment. In this case, SAM CLI did not resolve that conditional into a plain string before it built the local invocation config. So by the time /jobs was called, function_name was not something like ApiFunction. It was still the parsed representation of the CloudFormation condition.

That is how a valid deployment template became an invalid local runtime input.

The first fix looked obvious: pass a concrete function name into SAM local.

--parameter-overrides "LambdaFunctionName=rcyc-eil-sync-api-local ..."

That was not a bad guess. If the conditional FunctionName was the problem, then giving the parameter a real value should have given SAM something concrete to resolve.

But the error still appeared.

That failure was useful. It showed that the issue was not only the missing value of LambdaFunctionName. The built SAM template still carried the conditional Fn::If shape, and SAM local was still walking through that shape during local invocation.

In other words, the fix could not be only “supply a better parameter.” The local runner needed a simpler template.

The fix was to separate the deployment template from the local runtime template.

The production template.yaml can keep the AWS-specific behavior: optional function names, aliases, and provisioned concurrency. Those belong in the deploy path.

For local development, I added a smaller template.local.yaml:

Resources:
  ApiFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: .aws-sam/build/ApiFunction
      Handler: com.rcyc.eil.syncapi.lambda.StreamLambdaHandler::handleRequest
      MemorySize:
        Ref: LambdaMemorySize

The important part is what is missing.

There is no conditional FunctionName. There is no AutoPublishAlias. There is no ProvisionedConcurrencyConfig. The local template points directly at the built Lambda artifact and gives SAM local a shape it can execute without interpreting deploy-only CloudFormation.

Then the startup script was changed to use the local template by default:

SAM_TEMPLATE_FILE="${SAM_TEMPLATE_FILE:-template.local.yaml}"

sam local start-api \
  --template "${SAM_TEMPLATE_FILE}" \
  --port "${PORT}" \
  --warm-containers "${WARM_CONTAINERS_MODE}" \
  --parameter-overrides "${parameter_overrides[@]}"

That removed the OrderedDict crash path because SAM local no longer had to treat a conditional CloudFormation object as the Lambda function name.

I kept warm containers enabled.

That detail matters. The easy escape would have been to remove --warm-containers and accept slower local requests. But this API backs a job monitoring dashboard. During development, the dashboard repeatedly calls /jobs, and waiting through a Java/Spring Lambda cold start on each request makes the UI feel broken even when the backend is correct.

So the goal was not only “make the exception disappear.” The goal was “make local invocation stable while keeping repeated dashboard requests fast.”

With the local template simplified, the startup script could keep eager warming:

WARM_CONTAINERS_MODE="${WARM_CONTAINERS_MODE:-EAGER}"

That kept the developer experience aligned with the product requirement: a monitoring UI should feel responsive while engineers are testing it.

I verified the fix in layers.

First, the startup script still had to be valid shell:

bash -n scripts/start-local-api.sh

Then the local template had to be a valid SAM template:

sam validate -t template.local.yaml

The important check was sam local invoke with debug logging:

sam local invoke ApiFunction -t template.local.yaml --event /dev/null --debug

In my constrained validation environment, Docker access was blocked, so the Lambda could not fully start there. But the failure moved to the expected place: container runtime setup. SAM no longer crashed while building environment variables, and the debug output showed the function resolving as a plain ApiFunction with CodeUri='.aws-sam/build/ApiFunction'.

That was the evidence I needed. The old failure happened before Docker. The new failure happened at Docker access. The OrderedDict path was gone.

If you hit this error, I would check these things first:

• Look for FunctionName using Fn::If, Ref, or another CloudFormation expression.
• Check whether sam local is using .aws-sam/build/template.yaml instead of the source template you think it is using.
• Run with --debug and see whether the function resolves to a plain string or a parsed object.
• Try a local-only template that removes deploy-only properties like AutoPublishAlias, ProvisionedConcurrencyConfig, and conditional function names.
• Keep performance settings like --warm-containers EAGER only after the local template shape is simple enough for SAM to execute reliably.

The checklist is short because the bug is not really about /jobs, Java, or Spring Boot. It is about the value SAM local uses as the Lambda function name.

The final local flow looked like this:

sam build
./scripts/start-local-api.sh 3001

Then test the endpoint:

curl -u 'rcyc-api:your-password' \
  'http://127.0.0.1:3001/jobs?page=0&size=10&sortBy=jobId&sortDir=DESC'

If startup needs to be debugged without eager container initialization, the script still allows that:

WARM_CONTAINERS_MODE=LAZY ./scripts/start-local-api.sh 3001

The main lesson for me was not specific to AWS SAM.

When a request fails before your application code gets control, the endpoint is only where the symptom appears. The fix belongs at the boundary that created the bad runtime value.

In this case, /jobs was not the problem. Spring Boot was not the problem. The problem was that a deployment-friendly CloudFormation expression leaked into a local runtime path that expected a plain function name.

Once that boundary was split into a deploy template and a local template, the error became easy to reason about and the local API could stay fast.