I Built an Automated Insurance Claims Processor on AWS. Terraform Broke It in an Interesting Way.

Derek Jackson — Tue, 01 Sep 2026 18:08:31 +0000

I built an automated insurance claims document processor on AWS. A claim PDF is uploaded and you get back a clean, structured database record with no manual data entry. Then I rebuilt the whole system in Terraform. The rebuild caused tricky bugs at several stages. This post walks through those bugs in detail. Each bug followed the same pattern: everything looks fine, nothing works, and no error appears anywhere.

Why this project

As someone with a background in insurance operations, I wanted at least one of my AWS portfolio projects to reflect that. I chose a build that bridges my operational expertise with Fintech, creating a natural translation of my real-world domain knowledge into the cloud ecosystem.

The pipeline processes a Long-Term Disability (LTD) claim form. This form is modeled on the structure of a real LTD claim and extracts an allowlist of the 14 fields necessary for claims processing: policy number, date last worked, coverage start date, etc.

I decided to build it twice. First manually through the AWS Console, to visualize each service and their relationship with one another. Then I rebuilt the entire thing as IaC using Terraform, so the infrastructure is fully reproducible. I split the code into modules for compute, database ,iam, messaging and storage, and used the rebuild to add an SQS queue and DLQ that the manual build did not have. That second pass is where things got interesting.

Architecture

A claim form is uploaded s3://bucket/incoming/. The upload triggers Lambda #1. Lambda #1 starts an asynchronous Textract job and returns immediately.

Why asynchronous? This is a multi-page document. Textract's synchronous API only handles single-page documents and it's bound by Lambda's execution timeout. So an asynchronous job was the only option.

Textract reads the S3 object. When the job finishes, Textract publishes a completion message to an SNS topic. SNS delivers this message to an SQS queue. A dead-letter queue (DLQ) sits behind this SQS queue to catch failed messages.

The SQS layer adds automatic retries (up to 3 attempts), and a safety net for messages that fail permanently. Without the DLQ, an unread form would be lost.

Once the message is delivered to SQS, it triggers Lambda #2. Lambda #2 retrieves the full Textract results, filters the results down to the 14-field allowlist and writes an item to DynamoDB.

Rebuilding in Terraform: three hiccups

The Console-built version worked without any of the problems below. Each problem I encountered was a result of the additions I made in the Terraform version.

Problems and Fixes

1. S3 bucket name collision
Although my original bucket was destroyed during the teardown, the S3 namespace can take time to release a deleted name. This triggering a temporary collision:

**Error**: creating S3 Bucket (derek-jackson-claims-processor-2026): BucketAlreadyExists

  with module.storage.aws_s3_bucket.claims,
  on modules/storage/main.tf line 1, in resource "aws_s3_bucket" "claims":
   1: resource "aws_s3_bucket" "claims" {

Fix: I removed the hardcoded bucket name and generated a random suffix with Terraform's random_id resource instead. A small tweak, but a permanent fix for any future rebuild.

2. A three-stage KMS permission chain
The console version used AWS's default encryption everywhere. When I did the Terraform build, I decided to use a Customer Managed Key (CMK) — rather than AWS's default managed keys — for S3, DynamoDB, SNS, and SQS, in order to control exactly which services and roles could use the key. Compliance-driven environments commonly require CMKs for this reason. Although I used a single key for simplicity, in a live environment, the principle of least privilege would dictate using isolated keys per resource to minimize the blast radius.

The lesson here: a KMS key's own policy is a separate trust boundary from IAM, which wasn't obvious to me until I hit it. You can grant a Lambda's IAM role kms:Decrypt on the key. But that grant does not let another AWS service, acting on your behalf, use that key. The key's own policy must name that service directly because, by default, a CMK only trusts standard users and roles within your account and automatically blocks automated AWS services.

The pipeline touches three separate points where a service other than a Lambda role needs to work with KMS-encrypted data, and each failure was completely silent; nothing threw a visible error, things just stopped working one step further down the chain:

Textract → S3 input (read)

The bug: Textract's async jobs run in Textract's own backend using it's own service identity, not the Lambda's temporary credentials.
The fix: I granted textract.amazonaws.com the kms:Decrypt permission directly in the CMK's key policy.

Textract → SNS topic (publish)

The bug: I confirmed via the CLI that the Textract job itself was succeeded — JobStatus: SUCCEEDED — but nothing was reaching SQS. The dedicated textract-sns-role had sns:Publish permission but no KMS permission.
The fix: I added kms:GenerateDataKey and kms:Decrypt to textract-sns-role's IAM policy.

SNS → SQS queue (delivery)

The bug: this is a documented AWS architectural requirement, not something I found by trial and error. The SNS service principal itself needs an explicit KMS grant to deliver messages into an encrypted SQS queue.
The fix: I granted sns.amazonaws.com the same two KMS actions in the key's policy. After applying this fix, a message finally reached the dead-letter queue! Showing progress, and confirming the full Textract → SNS → SQS chain was finally delivering messages. It landed in the DLQ rather than succeeding outright because Lambda #2 still had a separate bug which I addressed next.

The lesson here: whenever Service A hands data to Service B, and B's resource is encrypted with a customer managed key, Service A needs its own grant on that key. Being able to call the API (sns:Publish, sqs:SendMessage) and being able to use the KMS key the resource is encrypted with are separate permissions.

3. SQS vs. SNS event shape
With the KMS chain resolved, a message finally reached SQS, but Lambda #2 was still failing. Fortunately, now with a visible error:

[ERROR] KeyError: 'Sns'
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 120, in lambda_handler
    message = json.loads(event["Records"][0]["Sns"]["Message"])

Lambda #2's code assumed a direct SNS trigger. But this architecture routes through SQS and since the SNS→SQS subscription doesn't have raw message delivery enabled, SQS wraps the entire SNS envelope as a JSON string inside its own body field leaving two layers to unwrap, not one:

sns_envelope = json.loads(event["Records"][0]["body"])
message = json.loads(sns_envelope["Message"])

This is a good example of an implicit contract bug: two pieces of code each made a reasonable assumption independently, and those assumptions quietly contradicted each other.

Debugging method

I verified each hop in the chain on its own, starting closest to the source:

Queried Textract's job status directly via the CLI. This step bypasses the entire pipeline, and confirmed the Textract job itself succeeded, ruling that stage out and pointing to a problem in the message delivery.
Checked SQS and DLQ message counts. This step confirms whether a message ever reached SQS. It showed no message arriving. This pointed to a delivery failure between SNS and SQS, caused by the missing KMS grants.
After adding the missing grants, I checked CloudWatch logs which surfaced the KeyError: 'Sns' error. This error pointed to the SQS/SNS event-shape mismatch in Lambda #2.

4.Fixed the event-shape mismatch by unwrapping the SQS body before the SNS message inside it. This change let Lambda #2 correctly parse the message. It processed the claim data without error. The pipeline then ran end to end, from upload to a clean DynamoDB record, with no further issues.

Proof it works
The original console-built pipeline, tested against a real-structured LTD claim form:

The rebuilt Terraform pipeline, tested end-to-end against a second mock document with intentionally obvious fake data:

DynamoDB item from the Terraform rebuild, part 1 — claim_id, city: Sampleville, coverage_start_date, date_last_worked, date_of_birth, earnings, employer_contact_name, employer_name: Acme Test Corp

DynamoDB item from the Terraform rebuild, part 2 — full_name: John Q. Testperson, job_title, policy_number: TEST-000000, social_security_number: 123-45-6789, state, street_address, zip_code: 00000

And the dead-letter queue moment that confirmed the KMS chain was finally resolved — the first message to make it all the way from Textract through SNS to SQS:

SQS console showing the callback queue at 0 messages and the dead-letter queue with 1 message available

What I'd do next time: AWS Lambda durable functions

AWS released Lambda durable functions in December 2025. This feature fits the "start a task, wait for a callback" pattern used in this project.

Using durable functions would merge Lambda #1 and Lambda #2 into a single function which would use a waitForCallback() operation. This change would remove the SQS/DLQ layer, because durable functions would handle retry and durability themselves.

A thin relay lambda would still be necessary since Textract can't call the durable execution callback API directly. The relay function would translate the SNS notification into that call.

The tradeoff with this approach is the SQS/DLQ layer is no longer needed, which also removes the need for the SNS to SQS KMS grant. This simplified infrastructure would only require a relay function rather than a queue-based messaging layer with its own encryption requirements.

Code and Terraform modules for this project are on my GitHub: https://github.com/DIJSAA/claims-processor-terraform

DEV Community: Derek Jackson

I Built an Automated Insurance Claims Processor on AWS. Terraform Broke It in an Interesting Way.