DEV Community

Cover image for Shipping a vision-model verdict on Bedrock and Lightsail
AWS Community Builders profile image xbill
xbill for AWS Community Builders

Posted on

Shipping a vision-model verdict on Bedrock and Lightsail

#aws #ai #python #showdev

Built 2026-08-15 against us.amazon.nova-lite-v1:0 via the Bedrock Converse API. FastAPI on Python 3.13, deployed to an Amazon Lightsail container service (nano, scale 1) in us-east-1. Scored against the live deployment, not localhost: 20/20 on the fixture set, median 880 ms per scan.

Live: Dog or Not: Lite · Source: github.com/xbill9/dog-or-not-lite · Built for the AWS Weekend Challenge: Build a Creative App.

TL;DR

Make the model fill in a schema instead of writing a sentence. The Converse API's toolConfig plus toolChoice forces a named function call, so is_dog arrives as a boolean because it was declared as one. Every image comes back in the same shape — including the ambiguous ones, which is exactly where free-text output gets creative and a string-matching parser gets it wrong.

The app is a webcam scanner that tells you whether the thing you are holding up is a dog. One HTML page, one POST /api/scan, one model call, no build step, no framework. The whole backend is 285 lines.

Three AWS specifics are worth the price of admission:

  • Lightsail container services have no IAM task role. There is nothing to attach a policy to, so the container needs a real access key as an environment variable. The mitigation is scope, not secrecy.
  • A cross-region inference profile is authorized against every region it routes to. With the policy pinned to us-east-1, a call made to us-east-1 was denied naming us-west-2. Measured, not inferred.
  • --platform linux/amd64 is not optional. An arm64 image builds, pushes and deploys cleanly, then crash-loops with an exec format error that never mentions architecture.

And a mock mode that answers every scan locally is what made the frontend free to build — no credentials, no model access, no bill.

1. The shape: one route, one call

The classification rule is the only opinionated part. is_dog is true only for a living domestic dog: a wolf is not a dog, nor is a coyote, fox, plush toy, bronze statue, cartoon, or person in a costume. That is a choice rather than a fact, and it is what makes the thing measurable — "is this a dog" is solved zero-shot by any modern vision model and has nothing to measure.

The second rule keeps it usable: judge the subject depicted, never the medium carrying it. People test this by holding a photo up on their phone, so a photograph of a real dog is a dog.

Everything else is plumbing:

browser ──HTTPS──> Lightsail container service ──> Amazon Bedrock
 camera             (nano, 1 node, FastAPI)          Nova Lite
 or upload           serves the page AND              vision + tool use
                     the /api/scan route
Enter fullscreen mode Exit fullscreen mode

Two services total. One container, one model, one IAM user. No load balancer, no bucket, no API Gateway, no CDN to invalidate.

Nova Lite is the cheapest Bedrock model that takes an image and supports tool use, which is the exact intersection this needs. The us. prefix matters: it is an inference profile, and in several regions Nova is only served through one. Invoking the bare amazon.nova-lite-v1:0 there fails with a ValidationException that never mentions profiles.

2. Force the verdict into a schema

Declare the tool, then require it. The schema is where the classification rule actually lives — the field descriptions do more work than the system prompt:

TOOL_CONFIG = {
    "tools": [{
        "toolSpec": {
            "name": "report_verdict",
            "description": "Report whether the subject presented to the "
                           "scanner is a dog. Call this exactly once, for "
                           "every image, always.",
            "inputSchema": {"json": {
                "type": "object",
                "properties": {
                    "is_dog": {
                        "type": "boolean",
                        "description": "True only for an actual living dog. "
                                       "False for a wolf, coyote, fox, plush "
                                       "toy, statue, drawing, cartoon or costume.",
                    },
                    "confidence": {"type": "integer", "description": "0-100."},
                    "subject": {
                        "type": "string",
                        "description": "What it actually is, three words or fewer: "
                                       "'golden retriever', 'grey wolf', "
                                       "'ceramic figurine'.",
                    },
                    "is_cat": {
                        "type": "boolean",
                        "description": "True if the subject is a cat. Separate "
                                       "field because a cat is not merely a non-dog.",
                    },
                },
                "required": ["is_dog", "confidence", "subject", "is_cat"],
            }},
        }
    }],
    # Without this, Nova will sometimes narrate instead of calling the tool.
    "toolChoice": {"tool": {"name": "report_verdict"}},
}
Enter fullscreen mode Exit fullscreen mode

The call itself, with the image as raw bytes — Converse takes bytes directly, so no base64 round trip on this side:

response = bedrock().converse(
    modelId="us.amazon.nova-lite-v1:0",
    system=[{"text": SYSTEM_PROMPT}],
    messages=[{"role": "user", "content": [
        {"image": {"format": "jpeg", "source": {"bytes": raw}}},
        {"text": "Identify the subject."},
    ]}],
    toolConfig=TOOL_CONFIG,
    inferenceConfig={"maxTokens": 256, "temperature": 0.2},
)
Enter fullscreen mode Exit fullscreen mode

Pulling the answer out is a loop over content blocks, not a regex:

for block in response["output"]["message"]["content"]:
    use = block.get("toolUse")
    if use and use.get("name") == "report_verdict":
        return Verdict(**use["input"])
raise HTTPException(status_code=502, detail="model did not return a verdict")
Enter fullscreen mode Exit fullscreen mode

toolChoice makes that last line close to unreachable. Keep it anyway — a 502 naming the cause beats a KeyError traceback.

Two things worth stealing:

  • Put the rule in the field description, not only the prompt. The is_dog description enumerating wolf/coyote/fox/plush/statue is read at the point of decision.
  • is_cat is a separate boolean, not a value of subject. Anything the UI branches on should be its own typed field. Parsing subject == "tabby cat" to decide whether to show a different state is how you end up back in string-matching.

3. Build the whole frontend with a mock

One environment variable short-circuits the model call and cycles four canned verdicts:

MOCK=1 ./run.sh          # http://127.0.0.1:8080, no credentials, no bill
Enter fullscreen mode Exit fullscreen mode 
MOCK_VERDICTS = [
    {"is_dog": True,  "confidence": 97, "subject": "golden retriever", "is_cat": False},
    {"is_dog": False, "confidence": 84, "subject": "grey wolf",        "is_cat": False},
    {"is_dog": False, "confidence": 91, "subject": "tabby cat",        "is_cat": True},
    {"is_dog": False, "confidence": 62, "subject": "plush dachshund",  "is_cat": False},
]
Enter fullscreen mode Exit fullscreen mode

Pick the four deliberately: the happy path, the case that makes the rule interesting, the special state, and a low-confidence one. This is the only way to reach every UI state on demand rather than by going and finding a wolf.

Build the client lazily or this mode does not work at all — constructing a boto3 client at import time fails on a machine that has never authenticated, which is precisely the machine MOCK=1 is for:

_bedrock = None

def bedrock():
    global _bedrock
    if _bedrock is None:
        _bedrock = boto3.client("bedrock-runtime", region_name=AWS_REGION, ...)
    return _bedrock
Enter fullscreen mode Exit fullscreen mode

It paid for itself on one bug. The confidence meter had:

transition: width 0.5s ease, background 0.3s ease;
Enter fullscreen mode Exit fullscreen mode

That background transition left the bar showing the previous verdict's colour indefinitely — green under NOT A DOG, red under the cat state — while the verdict text, which had no transition, switched correctly. Two halves of the same readout disagreeing, permanently, each looking fine alone. You only see it if you can fire three verdicts in two seconds. A state indicator should snap to the state; only the fill level was ever worth animating.

4. Scope the IAM user, and expect a region you never asked for

Lightsail container services cannot assume an IAM role. On ECS or Lambda you attach a policy to a role and the SDK finds credentials. Lightsail has nothing to attach to, so the container gets a real key pair as environment variables — and deployment environment variables are readable afterwards through get-container-services. There is no way to make that elegant. The honest response is to scope the key until it is boring:

./iam-setup.sh    # creates the user, writes ~/dogornot-lite.key, chmod 600
Enter fullscreen mode Exit fullscreen mode

It mints a new access key every run and deletes the old ones, so a re-run rotates rather than fails. The policy is the interesting part:

{
  "Effect": "Allow",
  "Action": ["bedrock:InvokeModel"],
  "Resource": [
    "arn:aws:bedrock:*::foundation-model/amazon.nova-lite-v1:0",
    "arn:aws:bedrock:*:<ACCOUNT_ID>:inference-profile/us.amazon.nova-lite-v1:0"
  ]
}
Enter fullscreen mode Exit fullscreen mode

Note the wildcard regions, and note that you need both ARNs. A cross-region inference profile routes across regions, and Bedrock authorizes InvokeModel against the underlying foundation-model ARN in each one. With the policy pinned to us-east-1, a call made to us-east-1 fails like this:

AccessDeniedException: ... not authorized to perform: bedrock:InvokeModel
on resource: arn:aws:bedrock:us-west-2::foundation-model/amazon.nova-lite-v1:0
Enter fullscreen mode Exit fullscreen mode

us-west-2 was never requested and never configured. I only state this because I narrowed the policy on purpose to watch it fail and then restored it. If you are staring at a Bedrock denial naming a region you never asked for, this is why.

One more denial worth naming: an AccessDeniedException at the first call, before any of the above, usually means Bedrock model access has not been granted for Nova in that account. The console calls it "Model access", and it is per-region.

5. Deploy: one container service 

aws login          # or any credential source
./iam-setup.sh     # one-time
./deploy.sh        # build, push, deploy, print the URL
Enter fullscreen mode Exit fullscreen mode

deploy.sh is idempotent — re-running it ships a new deployment version to the same service on the same URL. Budget 5–10 minutes on the first run, most of it Lightsail provisioning the service before it will accept a deployment at all.

Four things in it that are not obvious:

Build for x86 explicitly. Lightsail nodes are amd64. On an arm64 laptop the default build deploys cleanly and then crash-loops with an exec format error that never says "architecture".

docker build --platform linux/amd64 -t dog-or-not-lite:latest .
Enter fullscreen mode Exit fullscreen mode

lightsailctl is a separate binary. aws lightsail push-container-image is a thin wrapper around it and fails confusingly without it. Check with command -v lightsailctl, not by looking in /usr/local/bin — it may be installed under ~/.local/bin.

Read the pushed image reference back from the API, rather than scraping the push output, which prints it in prose:

IMAGE_REF=$(aws lightsail get-container-images --service-name dog-or-not-lite \
    --query 'containerImages[0].image' --output text)
Enter fullscreen mode Exit fullscreen mode

Do not be alarmed that the version is not sequential from 1. A brand-new service's first image came back as :dog-or-not-lite.scanner.219. That is normal.

Keep the secret off the command line. Build the containers JSON with python so the key is JSON-escaped rather than shell-interpolated, write it to a chmod 700 tempdir, and pass it by reference — ps never sees it:

aws lightsail create-container-service-deployment \
    --service-name dog-or-not-lite \
    --containers "file://$TMP_DIR/containers.json" \
    --public-endpoint "file://$TMP_DIR/endpoint.json"
Enter fullscreen mode Exit fullscreen mode

Point the health check at a route that exists (/healthz, intervalSeconds: 10, healthyThreshold: 2). And if you serve static files from the same process, mount them last — a StaticFiles at / mounted before your routes shadows every one of them:

app.mount("/", StaticFiles(directory="static"), name="static")  # must stay last
Enter fullscreen mode Exit fullscreen mode

6. Measure it before you believe it

/healthz passing proves the container booted, not that Bedrock is reachable from inside it. The only real verification is scoring the deployed URL:

./check.py --url https://<service>.us-east-1.cs.amazonlightsail.com --min-rate 0.9
Enter fullscreen mode Exit fullscreen mode

20 images, each compared against a hand-checked is_dog in fixtures/fixtures.json. Stdlib only, so it runs against a deployed URL from anywhere without installing anything. Two details make it worth having:

  • The fixtures are stored at 640×480 q70 — the exact format the browser sends. The harness exercises the same payload the real client does, not a pristine 4000px original the app will never see.
  • The expectations were verified by eye before being committed. Generate the input, never the expectation. An eval whose ground truth came out of a model is measuring agreement, not accuracy.
Metric Against the deployed service
Correct 20/20
Median latency 880 ms
Wolves, foxes, statues, cats all classified correctly
Breeds named unprompted "beagle dog", "corgi dog", "german shepherd"

Read that as a smoke test, not a benchmark. Twenty clean, well-lit, subject-fills-frame images say the prompt works and the plumbing is right. They say very little about a dog photographed badly at dusk, and I would not claim from this that the rule is robust.

7. What it costs, and how to turn it off

The standing cost is the container service, not the model. A Lightsail container service bills for as long as it exists, whether or not anyone uses it — roughly $7/month at nano. Deleting the deployment does not stop that. You have to delete the service:

aws lightsail delete-container-service --service-name dog-or-not-lite
aws iam delete-user-policy --user-name dog-or-not-lite --policy-name InvokeNovaLite
aws iam list-access-keys --user-name dog-or-not-lite   # delete each, then:
aws iam delete-user --user-name dog-or-not-lite
Enter fullscreen mode Exit fullscreen mode

Per-scan Bedrock cost is small by comparison — a 640×480 q70 JPEG lands at 40–60 KB — but I have not metered it over a long enough run to publish a number. The app logs inputTokens and outputTokens on every scan if you want your own.

8. Reproduction 

git clone https://github.com/xbill9/dog-or-not-lite
cd dog-or-not-lite
pip install -r requirements.txt

# Frontend work: no credentials, no model access, no bill
MOCK=1 ./run.sh                      # http://127.0.0.1:8080

# Real model calls locally (needs Bedrock model access for Nova in your region)
./run.sh
./check.py                           # score 20 fixtures against localhost

# Deploy
./iam-setup.sh                       # scoped IAM user + key at ~/dogornot-lite.key
./deploy.sh                          # build (amd64) → push → deploy → print URL
./check.py --url <printed url> --min-rate 0.9
Enter fullscreen mode Exit fullscreen mode

One caveat if you use aws login locally: those credentials require botocore[crt], which is deliberately not in requirements.txt. Without it boto3 reports Missing Dependency: Using the login credential provider requires ... botocore[crt] and the app returns 502 bedrock unreachable. Install it in your virtualenv only — the container authenticates with a static key pair, which needs no CRT, and adding it would put ~20 MB in the image for nothing.

Links

Fixture images from Wikimedia Commons, attributed in the repository. Bark sound effects generated with ElevenLabs.

Top comments (1)

Collapse
 
reidmarlow profile image
Reid Marlow •

This is the kind of deployment note that saves someone an hour. The amd64 build flag and the inference-profile prefix are exactly the two failures that look unrelated until you hit them. I also like that you scored the live deployment instead of localhost. Did the 880 ms median include cold-ish container starts, or only warm scans?