Run Amazon Bedrock locally, with real completions from Ollama

MiniStack 1.4.0 ships four new services that emulate Amazon Bedrock end to end, and one env var that turns the mock into a real LLM backend.

The default: deterministic, shape-perfect mocks

Out of the box, Converse and InvokeModel return deterministic mock responses whose wire shape matches the model family you asked for. Anthropic, Titan, Nova, Llama, Mistral, Cohere and AI21 each have distinct response schemas, and MiniStack selects the right one by model ID prefix. Streaming operations (ConverseStream, InvokeModelWithResponseStream) use the real eventstream wire format, so your SDK's streaming parser runs exactly as against AWS.

That means your test suite exercises the real code paths: request building, response parsing, stream handling. No network call, no API bill.

One honest caveat: token counts are a chars/4 heuristic. Shape-correct, but don't assert exact values against a mock.

The upgrade: real completions from your local model

docker run -p 4566:4566 \
  -e MINISTACK_BEDROCK_PROXY_URL=http://host.docker.internal:11434/v1 \
  ministackorg/ministack

With the proxy set to any OpenAI-compatible /chat/completions endpoint (Ollama, llama.cpp, vLLM), MiniStack translates your Bedrock prompt to OpenAI shape, forwards it, and translates the response back into Bedrock shape:

bedrock = boto3.client("bedrock-runtime", endpoint_url="http://localhost:4566",
                       region_name="us-east-1", aws_access_key_id="test",
                       aws_secret_access_key="test")
resp = bedrock.converse(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"role": "user", "content": [{"text": "Hello from MiniStack!"}]}],
)
print(resp["output"]["message"]["content"][0]["text"])  # a real completion

Unchanged AWS SDK. Real model. Zero cloud.

If the proxy endpoint is unreachable, MiniStack falls back to the mock silently. Your CI doesn't care whether Ollama is running.

The rest of the surface

• bedrock (control plane): 66 operations verified against botocore. Foundation model catalog with real model IDs, inference profiles, guardrails with versioning, custom and imported models, provisioned throughput, and the customization, import, copy and batch-invocation job families.
• bedrock-agent: 72 operations. Agents with versions, aliases and action groups, knowledge bases with data sources and ingestion jobs, flows, prompts.
• bedrock-agent-runtime: 31 operations. InvokeAgent, Retrieve and RetrieveAndGenerate, reranking, the full session API, flow executions.

All four services are account and region scoped (1.4.0 also shipped multi-region support, separate post).

The Bedrock work was led by dcabib.

Try it

docker run -p 4566:4566 ministackorg/ministack

MIT, free forever. Changelog: https://ministack.org/blog/changelog-v1-4-0.html

Comments

[Mike Czerwinski]

The falls-back-to-mock-silently-if-the-proxy-is-unreachable detail is the right default for CI specifically, tests shouldn't flap depending on whether Ollama happens to be running, but it's worth flagging in the docs loudly since a silent fallback is exactly the kind of thing that can mask a real config mistake in a dev environment where you expected the real model to answer. The honest caveat on token counts being a chars/4 heuristic rather than a measurement is good practice, better to say so upfront than have someone assert on an exact value against a mock and get burned later. Shape-correct-not-value-correct is a useful distinction for anyone building test fixtures against a cloud API. 66+72+31 operations verified against botocore is a serious amount of surface area for a mock layer, curious how you're keeping that in sync as AWS ships new Bedrock operations, is there a regression check against the live API schema or is it a manual review cadence.