If you’ve ever tried adding an AI feature to an app, you know that you should set up a server, figure out GPU hosting, and spend half your week reading infrastructure docs. Cloud providers finally realized nobody wants to do that.
Let's get acquainted with Amazon Bedrock. Instead of managing servers, you get a single API endpoint that connects to hundreds of foundation models, like Claude, Llama, Nova, Mistral, and more.
No infrastructure: No servers or GPUs to configure.
Pay-per-token. You pay only for what you use.
Prompts and responses are not used to train models and are not shared with providers.
Inside you can find lists of over 200 serverless and marketplace models. You can filter by provider (Amazon Nova, Anthropic Claude, Meta Llama, etc.) to check parameters and capabilities.
Playground lets you test prompts in your browser before writing code. Pick a lightweight option like Amazon Nova 2 Light, give it a system prompt, and run a quick test.
Some models (like Meta Llama) have regional restrictions or licensing limits. Providers like Anthropic require a one-time use-case form per account (company name, website, industry, and use-case description) before granting model access.
Bedrock uses the Converse API, providing a uniform request structure across all models. The payload requires three main fields: modelId , messages , inferenceConfig .
import boto3
from config import REGION, MODEL_ID
bedrock = boto3.client("bedrock-runtime", region_name=REGION)
start = time.time()
response = bedrock.converse(
modelId=MODEL_ID,
messages=[{"role": "user", "content": [{"text": "Write a one-line welcome message for BrightCart, an online electronics store."}]}],
inferenceConfig={"maxTokens": 200, "temperature": 0.5},
)
seconds = time.time() - start
print("Model:", MODEL_ID)
print("Answer:", response["output"]["message"]["content"][0]["text"])
usage = response["usage"]
print(f"Tokens: {usage['inputTokens']} in, {usage['outputTokens']} out · {seconds:.1f} s")
In config.py
REGION = "us-east-1"
# Model IDs (copy the newest ones from Bedrock console > Model catalog > model card > "Inference profile ID")
NOVA = "us.amazon.nova-2-lite-v1:0"
CLAUDE = "global.anthropic.claude-sonnet-4-5-20250929-v1:0"
MODEL_ID = os.getenv("MODEL_ID", NOVA)
Two Metrics we have to monitor
Token Breakdown: Models process fragments of words called tokens, not raw text. You pay for both input tokens (your prompt) and output tokens (the generated response). Different models split text differently-the same prompt might cost 60 tokens on one model and 25 on another. Keep an eye on the usage stats returned in the API payload.
Temperature Setting: Controls randomness.
Low (0.1–0.3): Keeps outputs deterministic and grounded. Perfect for customer support, document parsing, or factual Q&A.
High (0.7–1.0): Adds variety and creative range. Great for marketing copy or creative brainstorming.
By using AWS Bedrock in combination with Python and Boto3, you gain a convenient and scalable way to run advanced large language models (LLMs) within your own infrastructure-without the need to configure servers or lose control over costs.
Source code for all hands-on episodes: GitHub repository
I’ll be sharing plenty of hands-on videos about AWS, AI and deployment on my YouTube channel, @CodeAiFlow every week. This article follows the second episode of the series, if you'd rather watch the setup step by step. I'm also an AWS Community Builder: my profile on AWS Builder Center.
Top comments (0)