Originally published on kuryzhev.cloud
Your internal Slack runbook bot is sending every incident prompt to OpenAI's servers — and your security team doesn't know yet. This is the exact situation I've seen play out on three separate teams in the past year. Someone wires up a quick chatbot for incident triage or runbook lookup, ships it, and six months later a SOC 2 audit uncovers that sensitive operational data has been leaving the AWS environment entirely. Running this Bedrock vs OpenAI API compliance checklist before you commit to either platform takes about 30 minutes and can save you a very uncomfortable conversation with your CISO.
Why This Checklist Exists
Internal DevOps chatbots — Slack bots that query runbooks, incident triage assistants, on-call helpers that summarize recent alerts — sit at the intersection of three concerns that rarely get audited together: cost control, data residency, and IAM governance. Most teams default to OpenAI because the API is fast to integrate and the models are excellent. I get it. I've done it too. But the decision deserves more than "OpenAI is easy."
The specific pain point is this: when your chatbot processes incident data, it's often sending hostnames, internal service names, error messages, and occasionally credentials that leaked into logs. If that traffic goes to api.openai.com, it's leaving your AWS account boundary. That matters enormously if your org has a BAA requirement, a SOC 2 Type II mandate, or operates under FedRAMP constraints.
AWS Bedrock solves this by keeping inference within your AWS account. The trade-off is more setup overhead and some real gotchas around model availability and logging configuration that aren't obvious from the docs. This checklist covers AWS-native teams running workloads in us-east-1 or us-west-2, comparing Bedrock (Claude 3 Sonnet, Claude 3 Haiku, Titan) against OpenAI API (gpt-4o, gpt-3.5-turbo). It's not a general AI evaluation — it's an infrastructure decision framework for teams that already have an AWS footprint and need to make a defensible architecture choice.
For more context on securing AWS workloads in CI/CD pipelines, the kuryzhev.cloud DevOps archive has related patterns worth reviewing before you finalize your setup.
The Checklist: Bedrock vs OpenAI API Decision Points
Run through each gate in order. If you hit a hard blocker, stop and document it before continuing.
- Cost gate. Calculate your expected token volume before writing a single line of integration code. Bedrock Claude 3 Haiku costs $0.25 per 1M input tokens and $1.25 per 1M output tokens in us-east-1 on-demand. OpenAI gpt-4o costs $5.00 per 1M input and $15.00 per 1M output. At 10M output tokens per month — not unusual for an active team chatbot — that's $12.50 on Bedrock Haiku versus $150 on gpt-4o. A 12x difference that compounds fast as chatbot adoption grows. Also factor in AWS data transfer costs if your application runs outside a VPC endpoint for Bedrock; that adds $0.01/GB but is easy to miss in initial estimates.
- Compliance gate. Check whether your organization has a BAA requirement (HIPAA), SOC 2 Type II mandate, or FedRAMP boundary before choosing. Bedrock inference stays within your AWS account by default. OpenAI data leaves your AWS environment entirely. OpenAI does offer a "zero data retention" tier for enterprise customers, but here's the critical detail: it is not the default. Standard API tier retains data for 30 days. I've watched teams sign up, skip this detail, and fail a HIPAA audit because they assumed ZDR was automatic. If compliance is a hard requirement, Bedrock wins this gate by default.
-
IAM gate. Bedrock uses a standard
aws_iam_rolewithbedrock:InvokeModelpermission scoped to a specific model ARN. OpenAI requires storing an API key as a secret — ideally in AWS Secrets Manager or SSM Parameter Store. The question to audit: who in your org currently hassecretsmanager:GetSecretValueaccess? In practice, that permission is often over-granted. With Bedrock, you're working within IAM, which most teams already audit. With OpenAI, you're adding a secret rotation burden that frequently gets neglected. - Latency gate. Bedrock invocation through a VPC endpoint adds roughly 80–150ms compared to direct OpenAI API calls at 50–100ms for first token. For a Slack bot, this is imperceptible. For a real-time autocomplete use case, measure it with your actual VPC configuration before committing. Don't assume — benchmark.
-
Model availability gate. Bedrock requires you to explicitly request model access per region. Claude 3 Opus was not available in all regions as of mid-2024. Verify before your architecture meeting, not after:
Run this for every region you intend to deploy in. I've seen teams design around Claude 3 Sonnet only to discover it requires a manual access request with a 24-hour approval window — which is a problem when you're mid-sprint.aws bedrock list-foundation-models \ --region us-east-1 \ --output table \ --query "modelSummaries[?contains(modelId,'claude')].[modelId,modelLifecycle.status]" -
Fallback gate. Define your retry behavior before you hit production throttling. Bedrock throws
ThrottlingException; implement exponential backoff with jitter, base delay 1s, max 60s, max 5 attempts. OpenAI returns HTTP 429 with aRetry-Afterheader. The retry logic is different enough that you need explicit handling for both if you're building a fallback pattern. Also confirm you're usingboto3>=1.28.57— earlier versions don't include thebedrock-runtimeclient and will fail silently in ways that are genuinely confusing to debug.
Commonly Missed Items
These are the items that don't show up in the official getting-started guides but cause production incidents or compliance failures.
Bedrock Provisioned Throughput is not on-demand. If you accidentally provision a Model Unit instead of using on-demand inference, you're committing to approximately $21.50/hour for a minimum of one month. That's roughly $15,480/month. I have personally seen this happen when someone clicked the wrong option in the console during a proof-of-concept. Always verify your invocation type in Terraform before applying. Provisioned Throughput makes sense at scale — it does not make sense for a chatbot pilot.
Model Invocation Logging is off by default. This one fails audits regularly. Teams assume that because Bedrock is an AWS service, it logs everything like other AWS services do. It does not. You must explicitly enable it:
aws bedrock put-model-invocation-logging-configuration \
--logging-config '{
"cloudWatchConfig": {
"logGroupName": "/aws/bedrock/chatbot-invocations",
"roleArn": "arn:aws:iam::ACCOUNT_ID:role/bedrock-logging-role",
"largeDataDeliveryS3Config": {
"bucketName": "your-bedrock-logs-bucket",
"keyPrefix": "invocation-logs/"
}
},
"s3Config": {
"bucketName": "your-bedrock-logs-bucket",
"keyPrefix": "invocation-logs/"
},
"textDataDeliveryEnabled": true,
"imageDataDeliveryEnabled": false,
"embeddingDataDeliveryEnabled": false
}'Note that textDataDeliveryEnabled: true logs prompt and response content. Depending on your compliance posture, you may want this on for audit purposes — or off if prompts contain PII. Make the decision explicitly, not by omission.
Cross-region inference profiles break data residency assumptions. Bedrock inference profile IDs with a us. prefix — for example, us.anthropic.claude-3-5-sonnet-20241022-v2:0 — indicate cross-region routing. Traffic may transit us-west-2 even if your primary region is us-east-1. If your data residency policy requires all inference to stay in a single region, use the standard model ID without the us. prefix and verify you're not inadvertently calling a cross-region profile.
Watch out for the wrong boto3 client. Using the bedrock client instead of bedrock-runtime for inference calls throws UnrecognizedClientException. The bedrock client is for management APIs (listing models, managing provisioned throughput). The bedrock-runtime client is for actual inference. This trips up almost everyone the first time.
Automation Ideas
Running this checklist manually once is fine for the initial decision. Enforcing it programmatically across your team is how you prevent drift.
Terraform module for least-privilege Bedrock IAM. The following module provisions the IAM role and CloudWatch log group your chatbot needs, scoped to a specific model ARN rather than a wildcard. See the Terraform AWS IAM role policy documentation for the full resource reference.
# terraform/bedrock_chatbot_iam.tf
# Provisions least-privilege IAM role for internal DevOps chatbot
# using Bedrock Claude 3 Haiku — avoids wildcard model permissions
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.40"
}
}
}
variable "chatbot_service_name" {
description = "Name tag for the internal chatbot service (e.g. slack-runbook-bot)"
type = string
}
variable "aws_region" {
default = "us-east-1"
}
# IAM role assumed by the chatbot Lambda or ECS task
resource "aws_iam_role" "bedrock_chatbot_role" {
name = "${var.chatbot_service_name}-bedrock-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Principal = { Service = "lambda.amazonaws.com" }
Action = "sts:AssumeRole"
}]
})
tags = {
Service = var.chatbot_service_name
ManagedBy = "terraform"
Compliance = "internal-chatbot"
}
}
# Scoped Bedrock invoke policy — specific model ARN, not wildcard
resource "aws_iam_role_policy" "bedrock_invoke_policy" {
name = "bedrock-invoke-haiku-only"
role = aws_iam_role.bedrock_chatbot_role.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "AllowBedrockInvokeHaiku"
Effect = "Allow"
Action = ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"]
# Pin to specific model version — do NOT use wildcard
Resource = "arn:aws:bedrock:${var.aws_region}::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
},
{
Sid = "AllowBedrockInvocationLogsWrite"
Effect = "Allow"
Action = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
Resource = "arn:aws:logs:${var.aws_region}:*:log-group:/aws/bedrock/chatbot-invocations:*"
}
]
})
}
# CloudWatch log group for Bedrock model invocation logging
# Must be paired with aws bedrock put-model-invocation-logging-configuration
resource "aws_cloudwatch_log_group" "bedrock_invocation_logs" {
name = "/aws/bedrock/chatbot-invocations"
retention_in_days = 90 # Adjust per your compliance retention policy
tags = {
Service = var.chatbot_service_name
Compliance = "audit-trail"
}
}
output "chatbot_role_arn" {
description = "IAM role ARN to assign to chatbot Lambda/ECS task"
value = aws_iam_role.bedrock_chatbot_role.arn
}Python cost estimator as a CI gate. Run this before any chatbot deployment to output a cost comparison table. Wire it into your CI pipeline as a pre-deployment step so token volume estimates are always reviewed. See the AWS Bedrock model IDs documentation for current pricing references per region.
#!/usr/bin/env python3
# cost_estimator.py — Compare Bedrock vs OpenAI monthly token costs
# Run: python3 cost_estimator.py --input-tokens 5000000 --output-tokens 2000000
# Requires: no external dependencies (stdlib only)
# boto3>=1.28.57 needed separately for actual Bedrock calls
import argparse
# Pricing per 1M tokens (USD, on-demand, us-east-1, as of mid-2024)
PRICING = {
"bedrock_claude3_haiku": {"input": 0.25, "output": 1.25},
"bedrock_claude3_sonnet": {"input": 3.00, "output": 15.00},
"openai_gpt35_turbo": {"input": 0.50, "output": 1.50},
"openai_gpt4o": {"input": 5.00, "output": 15.00},
}
def calculate_cost(model: str, input_tokens: int, output_tokens: int) -> float:
p = PRICING[model]
return (input_tokens / 1_000_000 * p["input"]) + \
(output_tokens / 1_000_000 * p["output"])
def main():
parser = argparse.ArgumentParser(description="Bedrock vs OpenAI cost estimator")
parser.add_argument("--input-tokens", type=int, required=True,
help="Monthly input token volume")
parser.add_argument("--output-tokens", type=int, required=True,
help="Monthly output token volume")
args = parser.parse_args()
print(f"\n{'Model'::<30} {'Monthly Cost (USD)':>20}")
print("-" * 52)
results = {}
for model in PRICING:
cost = calculate_cost(model, args.input_tokens, args.output_tokens)
results[model] = cost
print(f"{model:<30} ${cost:>19.2f}")
# Highlight cheapest option
cheapest = min(results, key=results.get)
print(f"\n✅ Cheapest: {cheapest} at ${results[cheapest]:.2f}/month")
# Flag if Provisioned Throughput floor is relevant
pt_monthly = 21.50 * 24 * 30 # 1 Model Unit, 1 month minimum
if results.get("openai_gpt4o", 0) > pt_monthly:
print(f"⚠️ gpt-4o cost exceeds Bedrock PT floor (${pt_monthly:.0f}/mo) — evaluate Provisioned Throughput")
if __name__ == "__main__":
main()
# Example output for --input-tokens 5000000 --output-tokens 2000000:
# bedrock_claude3_haiku $ 3.75
# bedrock_claude3_sonnet $ 45.00
# openai_gpt35_turbo $ 5.50
# openai_gpt4o $ 55.00
# ✅ Cheapest: bedrock_claude3_haiku at $3.75/monthSecret scanning enforcement. Add a pre-commit hook using trufflehog v3 to block OpenAI API key commits before they hit your repo. Install with brew install trufflehog or pull the container: docker run ghcr.io/trufflesecurity/trufflehog:latest. Run it as: trufflehog filesystem . --only-verified. Pair this with an AWS Config managed rule — SECRETSMANAGER_SECRET_UNUSED — to flag stale OpenAI keys that haven't been rotated in 90 days. Neither of these is a substitute for the other; you need both the pre-commit gate and the ongoing compliance check.
The Bedrock vs OpenAI API compliance decision isn't about which model is smarter. It's about whether your architecture can withstand a security review six months from now. Run the checklist, automate the gates, and make the decision explicitly rather than by default.
Top comments (0)