DEV Community

Cover image for I Had No Idea What My Bedrock Models Were Consuming — So I Built a Tool That Reports It Every Morning
Utkarsh Rastogi
Utkarsh Rastogi

Posted on

I Had No Idea What My Bedrock Models Were Consuming — So I Built a Tool That Reports It Every Morning

#aws #serverless #opensource #ai

The Wake-Up Call

I've been building with Bedrock for a while now — different projects, different models. AI Naagrik (multi-agent system), AI Ops Sentinel (error monitoring), a KMS cost tracker, community demos. Each one uses a different combination of models.

Last month, after shipping AI Naagrik, I opened Cost Explorer just to see the state of things. What I found: Nova Lite, Nova Micro, Titan Embed, and Guardrails — all actively consuming tokens. From what I thought was "just a chatbot." The AWS credits covered it, but the token volume was way more than I expected.

Then I realized — I had model access enabled for Haiku from weeks ago when I was "just trying it." I had a scheduled Lambda still running that I thought I'd stopped. And I'd been switching between Sonnet and Nova mid-development without tracking which project was using what.

None of this was visible to me day-to-day. I only found out because I randomly decided to check. If I hadn't? I'd still have no idea.

And that's the real problem — I build on AWS regularly, and even I had zero daily visibility into my AI spending. If that's my situation, I know it's everyone's situation.

Why This Matters More Than You Think

You know how you check your phone's screen time report and go "wait, 4 hours on Instagram?!" — that shock? That's what happened with my Bedrock usage. Except there's no weekly report built in. There's no notification that says "hey, you burned through 45K input tokens yesterday."

AWS gives you Cost Explorer. Cool. But:

  • You have to remember to check it. (You won't.)
  • It shows numbers without context. "$8 on Sonnet" — is that normal? Is it going up? No idea.
  • Zero recommendations. It won't tell you "you're using a $15/M-token model for tasks a $0.30/M-token model handles fine."
  • No trend awareness. A slow 10% daily creep becomes 200% in a month. No one notices until the bill arrives.

I don't want a dashboard I have to visit. I want information that comes to me. Every morning. Like a daily standup for my wallet.

So I Built This Thing

AI Bedrock Cost Lens — a Lambda that wakes up every morning, looks at everything Bedrock-related in my account, and sends me a report. Think of it as a financial advisor for your AI workloads, except it actually shows up daily and doesn't charge a percentage.

AI Bedrock Cost Lens Architecture

Three schedules. One Lambda. An email in my inbox before I finish my morning coffee.

What Shows Up in My Inbox

Every morning — quick snapshot, no fluff:

📊 BEDROCK COST LENS — Daily
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

💰 Yesterday: $8.40 | This Week: $62.00 | This Month: $185.00
   580 invocations | 920ms avg latency

MODEL BREAKDOWN:
  Claude 3.5 Sonnet: $6.72 (80%)
  Nova Lite:         $0.84 (10%)
  Titan Embed:       $0.84 (10%)
Enter fullscreen mode Exit fullscreen mode

Takes 10 seconds to read. Either everything's normal or something jumps out.

Every Monday — the one with teeth:

📊 BEDROCK COST LENS — Weekly Summary
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

🚨 ALERT: Weekly spend ($62.00) exceeds your $50/week threshold!

📈 Week-over-Week: +18.5%
   This week: $62.00 | Last week: $52.30

MODEL BREAKDOWN:
  Claude 3.5 Sonnet:  $49.60 (80%)
  Nova Lite:          $5.58  (9%)
  Titan Embed v2:     $6.82  (11%)

🤖 AI RECOMMENDATIONS:
  1. 80% of spend is Claude Sonnet. If any of those calls are 
     simple classification/routing, Nova Lite handles that at 
     1/50th the price. → Potential saving: ~$30/month

  2. Input tokens (45K) are 4x output (12K). This usually means 
     long system prompts being sent every call. Consider caching 
     or trimming. → Potential saving: ~$10/month
Enter fullscreen mode Exit fullscreen mode

The AI recommendations only appear on weekly/monthly reports. They're generated by Nova Lite (the cheapest model) analyzing the actual data patterns. No hallucinations — if there's nothing actionable, it says so.

"But Doesn't Cost Explorer Already Do This?"

Yeah, I get this question a lot. Short answer: Cost Explorer shows you data. This tells you what to do about it.

AWS Cost Explorer AI Bedrock Cost Lens
Delivery You go look at it (you won't) Comes to your inbox every morning
Frequency Whenever you remember Daily, weekly, monthly — automatic
Model breakdown Yes, but buried in filters Front and center, every report
Budget alerts Separate service (AWS Budgets), separate config Built in — threshold breach = 🚨 in subject line
Week-over-week trend Manual comparison, no memory Automatic via S3 Annotations history
Recommendations None. Just numbers. AI-generated: "switch this model, trim those prompts"
Token-level insight Not in Cost Explorer (CloudWatch only) Pulls both — tokens, latency, invocations in one report
Setup time Always available, but requires manual exploration 3-minute deploy, then forget about it
Action required from you Open console → navigate → filter → interpret Read email. Done.

Think of it this way: Cost Explorer is the source. This tool is the analyst that reads the source, compares it to last week, adds context, and slides a summary under your door every morning.

The biggest difference? Push vs pull. Cost Explorer waits for you to visit. This project doesn't wait — it shows up whether you asked or not. And that's the point. The days you'd forget to check are exactly the days something weird is happening.

How It Works Under the Hood 

EventBridge (3 schedules)
  ├── Daily (8 AM)          → {"report_type": "daily"}
  ├── Weekly (Monday 8 AM)  → {"report_type": "weekly"}
  └── Monthly (1st, 8 AM)   → {"report_type": "monthly"}
        ↓
Lambda (Python 3.12)
  ├── cost_service.py     → Pulls from Cost Explorer API
  ├── metrics_service.py  → CloudWatch (invocations, tokens, latency)
  ├── ai_service.py       → Bedrock Nova Lite (recommendations)
  ├── storage_service.py  → S3 Annotations (trend history)
  └── email_service.py    → SES (HTML report)
Enter fullscreen mode Exit fullscreen mode

One function handles all three report types. EventBridge tells it what to generate. Daily runs skip the AI call entirely (why pay for recommendations on a single day's data?). Weekly/monthly get the full treatment.

The Part I'm Most Proud Of: S3 Annotations for Memory

The tool needed memory. "Was this week more expensive than last week?" requires knowing what last week looked like.

Options I considered:

  • DynamoDB — way overkill for one number per day
  • A new S3 file per day — messy, hundreds of objects piling up
  • S3 Annotations — one object, many annotations attached to it. Clean.

Here's how it works. One file sits in S3. Every day, a new annotation gets attached:

s3.put_object_annotation(
    Bucket=bucket,
    Key="analysis/bedrock-costs.json",
    AnnotationName="analysis.20260724",
    AnnotationPayload=json.dumps({
        "daily": 8.42, 
        "weekly": 48.20,
        "models": {"nova-lite": 2.10, "sonnet": 6.32}
    }).encode()
)
Enter fullscreen mode Exit fullscreen mode

Week-over-week comparison? Read the annotation from 7 days ago:

last_week = s3.get_object_annotation(
    Bucket=bucket,
    Key="analysis/bedrock-costs.json",
    AnnotationName="analysis.20260717"
)
Enter fullscreen mode Exit fullscreen mode

No database. No cleanup jobs. No TTL. Up to 1,000 annotations = ~3 years of daily history on a single object. This is the kind of AWS feature that doesn't get enough love.

Gotchas That'll Save You Hours

Things I learned so you don't have to bang your head:

Gotcha What happens Fix
Cost Explorer data delay You query today, get nothing Always query 2 days back
End date is exclusive end="07-25" returns up to 07-24 Add 1 day to your intended end
$0 with active credits Usage exists but shows no charge Still useful for model breakdown
CE API region locked Fails from any region except us-east-1 Force region in boto3 client

That last one is sneaky. Even if your Lambda runs in Mumbai, the Cost Explorer client must target us-east-1. Undocumented fun.

The Honest AI Recommendations

I hate AI tools that make up impressive-sounding advice with no basis. This one only speaks when it sees real patterns:

✅ "80% on Sonnet — consider Nova for simple tasks" (backed by actual usage split)
✅ "Input tokens 6x output — suggests oversized prompts" (backed by token metrics)
✅ "18% week-over-week increase — new workload?" (backed by S3 annotation history)

❌ Will NOT guess your use case
❌ Will NOT hallucinate savings numbers
❌ Returns "No actionable recommendations" when data is insufficient

If your spend is $0 (credits covering everything), it tells you straight up. No fake insights.

What's Powering This — 8 Services

Service Role
Lambda Runs the whole analysis (~5 sec per execution)
EventBridge 3 cron schedules — daily, weekly, monthly
Cost Explorer Model-level spend data
CloudWatch Metrics Invocations, latency, token counts
Bedrock Nova Lite Generates recommendations (cheapest model)
S3 + S3 Annotations Trend memory without a database
SES Email delivery
CloudWatch Dashboard 4-panel visual (invocations, latency, tokens, errors)

Monthly running total: under $4. Mostly the CloudWatch Dashboard at $3 flat. Without it, you're looking at ~$1/month. That's less than the random test invocations you forget to turn off.

Who's This Actually For?

🎓 Students — You're experimenting with Bedrock on credits. This tells you before they run out.

🚀 Solo builders / indie hackers — Multiple AI projects, no dedicated ops team. Daily email = instant awareness.

👥 Startup teams — "Which feature is eating our AI budget?" answered every Monday morning.

🏢 Anyone with Bedrock in production — Gradual drift is the silent killer. This catches it early.

Get It Running (3 minutes, literally) 

git clone https://github.com/Utkarshlearner/ai-bedrock-cost-lens.git
cd ai-bedrock-cost-lens
cp .env.example .env   # Set ALERT_EMAIL + AWS_REGION
./deploy.sh all        # 6 CloudFormation stacks
./test.sh              # Sends a test report to your inbox right now
Enter fullscreen mode Exit fullscreen mode

Tomorrow at 8 AM UTC, your first real report arrives. No further action needed.

But honestly? The email version handles 90% of the problem. I've been using it for weeks and it's changed how I think about my AI projects. Five seconds every morning = no more surprises.

Try It, Break It, Tell Me What's Missing

📂 [ai-bedrock-cost-lens(https://github.com/Utkarshlearner/ai-bedrock-cost-lens)

Star it if you find it useful. Open an issue if something breaks. PR if you want to add Slack support before I do. 🙌

Built by Utkarsh Rastogi — AWS Community Builder

More projects: dev.to/awslearnerdaily

Top comments (0)