DEV Community: Mohamed Sorour

Chess.com for Debugging: How BugHunt Runs Real-Time Multiplayer on DynamoDB with Zero Servers

Mohamed Sorour — Sun, 28 Jun 2026 20:43:09 +0000

Every developer debugs. It's the skill you use more than any algorithm — reading unfamiliar code, spotting what's wrong, fixing it under pressure. Yet there's no competitive platform for it. LeetCode tests algorithms. HackerRank tests data structures. Nothing tests the skill you actually use at 2am in production.

BugHunt changes that. It's a real-time 1v1 debugging game built to handle a million daily active users on a single DynamoDB table with no servers to manage: two players are matched by Elo rating, see the same buggy code snippet, race to identify the bug, and earn chess-style ratings. Three rounds, 120 seconds each, fastest accurate answer wins. At 1M DAU × 5 games/day that's ~290 game resolves per second at peak — all absorbed by DynamoDB on-demand, with no provisioned capacity, no hot-partition tricks, and no compute layer beyond Vercel serverless functions. It's live at bughunt-beryl.vercel.app.

Why DynamoDB — and Why It Scales

The H0 hackathon Track 3 asks for a million-scale global app. Before writing a line of code, I did the math: 1M DAU × 5 games/day = 5M games/day ≈ 58 games/s average, ~290/s at peak (5× burst). Each game generates ~12 DynamoDB writes across game state, player answers, and Elo updates — call it 3,500 WCU/s peak. DynamoDB on-demand handles this trivially: UUID partition keys distribute writes uniformly, there are no hot partitions, and capacity scales automatically with no provisioning.

The table uses Global Tables replicated to us-east-1, eu-west-1, and ap-southeast-1 — reads route to the nearest region. The leaderboard is a materialized view (constant-time reads at any user count, explained below). Every race condition is settled by ConditionExpressions (never locks). The result: a multiplayer game that scales horizontally without any changes to the application layer.

When I listed every read and write the app would make, every single one was either a key lookup or a single-partition query:

What	Key pattern	Cost
Load a profile	`USER#id` / `PROFILE`	1 RCU
Find active game	GSI1: `ACTIVE_GAME#userId`	1 query
Load game + my answers	`GAME#id` / `META`, `PLAYER#uid`	2 reads
Submit an answer	Conditional update on `answers[i]`	1 WCU
Top-100 leaderboard	`LEADERBOARD#GLOBAL` descending, limit 100	1 query

No joins. No aggregations at read time. This is DynamoDB's sweet spot — and on-demand capacity means I never provision anything. The table scales with traffic, period.

One Table, Every Entity

Everything in BugHunt lives in one DynamoDB table called bughunt-main:

Users — profiles, Elo, match history, social follows
Games — game state, per-player answers, post-game chat
Bugs — the actual code snippets with options and explanations
Leaderboards — materialized RANK# rows, never computed
Tournaments — brackets and registrations
Organizations — team leaderboards
Daily challenges — one per day with streak tracking

Two GSIs cover the non-key access patterns: GSI1 maps active games to users (so matchmaking can find your in-progress game), and GSI2 maps emails to user IDs (for OAuth account linking).

The Hard Part: Multiplayer on Serverless

A real-time multiplayer game is a pile of race conditions. Both players submitting in the same millisecond. Game resolution triggered twice. A rematch accepted from both ends simultaneously. A tournament slot claimed by two people.

In a traditional backend, you'd use database transactions or mutex locks. On serverless (no persistent state, no shared memory), those don't exist. Instead, I settled every race condition with DynamoDB ConditionExpressions:

// Submit answer — only succeeds if this round hasn't been answered yet
ConditionExpression: "attribute_not_exists(answers[2].submittedAt)"

// Resolve game — only one request wins
ConditionExpression: "#status = :active"

// Join private game — exactly-once player 2 assignment
ConditionExpression: "attribute_not_exists(player2Id) OR player2Id = :null"

The pattern is consistent: every write that can race carries a condition. The first writer wins; the second gets a ConditionalCheckFailedException and handles it gracefully (usually by returning the current state). No corruption, no locks, no transactions.

The Leaderboard Is a Materialized View

Most leaderboards scan every user, sort by score, and return the top N. That's an O(n) query that gets slower as your user base grows.

BugHunt's leaderboard is written, never read-computed. Here's how:

When a game resolves, resolveGame() updates both players' Elo on their profiles, then stamps p1EloBefore, p1EloAfter, p2EloBefore, p2EloAfter onto the game META item.
That MODIFY event flows through DynamoDB Streams to a Lambda function.
The Lambda moves each player's RANK#<zero-padded-elo>#<userId> row under the LEADERBOARD#GLOBAL partition. Delete the old Elo row, write the new one.
Reading the top 100 is one descending Query on the leaderboard partition. Constant time, regardless of user count.

The zero-padding trick (001262 for Elo 1262) turns DynamoDB's lexicographic sort key ordering into a numeric ranking. A per-user cursor row keyed by monotonic gamesPlayed count guards against cross-shard event reordering — even if two games for the same player arrive out of order, the cursor ensures the latest Elo always wins.

Real Multiplayer: Elo-Bucketed Matchmaking

When you click Find Match, BugHunt pushes your Elo rating into a Redis sorted set bucketed by skill range. The next player in a compatible Elo band atomically claims the match — a ZREM that returns > 0 is the only valid claim; two concurrent callers can't both grab the same opponent. The winner creates a game in DynamoDB with a conditional write, and both clients get their game ID back within milliseconds. If the ZREM wins but the game creation later fails (no bugs available, conditional conflict), the claimed opponent is re-enqueued — no one gets silently dropped.

This is the primary experience: humans racing humans at similar skill levels. Bots only enter when you've waited more than ten seconds with no human in range.

Bots Without Servers

Hackathon demos die on empty matchmaking queues. But running a bot server defeats the "zero servers" architecture. BugHunt's bots have no process at all.

When a player has been in the matchmaking queue for 10 seconds with no human opponent, the next matchmake poll creates a game against a bot. From that point, the human's own requests power the bot's turns:

Every status poll / SSE tick calls maybePlayBotRound(game)
It checks whether the bot's deterministic think delay (seeded from sha256(gameId:round)) has elapsed
If yes, it writes the bot's answer through the exact same conditional-write path humans use
If another concurrent request already wrote it, the conditional write fails silently

The bot's answer quality is Elo-calibrated: a probability curve based on the bot's rating vs. the bug's difficulty, optionally overridden by Amazon Bedrock Nova (which literally reads the code and picks an answer with a 2.5-second timeout).

Real-Time Without WebSockets

Vercel serverless functions can't hold WebSocket connections. BugHunt uses Server-Sent Events (SSE) with layered fallbacks:

Primary: TCP-based Redis pub/sub via ioredis — the game route subscribes to a game:<gameId> channel and pushes events instantly
Safety net: 10-second DynamoDB poll inside the same SSE connection catches anything pub/sub misses
Fallback: If Redis is completely unavailable, the client polls /api/game/status every 3 seconds

This means the game works even if Redis is down — it just updates slightly slower. No single point of failure.

Scale Math — The Honest Version

Rather than hand-wave "it'll scale," I documented every known limit and its mitigation path:

Flow	Peak load	Verdict
Game writes (~12 writes/game)	~3,500 WCU/s	On-demand + UUID keys absorb trivially
Matchmaking (zadd/zrange/zrem)	~700 ops/s @ 2K concurrent	Redis sorted sets, sharded by Elo band (~15 buckets)
Leaderboard reads	~1 Query/60s/warm instance	In-memory cache; effectively free
SSE (DynamoDB polling fallback)	~50K reads/s @ 100K concurrent	DynamoDB fine; real cost is function-hours

Known limits with mitigation paths:

Leaderboard partition write rate — at ~1,000 WCU/partition/s the limit is ~125 resolves/s (≈0.4M DAU). Fix: Lambda skips writes for players whose Elo is far below the top-100 cutoff. >99% of games involve no top-100 candidate at million scale.
BUG#INDEX item size — one item, 400KB cap, safe to ~10K bugs. Fix: shard by difficulty (5 smaller items, same optimistic-versioning write path).
SSE on serverless — held-open functions are the dollar cost. Redis TCP pub/sub (when REDIS_URL is set) reduces per-game DynamoDB reads from 0.5/s to ~0.1/s. Final fallback: client polling.
Multi-region writes — currently pinned to us-east-1. Fix: multi-region Vercel functions + region-affinity matching (players matched through one queue, game writes share a region).

Full capacity documentation: docs/ARCHITECTURE.md

The Stack

Layer	Technology
Frontend	Next.js 16 App Router (Turbopack)
Deployment	Vercel (global edge)
Database	DynamoDB single-table, on-demand, Global Tables
Stream processing	DynamoDB Streams + Lambda
Real-time coordination	Upstash Redis (matchmaking, pub/sub, rate limiting)
AI	Amazon Bedrock Nova Lite (bug quality filter, bot opponent)
Auth	NextAuth v5 (Google + GitHub OAuth)
Scheduling	Upstash QStash (daily challenge, tournament tick)

What I Learned

Single-table design is a forcing function. You must know every access pattern before you write a line of code. That sounds painful, but it means your data model serves your app perfectly — no ORM surprises, no N+1 queries, no migration headaches.

On-demand DynamoDB makes 95% of scale free. The engineering is in the other 5%: hot partitions, item-size ceilings, and held-open serverless functions. Do that math in the open. Your future self — and apparently hackathon judges — will thank you.

ConditionExpressions replace locks. Every race condition in a multiplayer game can be expressed as a conditional write. The mental model shift from "acquire lock, do work, release lock" to "attempt write, handle rejection" is genuinely better for serverless architectures.

Try BugHunt: bughunt-beryl.vercel.app

Source code: github.com/mohamedsorour1998/bughunt

Built for the H0 Hackathon — Vercel + AWS Databases. #H0Hackathon

How an AWS Agent Helped Me Optimize Resources Effortlessly

Mohamed Sorour — Tue, 30 Sep 2025 05:18:21 +0000

AWS Resource Optimizer Agent

GitHub

I built a production-ready AWS Resource Optimizer Agent with Amazon Bedrock AgentCore Gateway, Claude Sonnet 4.5, and the Strands Agents SDK, and it completely transformed how I manage my cloud environment. Instead of digging through CloudWatch dashboards or juggling CLI commands, I can now just ask questions like “Which EC2 instances are underutilized?” or “Show me Lambda errors from the last 24 hours”—and the agent instantly pulls the right metrics, logs, or EBS data from over 137 AWS tools. I can run it locally for testing or deploy it to AgentCore Runtime for secure, scalable use, with built-in OAuth, memory, and semantic search that make it simple and powerful. Setup took only a few steps—create an IAM role, load the Smithy specs for CloudWatch, Logs, and EBS, and the agent was live. It saves me hours of manual work and gives me confidence my infrastructure is running efficiently—I can’t imagine going back.

📋 Table of Contents

Overview
Architecture
Deployment Options
- Option 1: Local Development
- Option 2: AgentCore Runtime
API Targets & Capabilities
Setup Guide
Configuration Reference
Troubleshooting
Project Structure

Overview

What It Does

AI-powered agent that monitors and optimizes AWS resources through natural language conversations using:

CloudWatch Metrics API: Monitor EC2, RDS, Lambda, S3, EBS performance
CloudWatch Logs API: Analyze application and infrastructure logs
EBS API: Manage volumes and snapshots

Key Features

✅ 137 AWS Tools Access: in AgentCore GatewayCloudWatch, Logs, and EBS APIs via AgentCore Gateway
✅ Semantic Search: Intelligent tool discovery via AgentCore Gateway
✅ Persistent Memory: Cross-session conversation continuity
✅ Ready: Ready local CLI and in progress to be Deployable to AgentCore Runtime
✅ OAuth Secured: Cognito-based authentication

Use Cases

Identify idle or underutilized resources
Analyze Lambda function errors
Monitor RDS database performance
Search CloudWatch logs for issues
Optimize EBS storage usage

Architecture

Components

AgentCore Gateway: MCP protocol server exposing AWS APIs as tools
Claude Sonnet 4.5: LLM for natural language understanding
Strands Agents: Agent orchestration framework
AgentCore Memory: Persistent conversation storage
Cognito: OAuth 2.0 authentication

Deployment Options

Option 1: Local Development (CLI)

Best for: Development, testing, rapid iteration

Prerequisites

Python 3.10+
AWS credentials configured
Smithy API specs in S3 (cost-explorer-smithy-api bucket)
IAM permissions for bedrock-agentcore, IAM, Cognito

Setup Steps

# 1. Install dependencies
pip install -r requirements.txt

# 2. Create infrastructure
cd setup
python3 01-create-iam-role.py
python3 02-create-gateway.py
python3 03-create-smithy-targets.py

# Wait 60 seconds for tools to sync

# 3. Run agent locally
cd ..
python3 agent.py

Usage:

python3 agent.py
👤 You: List my CloudWatch metrics
🤖 Agent: [Shows available metrics...]

Option 2: AgentCore Runtime (Production)

Best for: Production deployment, managed hosting, team environments

Prerequisites

AWS Account with bedrock-agentcore permissions
Python 3.10+
Boto3 installed
Model access: Anthropic Claude Sonnet 4.5 enabled in Bedrock console
Infrastructure from Option 1 setup (Gateway, Targets, IAM Role)

Deployment Steps

1. Configure environment variables:

# Copy template
cp .env.template .env

# Edit .env with your values from config.json
GATEWAY_URL=https://resource-optimizer-gateway-xxx.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp
COGNITO_CLIENT_ID=your_client_id
COGNITO_CLIENT_SECRET=your_secret
COGNITO_TOKEN_URL=https://your-domain.auth.us-east-1.amazoncognito.com/oauth2/token

2. Deploy to AgentCore Runtime:

# Option A: Use deploy script
./deploy.sh

# Option B: Manual deployment
pip install bedrock-agentcore-starter-toolkit
agentcore configure -e runtime_agent.py
agentcore launch

3. Test deployment:

# Quick test
agentcore invoke '{"prompt": "List my Lambda functions"}'

# Get agent ARN (from output or bedrock_agentcore.yaml)

4. Invoke from application:

import json
import boto3

client = boto3.client('bedrock-agentcore')

response = client.invoke_agent_runtime(
    agentRuntimeArn='arn:aws:bedrock-agentcore:us-east-1:ACCOUNT:agent/AGENT_ID',
    payload=json.dumps({"prompt": "Show me idle EC2 instances"}).encode()
)

# Process streaming response
for chunk in response.get("response", []):
    print(chunk.decode('utf-8'))

Runtime Management

View logs:

# Check CloudWatch Logs (location shown in deploy output)
aws logs tail /aws/bedrock-agentcore/agent/YOUR_AGENT_ID --follow

Update agent:

# Make code changes, then redeploy
agentcore launch

Delete runtime:

# Via CLI
aws bedrock-agentcore delete-agent-runtime --agent-runtime-arn YOUR_ARN

# Or use AWS Console

API Targets & Capabilities

CloudWatch Metrics (40 tools)

Purpose: Monitor AWS resource performance

Key Operations:

ListMetrics: Discover available metrics
GetMetricStatistics: Retrieve metric data points
GetMetricData: Query multiple metrics

Example Queries:

"Show me EC2 CPU utilization"
"List Lambda invocation metrics"
"Get RDS database read latency"

CloudWatch Logs (91 tools)

Purpose: Search and analyze logs

Key Operations:

DescribeLogGroups: List log groups
FilterLogEvents: Search logs with patterns
GetLogEvents: Retrieve log entries

Example Queries:

"Search Lambda logs for errors"
"Show recent log events for my application"
"Find API Gateway 5xx errors"

EBS (6 tools)

Purpose: Manage EBS volumes and snapshots

Key Operations:

ListSnapshotBlocks: List snapshot blocks
GetSnapshotBlock: Retrieve block data

Example Queries:

"List my EBS volumes"
"Show snapshot details"

Setup Guide

1. Create IAM Role

Script: setup/01-create-iam-role.py

What it creates:

Role name: ResourceOptimizerGatewayRole
Trust policy: Allows bedrock-agentcore.amazonaws.com
Permissions: CloudWatch, Logs, EBS, S3 access

Permissions granted:

{
  "CloudWatch": ["GetMetricStatistics", "ListMetrics", "GetMetricData"],
  "Logs": ["DescribeLogGroups", "FilterLogEvents", "GetLogEvents"],
  "EBS": ["ListSnapshotBlocks", "GetSnapshotBlock"],
  "S3": ["GetObject", "ListBucket"] // For Smithy specs
}

Run:

python3 setup/01-create-iam-role.py

2. Create AgentCore Gateway

Script: setup/02-create-gateway.py

What it creates:

Gateway with MCP protocol
Semantic Search enabled (intelligent tool selection)
Debug mode for detailed errors
Cognito OAuth authentication

Features:

protocolConfiguration = {
    "mcp": {"searchType": "SEMANTIC"}  // Critical for 137 tools
}

Run:

python3 setup/02-create-gateway.py

Output: Gateway URL stored in config.json

3. Create Smithy Targets

Script: setup/03-create-smithy-targets.py

What it creates:

CloudWatchTarget (371KB Smithy spec)
CloudWatchLogsTarget (645KB Smithy spec)
EBSTarget (92KB Smithy spec)

Smithy Specs Required:
Upload to S3 bucket cost-explorer-smithy-api:

cloudwatch-2010-08-01.json
cloudwatch-logs-2014-03-28.json
ebs-2019-11-02.json

Run:

python3 setup/03-create-smithy-targets.py
# Wait 60 seconds for tools to sync

Output: Target IDs stored in config.json

Configuration Reference

config.json Structure

{
  "aws": {
    "account_id": "339712964409",
    "gateway_role_arn": "arn:aws:iam::ACCOUNT:role/ResourceOptimizerGatewayRole",
    "region": "us-east-1"
  },
  "cognito": {
    "user_pool_id": "us-east-1_xxx",
    "client_id": "xxx",
    "client_secret": "xxx",
    "discovery_url": "https://cognito-idp.us-east-1.amazonaws.com/...",
    "token_url": "https://your-domain.auth.us-east-1.amazoncognito.com/oauth2/token"
  },
  "gateway": {
    "id": "resource-optimizer-gateway-xxx",
    "url": "https://resource-optimizer-gateway-xxx.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp",
    "name": "resource-optimizer-gateway"
  },
  "smithy_targets": [
    {
      "id": "XXXXX",
      "name": "CloudWatchTarget",
      "uri": "s3://cost-explorer-smithy-api/cloudwatch-2010-08-01.json"
    },
    {
      "id": "XXXXX",
      "name": "CloudWatchLogsTarget",
      "uri": "s3://cost-explorer-smithy-api/cloudwatch-logs-2014-03-28.json"
    },
    {
      "id": "XXXXX",
      "name": "EBSTarget",
      "uri": "s3://cost-explorer-smithy-api/ebs-2019-11-02.json"
    }
  ],
  "memory": {
    "id": "ResourceOptimizerMemory-xxx",
    "name": "ResourceOptimizerMemory"
  }
}

Environment Variables (AgentCore Runtime)

# Required
GATEWAY_URL=https://your-gateway.gateway.bedrock-agentcore.REGION.amazonaws.com/mcp
COGNITO_CLIENT_ID=your_client_id
COGNITO_CLIENT_SECRET=your_client_secret
COGNITO_TOKEN_URL=https://your-domain.auth.REGION.amazoncognito.com/oauth2/token

# Optional
AWS_REGION=us-east-1  # Defaults to us-east-1

Troubleshooting

Local Development Issues

Agent shows 0 tools

Cause: Targets not synced yet
Fix: Wait 60 seconds after running 03-create-smithy-targets.py

Permission denied errors

Cause: Missing IAM permissions
Fix:

# Add AgentCore permissions to your IAM user
aws iam put-user-policy --user-name YOUR_USER \
  --policy-name AgentCoreAdmin \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": "bedrock-agentcore:*",
      "Resource": "*"
    }]
  }'

Smithy spec too large error

Cause: Spec exceeds 2MB limit
Fix: We use specs under 2MB (CloudWatch 371KB, Logs 645KB, EBS 92KB)

AgentCore Runtime Issues

CodeBuild build error

Check:

View CodeBuild logs in AWS Console
Verify IAM permissions include CodeBuild access
Check requirements.txt is valid

Model access denied

Fix:

Enable Anthropic Claude Sonnet 4.5 in Bedrock console
Verify region matches (default: us-west-2 for runtime, us-east-1 for gateway)

Docker not found warning

Info: Can be ignored - CodeBuild doesn't need Docker unless using --local flag

Gateway Issues

list_gateway_targets returns 0 targets

Note: This API has known issues. Use get_gateway_target(targetId=X) instead

Tools not loading after target creation

Fix:

Wait 60 seconds for synchronization
Verify S3 bucket contains Smithy specs: aws s3 ls s3://cost-explorer-smithy-api/

Project Structure

aws-cost-optimization-agent/
├── agent.py                    # Local CLI agent
├── runtime_agent.py            # AgentCore Runtime agent
├── requirements.txt            # Python dependencies
├── config.json                 # Auto-generated configuration
├── .env.template               # Environment template
├── deploy.sh                   # Runtime deployment script
├── README.md                   # This file
├── COMPLETION-SUMMARY.md       # Project history
└── setup/
    ├── 01-create-iam-role.py       # IAM role creation
    ├── 02-create-gateway.py        # Gateway with semantic search
    └── 03-create-smithy-targets.py # All 3 API targets

Key Files

agent.py: Interactive CLI agent for local development
runtime_agent.py: Deployable agent for AgentCore Runtime
config.json: Auto-generated by setup scripts (not in git)
.env: Environment variables for runtime (not in git)

Technical Specifications

Performance

Startup: ~3 seconds (local), ~1 second (runtime)
Tool Loading: 137 tools (40 CloudWatch + 91 Logs + 6 EBS)
Response Time: 2-4 seconds for typical queries
Memory: Last 5 conversation turns (local), stateless (runtime)

Limits

Smithy Spec Size: 2MB maximum per target
Tool Name Length: 64 characters (Bedrock limitation)
Context Window: 200K tokens (Claude Sonnet 4.5)
Rate Limits: CloudWatch 400+ req/sec (high)

Technologies

Component	Technology	Version
LLM	Claude Sonnet 4.5	20250929
Agent Framework	Strands Agents	Latest
Gateway	AgentCore Gateway	MCP Protocol
Memory	AgentCore Memory	Short-term (30 days)
Auth	Amazon Cognito	OAuth 2.0
Runtime	AgentCore Runtime	Container-based

Security

Authentication Flow

Agent requests OAuth token from Cognito
Token passed in Authorization header to Gateway
Gateway assumes IAM role for AWS API calls
All communication over TLS

Best Practices

✅ Use least-privilege IAM policies
✅ Rotate Cognito client secrets regularly
✅ Enable CloudWatch logging for audit trail
✅ Use VPC endpoints for private access (optional)
✅ Review IAM role permissions quarterly

Resources

Support

For issues:

Check troubleshooting section above
Review CloudWatch Logs
Verify configuration in config.json
Check AWS service quotas

Status: ✅ Production Not Ready yet (Still facing some issues in runtime, once i solve it i will update the code and the post)
Last Updated: September 30, 2025
Total Tools: 137 across 3 API targets

Building Smart Learning Assistants with Amazon Nova: Why Hinting Is Better Than Answering

Mohamed Sorour — Fri, 16 May 2025 04:51:14 +0000

Have you ever had a teacher who simply provided you with the solutions to your homework? Yes,that was satisfying at the time, but did you really gain any knowledge? Most likely not.

That was the main obstacle we had to overcome in order to create an AI helper for RosettaCloud's interactive Labs. Our platform teaches shell programming and DevOps, and we wanted to develop an AI assistant that would do more than just solve problems for students—it should help them learn.

In this post, I'll describe how we used LangChain, Amazon Bedrock's Nova LLM, and RAG (Retrieval-Augmented Generation) system to create a hint-first instructional chatbot that have a knowledge base and a memory, Instead of depriving pupils of educational chances, our method guarantees that the assistant leads them toward discoveries.

The "Don't Just Tell Me" Philosophy
When learning technical skills like DevOps, Shell Scripting or Kubernetes, students need to develop troubleshooting abilities and pattern recognition. If an AI simply answers questions directly, it short-circuits the learning process.

Here is how our interactive-lab will look like:

For now lets focus on AI our AI assistant:

Here's our educational philosophy baked into our AI assistant:

First interaction: Provide a hint that guides the student toward the solution
Follow-up questions: Provide more direct assistance if the student is still stuck
Context-aware memory: Remember the conversation to provide increasingly targeted help
Nova's strength in context understanding and its ability to follow complex instructions made it perfect for this approach. Let's dive into how we built it.

Architecture Overview: The RosettaCloud Chatbot Flow

Our system follows a 7-step flow:

1- User asks a question in the Angular frontend
2- Question is sent via WebSockets to API Gateway
3- The AI Chatbot Lambda processes the request
4- Two parallel operations:
A- Fetch chat history from DynamoDB
B- Search for relevant documents in our LanceDB vector database
5- Amazon Bedrock's Nova processes everything
6- Response is streamed back to the client
7- Frontend renders the response with helpful references

This architecture gives us real-time interactions with context awareness, making the student experience feel like working with a patient tutor.

The Core Components

Angular Client side app

WebSocket Communication for Real-Time Responses

Students expect real-time help. We use WebSockets to create a streaming experience:

typescript
// Angular service for WebSocket communication
@Injectable({
  providedIn: 'root'
})
export class ChatbotService {
  private socket: WebSocket;
  private messageSubject = new Subject<ChatMessage>();

  constructor() {
    this.connectWebSocket();
  }

  private connectWebSocket() {
    // Connect to our API Gateway WebSocket endpoint
    this.socket = new WebSocket(environment.chatbotWebsocketUrl);

    this.socket.onmessage = (event) => {
      const data = JSON.parse(event.data);
      this.messageSubject.next(data);
    };
  }

  sendQuestion(question: string) {
    const payload = {
      prompt: question,
      session_id: this.getSessionId(),
      bedrock_model_id: 'amazon.nova-lite-v1:0',
      model_kwargs: {
        temperature: 0.7, 
        top_p: 0.9,
        max_tokens_to_sample: 512
      }
    };

    this.socket.send(JSON.stringify(payload));
  }

  getMessages(): Observable<ChatMessage> {
    return this.messageSubject.asObservable();
  }
}

The Lambda Backend: AI Chatbot with RAG

The heart of our system is the AI Chatbot Lambda which handles:

Processing the user's question
Retrieving relevant context from our knowledge base
Building a conversational RAG workflow
Streaming responses back to the client

Here's a simplified version of our ai_chatbot.py Lambda function:

def handle_message(event, context):
    connection_id = event.get('requestContext', {}).get('connectionId', '')
    api_endpoint = get_api_endpoint_from_event(event)

    body = json.loads(event.get("body", "{}"))
    prompt = body["prompt"]
    bedrock_model_id = body["bedrock_model_id"]
    model_kwargs = body["model_kwargs"]
    session_id = body["session_id"]

    try:
        # Initialize our streaming wrapper for Bedrock
        streamer = BedrockStreamer(connection_id, session_id, api_endpoint)

        # Create the RAG chain with LangChain
        conversation = streamer.create_rag_chain(
            LANCEDB_S3_URI, 
            KNOWLEDGE_BASE_ID, 
            bedrock_model_id, 
            model_kwargs
        )

        # Stream the response
        for response in streamer.stream_response(conversation, prompt):
            streamer.params["Data"] = response
            streamer.api_client.post_to_connection(**streamer.params)

        return {"statusCode": 200, "body": json.dumps("Success")}

    except Exception as e:
        print(f"Error: {str(e)}")
        # Error handling...

The BedrockStreamer class is where the magic happens:

Our chatbot system's foundation is the BedrockStreamer class, which manages the intricate interaction between human inquiries and AI answers. It establishes connections to Amazon Bedrock for AI processing and API Gateway for WebSocket communication. Fundamentally, this class builds the RAG (Retrieval Augmented Generation) pipeline by setting up the conversational components of LangChain, initializing vector retrievers from LanceDB, defining the "hint-first" instructional prompt method, and controlling the streaming response system. It manages important activities including using heartbeat messages to sustain WebSocket connections, adding instructional metadata to answers, and using DynamoDB to ensure session persistence. Instead of functioning as a unified teaching aid, the many elements would remain disjointed fragments in the absence of this overarching orchestration layer.


class BedrockStreamer:
    def __init__(self, connectionId, session_id, api_endpoint):
        self.api_client = boto3.client("apigatewaymanagementapi", 
                                       endpoint_url=api_endpoint)
        self.bedrock_client = boto3.client(service_name='bedrock-runtime', 
                                          region_name=BEDROCK_REGION)
        self.session_id = session_id
        self.doc_sources = []
        self.params = {
            "Data": "",
            "ConnectionId": connectionId
        }
        self.heartbeat_active = False

    def set_prompt(self, response_style="balanced"):
        # The key to our hint-first approach is in the system prompt
        system_prompt = (
            "You are an assistant specializing in DevOps. "
            f"Use a {response_style} style - be direct, clear and efficient. "
            "Use the following pieces of retrieved context to answer "
            "But do not include the context in your answer. "
            "You should not answer the question directly, but rather hint the user "
            "to the answer. if the user asked again, then you can answer. "
            "You should not answer any question not related to DevOps or SWE. "
            "If the user asks about something else, say that you are not able to "
            "answer that. If the user asks about DevOps, answer it. "
            "the question about DevOps. If you don't know the answer, "
            "say that you don't know. When showing code examples, "
            "format them properly with markdown syntax."
            "\n\n"
            "{context}"
        )

        # Conversation prompt for chat history context
        qa_prompt = ChatPromptTemplate.from_messages([
            ("system", system_prompt),
            MessagesPlaceholder(variable_name="chat_history"),
            ("human", "{input}"),
        ])

        return qa_prompt

Nova LLM: The Brain That Gets Educational Context

Amazon Bedrock's Nova model is particularly good at understanding the nuance between "give a hint" and "provide the answer." We configure it to provide educational guidance:

def create_rag_chain(self, lancedb_uri, knowledge_base_id, bedrock_model_id, model_kwargs):
    # Get our educational prompt template
    qa_prompt = self.set_prompt("educational")

    # Initialize vector retriever for our shell script knowledge base
    retriever = self.init_retriever(lancedb_uri, knowledge_base_id)

    # Initialize Bedrock LLM with Nova
    llm = BedrockChat(
        model_id=bedrock_model_id,
        model_kwargs=model_kwargs,
        streaming=True,
        client=self.bedrock_client
    )

    # Create chain components using LangChain
    history_aware_retriever = create_history_aware_retriever(llm, retriever)
    question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)
    rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)

    # Make it conversational with DynamoDB chat history
    conversational_rag_chain = RunnableWithMessageHistory(
        rag_chain,
        lambda session_id: DynamoDBChatMessageHistory(
            table_name=DYNAMO_TABLE,
            session_id=self.session_id
        ),
        input_messages_key="input",
        history_messages_key="chat_history",
    )

    return conversational_rag_chain

Stream response back via web sockets:

Angular show it to user:

Knowledge Base Indexing: Document Indexer Lambda

For our assistant to be effective, it needs access to high-quality learning materials. We built a document indexing system that processes shell scripts and extracts educational metadata.
Our chatbot system's knowledge base is created and maintained via the crucial background operation known as Document Indexing Flow.

The Document Indexer Lambda is automatically activated by an EventBridge trigger when shell scripts are uploaded to S3. The Lambda meticulously examines each script to collect educational metadata such as the question text, difficulty level, correct answers, and verification logic. The improved script representations are then converted into vector embeddings that capture semantic meaning using this Lambda using Amazon Bedrock's Titan Embeddings model. The contextual understanding of the chatbot is powered by a searchable knowledge base created by storing these vectors in LanceDB. Without human involvement, this automated pipeline guarantees that our chatbot always has access to up-to-date, correctly indexed instructional information, enabling the system to grow naturally as more learning resources are added.

def process_shell_script(text, filename):
    # Extract educational metadata from shell script comments
    lab_info = {
        'exercise_name': '',
        'difficulty': '',
        'learning_objectives': [],
        'question_number': '',
        'question': '',
        'question_type': '',
        'possible_answers': [],
        'correct_answer': '',
        'question_flag': '',
        'check_flag': ''
    }

    # Parse the shell script to extract question info
    # from comments and code blocks
    # ...

    # Create an enhanced representation that includes
    # educational context
    enhanced_text = f"""
SCRIPT NAME: {filename}

ORIGINAL CONTENT:
{text}

SUMMARY OF COMMENTS:
{chr(10).join(comments)}

MAIN COMMAND BLOCKS:
{chr(10) + chr(10).join([f"Block {i+1}:{chr(10)}{block}" for i, block in enumerate(command_blocks)])}

LAB QUESTION INFORMATION:
{formatted_lab_metadata}
"""

    return enhanced_text, lab_info

The Hint-First Approach in Action
Let's see how this actually works in practice. Here's a sample interaction:

Lessons Learned from Building with Nova

After implementing this system, we've learned several valuable lessons:

Nova does a remarkable job of adhering to subtle instructions and maintaining the "hint first" strategy. Streaming is crucial. Compared to waiting for full answers, the WebSocket method feels considerably more interactive.

We went through numerous iterations to find the ideal balance between being too helpful and being helpful. Careful prompt engineering matters. Context is king: The assistant is genuinely helpful when it combines conversation history and document retrieval.

Finding a balance between user delight and instructional value was the most challenging aspect. Students would become irritated if the helper was very restricted. They wouldn't learn if they were too helpful. We were able to achieve that balance because of Nova's adaptability.

The Significance of This for Education

There are two main types of traditional approaches to educational chatbots:

Basic Q&A bots that offer straightforward responses without any direction
Systems that are too tight and irritate kids

With Nova, our hint-first strategy establishes a compromise—a system that:

Promotes critical thinking
Offers scaffolded assistance
Increases directness gradually in accordance with student needs
Keeps in mind the context of the conversation to offer tailored assistance

This reflects the real job of effective teachers. They point students in the direction of answers rather than solving their difficulties.

Conclusion: AI in Education's Future

Developing educational AI differs greatly from developing a general-purpose helper. A well-designed RAG system and Amazon Nova's adaptability enabled us to develop something that genuinely improves learning rather than hinders it.

As we keep improving our system, we're investigating:

More advanced methods for knowledge retrieval
Progress monitoring to automatically modify hint levels
Integration with practical laboratory settings for instantaneous feedback

AI in education is not about substituting answer machines for instructors. The goal is to build assistants that, like the best human teachers, know when to guide, when to instruct, and when to hint.
Have you developed AI systems for education? Which strategies have you found work best for striking a balance between learning support and assistance? Leave a comment with your ideas!