arshad

Posted on Jul 23

How I Built a Serverless AI Agent to Protect Open-Source Maintainers from Burnout

#aws #python #serverless #ai

Open-source maintainers are burning out.

Every day, popular repositories are flooded with dozens of issues. While many are brilliant, a significant portion consists of incomplete bug reports, duplicate requests, or entitled and demanding comments. Sifting through this noise takes a massive mental toll.

To solve this, I built Maintainer Burnout Guard a completely serverless AI agent that automatically monitors public GitHub repositories, runs sentiment and completeness analysis on incoming issues, drafts context-aware replies, and delivers a clean executive digest straight to the maintainer's inbox.

Here is a deep dive into how I built it, the architecture, and the painful runtime lessons I learned along the way.

🏗️ The Architecture Layout

The entire stack is designed to be ultra-low-cost, zero-maintenance, and highly scalable. It runs natively on AWS using the AWS Serverless Application Model (AWS SAM):

Amazon EventBridge: Triggers an AWS Lambda function on a nightly cron schedule.
AWS Lambda (Python 3.12): The core agent orchestrator. It fetches recent issues via the GitHub API.
Amazon Bedrock (Nova Lite): Analyzes the text payload in parallel to score clarity, sentiment, and tone, while generating a context-aware technical response draft.
Amazon SES: Compiles the flagged items into a beautiful HTML email digest.
AWS SSM Parameter Store: Securely stores GitHub tokens (SecureString) and configuration paths.

🛠️ Core Code: The Analysis Engine

The heart of the agent is the analyze_issue module. It structures raw text data, wraps it in a comprehensive Few-Shot prompt template, and forces the LLM to output predictable JSON at the API protocol layer using the Amazon Bedrock Converse API:

def analyze_issue(issue: GitHubIssue, config: AppConfig) -> AnalysisResult:
    body_truncated = issue.body[:MAX_BODY_CHARS]
    user_text = _USER_PROMPT.format(title=issue.title, body=body_truncated)

    # Advanced exponential retry configuration to prevent API throttling
    retry_config = Config(
        retries={
            "max_attempts": 10,
            "mode": "standard"
        }
    )

    bedrock = boto3.client(
        "bedrock-runtime", 
        region_name=config.TARGET_REGION,
        config=retry_config
    )

    try:
        response = bedrock.converse(
            modelId=config.bedrock_model_id,
            system=[{"text": _SYSTEM_PROMPT}],
            messages=[{"role": "user", "content": [{"text": user_text}]}],
            inferenceConfig={"maxTokens": 512, "temperature": 0.0},
            additionalModelRequestFields={"responseStructure": {"type": "json"}}
        )
        text = response["output"]["message"]["content"]["text"].strip()
        data = json.loads(text)

        # ... validation and dataclass assembly logic ...

Hard-Earned Production Lessons

Building the logic locally was easy, but deploying it to live cloud infrastructure revealed several fascinating traps:

1. The Python Keyword Shadowing Crash

Early in development, I named my configuration folder structure lambda/ and a local notification module email/.

The Disaster: Python reserves lambda as a core keyword, and the AWS internal runtime environment imports a built-in library named email. Creating folders with these names overrode Python's standard library layout, causing the Lambda container to crash instantly on initialization with a mysterious ModuleNotFoundError: No module named 'email.parser'.
The Fix: Keep your code layout clean using unreserved namespaces like src/ and mailer/.

2. Mass Parallelism vs. API Rate Limits

Because network calls to Bedrock are I/O-bound, I implemented a ThreadPoolExecutor to handle chunks of issues concurrently.

The Disaster: When testing against a massive repository like microsoft/vscode, the script fired over 130 simultaneous API calls to Bedrock in a single millisecond. The endpoint immediately threw a ThrottlingException: Too many requests.
The Fix: I lowered the parallel max workers pool size and passed an explicit botocore.config.Config block configuring a 10-attempt exponential backoff strategy, pacing the requests gracefully.

3. Forcing JSON Without Markdown Fences

Even when told to return raw JSON, LLMs love wrapping outputs in markdown backticks (json ...).

The Disaster: The trailing characters broke json.loads(), triggering constant parsing errors.
The Fix: Passing "responseStructure": {"type": "json"} in the additionalModelRequestFields forces Amazon Nova to strip markdown indicators natively, ensuring the string remains highly parseable.

Try It Yourself!

The entire project is open-source, fully modularized, and ready to deploy to your own AWS account using a single terminal command.

Check out the full repository and setup instructions here:
👉 GitHub: i-arshii/maintainer-burnout-guard

If you are an open-source maintainer, what features would you add to this to protect your workflow? Let's discuss in the comments below!

DEV Community