DEV Community: Manikanta Suru

LLM Integration in CI/CD: Real Use Cases Beyond Code Completion

Manikanta Suru — Mon, 29 Jun 2026 23:09:22 +0000

A GitLab API token appeared in a README.md file. It was committed, pushed, and reviewed by two engineers before merging. Nobody caught it.

This was not a junior developer's mistake. Both reviewers were experienced. The token was on line 47 of a 200-line file, sandwiched between installation instructions. At the end of a sprint, with three other reviews queued, it looked like documentation. It was a secret sitting in plain sight.

We caught it the next day during a routine audit. By then it had been in the repository for 18 hours.

Three weeks later, after integrating an LLM into our Jenkins MR validation pipeline, the same pattern — a token embedded in documentation — would have been flagged automatically, before any human reviewer opened the diff. CRITICAL severity. Specific line. Suggested fix.

That single incident justified the integration. Everything else that followed confirmed it was the right architectural decision.

This article is about what we built, what we learned, and — more importantly — where LLM integration in CI/CD actually works versus where it creates more problems than it solves.

Why CI/CD Is the Right Integration Point

Before getting into specific use cases, it is worth being precise about why CI/CD pipelines are particularly well-suited for LLM integration.

LLMs are pattern recognizers operating on text. CI/CD pipelines generate enormous volumes of structured and semi-structured text at predictable moments: code diffs at merge time, build logs on failure, infrastructure plans before apply, deployment events, test results. This text is already being produced — the question is whether anything useful can be done with it beyond passing it to the next pipeline stage.

The answer, in specific cases, is yes. And the key word is specific.

The mistake most teams make when exploring LLM integration in CI/CD is treating it as a general capability rather than a targeted one. LLMs are not a replacement for deterministic tools — linters, static analyzers, test runners — that already exist in most pipelines. They are useful precisely where those deterministic tools fall short: contextual pattern recognition, natural language generation, and cross-cutting analysis that does not fit neatly into a rules-based system.

With that framing, here are five use cases that have worked in production.

Use Case 1: Automated MR/PR Code Review

What it does: Fetches the merge request diff, sends changed files to an LLM with a structured review prompt, and posts findings as inline comments directly on the MR — before any human reviewer opens it.

Implementation: A Jenkins pipeline triggers on every MR event via GitLab webhook. A Python script fetches the diff via GitLab API, sends each changed file to the LLM with file-type-aware prompts (separate prompts for Python, Terraform, JavaScript), and posts findings back to the MR via GitLab API.

The pipeline stage:

stage('AI Code Review') {
    when {
        expression {
            env.gitlabActionType == 'MERGE' ||
            env.gitlabActionType == 'UPDATE'
        }
    }
    steps {
        sh '''
            python3 scripts/claude_mr_review.py \
                --project-id ${gitlabMergeRequestTargetProjectId} \
                --mr-iid ${gitlabMergeRequestIid}
        '''
    }
    post {
        failure {
            echo 'AI review failed — pipeline continues'
        }
    }
}

The post { failure } block is non-negotiable. The AI review stage must never block a merge due to an LLM API failure or timeout.

What it catches at 100+ MRs per month:

Hardcoded credentials — API keys, database passwords, tokens embedded in code or documentation. Terraform security misconfigurations — security groups open to 0.0.0.0/0, unencrypted RDS instances, IAM wildcard policies. Missing error handling in async functions. Sensitive data written to application logs.

What prompt engineering actually required:

The first version produced 30+ findings per MR with no severity differentiation. Developers stopped reading the reviews within a week — which is a worse outcome than having no AI review. Three changes fixed it.

A hard maximum of 10 findings per review forces the model to prioritize. Mandatory severity classification (CRITICAL/HIGH/MEDIUM/LOW) lets developers triage without reading everything. File-type-specific prompts — Terraform reviews check different things than Python reviews — dramatically reduced irrelevant findings.

Cost consideration: At 100+ MRs per month with multiple files per MR, model selection matters significantly. A lightweight, high-throughput optimized model is fast and cost-efficient at this volume. Using a more capable reasoning model for automated code review is cost-inefficient — the task does not require it.

Use Case 2: Pipeline Failure Diagnosis

What it does: When a CI/CD pipeline fails, the failure logs are sent to an LLM that returns a plain-English diagnosis and suggested remediation steps — structured as an addition to the pipeline failure notification.

The problem it solves: Reading build logs is slow and requires context that shifts between engineers. A Kubernetes pod failure that surfaces as CrashLoopBackOff in the pipeline output requires pulling logs, reading describe output, and cross-referencing with recent changes. An experienced engineer can do this in 10–15 minutes. A less experienced engineer might take 45 minutes and still miss the root cause.

An LLM can read the same log output and return a structured diagnosis in under 30 seconds. Not always correct. Correct often enough to be the first thing an engineer checks before starting manual investigation.

What the output looks like:

DIAGNOSIS: OOMKill — container exceeded memory limit

The pod was killed by the kubelet due to memory exhaustion.
Log shows heap usage reaching 512Mi before termination,
which matches the configured memory limit.

ROOT CAUSE: JVM default heap settings (-Xmx not set)
allow heap to grow to 25% of available memory. The container
limit of 512Mi is insufficient for the application workload.

SUGGESTED FIX:
1. Increase memory limit:
   kubectl set resources deployment api-server --limits=memory=1Gi -n production
2. Or set JVM heap explicitly: JAVA_OPTS="-Xmx400m -Xms256m"
3. Check other replicas — same configuration affects all pods.

Implementation pattern: A post-failure webhook triggers a script that extracts the last 200 lines of pipeline logs and the specific failure stage output, sends them to the LLM, and appends the diagnosis to the Slack failure notification. Engineers receive context with the alert, not separately.

Use Case 3: Infrastructure Plan Review

What it does: Before terraform apply runs in the pipeline, the plan output is sent to an LLM for security and configuration review. Findings are posted as pipeline comments and optionally gate the apply stage for CRITICAL findings.

Why this matters: Terraform plan review is where the tired engineer problem is most acute. By the time an infrastructure change reaches the apply stage in a pipeline, it has typically been reviewed once — often quickly, often by someone with limited Terraform context. Security misconfigurations that are obvious in isolation become invisible in a 200-line plan output.

What it catches:

CRITICAL: aws_security_group.api — ingress rule allows 0.0.0.0/0
on port 5432 (PostgreSQL). Database should not be publicly accessible.
Fix: restrict to application security group ID only.

HIGH: aws_db_instance.main — storage_encrypted = false.
RDS instance will be created with unencrypted storage.
Fix: add storage_encrypted = true

MEDIUM: aws_s3_bucket.logs — missing required tags
(Environment, Owner). Compliance requirement.

Gate vs inform: For CRITICAL findings, the pipeline can be configured to require manual approval before apply proceeds. For HIGH and below, findings are informational — the apply runs but the team is notified. This balance avoids blocking legitimate changes while ensuring CRITICAL security issues get human review.

Use Case 4: Release Notes Generation

What it does: After a release merge, commit messages, PR titles, and PR descriptions from the release branch are sent to an LLM that produces structured release notes in a consistent format.

The toil it eliminates: Writing release notes is a task that happens at the end of a sprint when everyone is tired and focused on the next one. Output quality is inversely proportional to how close it is to the deadline. Notes are inconsistent across teams. Important changes get buried. Minor changes get inflated.

An LLM generating release notes from structured commit data produces consistent output, categorizes changes correctly — features, fixes, breaking changes, infrastructure — and drafts the communication that goes to stakeholders.

Important caveat: LLM-generated release notes require human review before publishing. The model cannot know which changes are significant from a product perspective — it can only categorize and summarize what the commits describe. The value is in eliminating the drafting toil, not removing the judgment.

Use Case 5: Incident Runbook Generation

What it does: During an incident, an engineer provides the service name, symptoms, and environment. The LLM generates a structured runbook with immediate actions, investigation steps, escalation criteria, and rollback procedures.

Why during the incident, not after: Post-incident runbooks are valuable for future incidents. An LLM-generated runbook during the incident is useful for the current one — particularly for incidents involving services the on-call engineer did not build or has not operated recently.

The output is not authoritative — it is a starting point that reduces the cognitive load of structuring an investigation when pressure is highest. An experienced engineer reviewing the generated runbook will immediately identify what is relevant and what is not. That judgment still requires a human.

What Does Not Work — The Honest Assessment

Business context is invisible to the LLM. Every LLM integration in CI/CD operates on artifacts — diffs, logs, plans — without understanding why the code was written this way, what the product requires, or what technical debt exists in adjacent systems. Use cases that require this context will produce low-quality output regardless of prompt quality.

Alert fatigue is a real failure mode. An AI review that produces too many findings with insufficient signal-to-noise ratio will be ignored. Ignored automated output is worse than no automated output — it conditions engineers to dismiss signals from that source, including future signals that matter. Prompt engineering to maximize precision over recall is essential.

Cost compounds at scale. At 100+ MRs per month with multiple files per MR, the token economics of model selection are significant. Benchmarking the most cost-efficient model that produces acceptable output quality for each use case is engineering work worth doing before production deployment.

Security requires careful architecture. Code diffs, infrastructure plans, and pipeline logs can contain sensitive information — partially redacted secrets, internal hostnames, service account names. Sending these to external APIs requires explicit data classification and policy decisions. For organizations with strict data residency requirements, self-hosted model deployment may be necessary.

Prompt management is ongoing work. Prompts that work well for a Python/Django codebase need adjustment for a Go microservices architecture. Prompts that produce good results for a startup with 10 engineers need tuning for a team of 100. Treating prompts as static configuration written once is a mistake — they require the same iteration discipline as any other system component.

Architecture Patterns That Work

Where to integrate in the pipeline: Pre-merge is appropriate for code review and infrastructure plan review — the value is in catching issues before they land. Post-merge is appropriate for release notes generation — it needs the complete merge context. Triggered on failure is appropriate for pipeline diagnosis — it needs the actual failure output.

Failure handling: Every LLM integration point should be non-blocking by default. allow_failure: true in GitLab CI, post { failure } in Jenkins. External API availability should never gate a deployment.

Prompt management as code: Store prompts in version-controlled files alongside the pipeline code that uses them. Changes to prompts should go through the same review process as changes to pipeline logic. This enables rollback when prompt changes degrade output quality and provides an audit trail.

Model selection by use case:

Use Case	Recommended Model Type	Reasoning
Code review (high volume)	High-throughput optimized	Volume makes cost significant
Failure diagnosis	Reasoning-optimized	Accuracy matters more than cost
Terraform security review	Reasoning-optimized	Security context requires deeper reasoning
Release notes	High-throughput optimized	Low complexity, high frequency
Runbook generation	Reasoning-optimized	Quality matters most under pressure

Where This Is Heading

The use cases described above share a common characteristic: the LLM produces text that a human reviews and acts on. The next evolution — already underway — moves toward LLMs that take action directly: identifying a failing test, generating a fix, opening a PR, and requesting human approval before merge.

This agentic pattern changes the risk profile significantly. An LLM that writes a comment a human ignores has limited blast radius. An LLM that opens PRs and modifies infrastructure requires careful guardrails, approval workflows, and explicit scope limitation.

The engineering work of building those guardrails — defining what the agent can and cannot do, implementing approval gates, auditing actions — is substantial. Teams that have done the work of integrating LLMs into CI/CD as informational tools will have the foundation and the operational experience to take that next step safely. Teams that have not will be starting from scratch.

Conclusion

CI/CD pipelines have always had the data that LLMs need — code diffs, build logs, infrastructure plans, deployment events. What has changed is that LLMs can now do something useful with that data at production scale and at a cost that justifies the integration.

The use cases that work are specific: pattern recognition in code review, natural language generation for diagnosis and documentation, cross-cutting security analysis that does not fit rule-based tools. The use cases that do not work are equally specific: anything requiring business context, anything where false positive rate creates alert fatigue, anything where the blast radius of incorrect output is unacceptable without human review.

The engineering teams that will get the most value from LLM integration in CI/CD are the ones that start with a specific problem, instrument it carefully, measure the signal-to-noise ratio honestly, and expand from there.

The question is not whether LLMs belong in your CI/CD pipeline. For most engineering organizations, they already do. The question is whether you are being deliberate about which problems they are solving and honest about where they fall short.

I Added Claude to Our MR Pipelines. It Now Reviews Every Code Change Before Humans Do.

Manikanta Suru — Tue, 12 May 2026 17:32:42 +0000

Three incidents. Three things that passed human code review and shouldn't have.

A developer pushed an API key into a test repository. It sat committed in the git history for days before anyone noticed.

A Terraform merge request got approved with a security group ingress rule open to 0.0.0.0/0. It went to pre production.

A GitLab API token ended up hardcoded in a README.md file. The README. The file everyone reads. Merged without a flag.

I'm the sole DevOps engineer at a pre-seed energy startup. I support a 10+ person engineering team across backend, frontend, and Terraform infrastructure. We process 100+ merge requests every month. I can't be in every MR. And even when senior engineers reviewed, things slipped through — especially at the end of the week, especially under deadline pressure, especially on Terraform where most developers aren't fluent.

So I added a Claude AI review stage to our Jenkins MR validation pipeline. Now every merge request gets an automated AI review before any human sees it — posting comments directly on the changed code in GitLab, flagging security issues, checking standards, catching the things tired humans miss.

Here's exactly how it works.

The Architecture: Jenkins + GitLab + Claude

Our setup uses GitLab for source control and Jenkins for CI/CD. When a developer opens or updates a merge request in GitLab, a webhook fires and triggers our Jenkins pipeline. Inside that pipeline lives a file called Jenkinsfile-mr-validation — our dedicated MR validation pipeline. One of its stages runs the Claude AI review.

The flow:

Developer opens/updates MR in GitLab
          │
          ▼
GitLab webhook fires → Jenkins triggered
          │
          ▼
Jenkinsfile-mr-validation runs
          │
    ┌─────┴──────────────────────┐
    │  Stage 1: Lint & Validate  │
    │  Stage 2: Unit Tests       │
    │  Stage 3: Claude AI Review │ ← this is what we're building
    │  Stage 4: Security Scan    │
    └────────────────────────────┘
          │
          ▼
Claude fetches MR diff from GitLab API
          │
          ▼
Claude reviews changed files
          │
          ▼
Comments posted back to GitLab MR inline
          │
          ▼
Developer sees AI feedback before human reviewer arrives

The AI review stage never blocks a merge. It's informational — a first-pass that runs automatically so human reviewers can focus on what actually needs human judgment.

The Jenkinsfile Stage

Inside Jenkinsfile-mr-validation, the Claude review stage looks like this:

pipeline {
    agent any

    environment {
        ANTHROPIC_API_KEY = credentials('anthropic-api-key')
        GITLAB_TOKEN = credentials('gitlab-api-token')
    }

    stages {

        stage('Lint & Validate') {
            // ... existing stages
        }

        stage('Claude AI Review') {
            when {
                expression { env.gitlabActionType == 'MERGE' || 
                             env.gitlabActionType == 'UPDATE' }
            }
            steps {
                script {
                    sh '''
                        pip install anthropic python-gitlab --quiet
                        python3 scripts/claude_mr_review.py \
                            --project-id ${gitlabMergeRequestTargetProjectId} \
                            --mr-iid ${gitlabMergeRequestIid}
                    '''
                }
            }
            post {
                failure {
                    echo 'Claude review failed — continuing pipeline'
                }
            }
        }

        stage('Security Scan') {
            // ... existing stages
        }
    }
}

Two important details:

The when condition ensures this stage only runs on actual MR events — not on branch pushes or scheduled builds. No point reviewing code that isn't being merged.

The post { failure } block means the pipeline continues even if the AI review script crashes or the Anthropic API times out. A failed AI review should never block a deployment.

Credentials are stored in Jenkins credential store — anthropic-api-key and gitlab-api-token — never hardcoded. Ironically, hardcoded credentials were one of the problems this system was built to catch.

The Python Review Script

import anthropic
import gitlab
import argparse
import os
import sys

def get_mr_changes(gl_client, project_id, mr_iid):
    """Fetch changed files from GitLab MR."""
    project = gl_client.projects.get(project_id)
    mr = project.mergerequests.get(mr_iid)
    changes = mr.changes()
    return changes['changes'], mr

def detect_file_type(filename):
    """Route to correct review prompt based on file type."""
    if filename.endswith(('.tf', '.tfvars')):
        return 'terraform'
    elif filename.endswith(('.js', '.ts', '.jsx', '.tsx', '.vue')):
        return 'frontend'
    elif filename.endswith(('.py', '.java', '.go', '.rb')):
        return 'backend'
    return 'general'

def build_system_prompt(file_type):
    """Build file-type-aware review prompt."""

    base = """You are a senior software engineer doing a code review.
Analyze the diff and identify real issues only.

Format every finding as:
SEVERITY: [CRITICAL/HIGH/MEDIUM/LOW]
FILE: [filename]
ISSUE: [specific description]
SUGGESTION: [concrete fix]

Rules:
- Maximum 10 findings per review
- Only report genuine issues — no hypotheticals
- CRITICAL = security risk or data loss potential
- HIGH = bug or significant quality issue
- MEDIUM = code quality concern
- LOW = style or minor improvement
- If nothing significant found, say so clearly"""

    terraform_checks = """

Terraform-specific checks:
- Security groups with ingress open to 0.0.0.0/0 on any port
- Unencrypted RDS instances, EBS volumes, or S3 buckets
- IAM policies with wildcard actions or resources
- Resources missing required tags (Name, Environment, Owner)
- Sensitive outputs without sensitive=true
- Hardcoded credentials or tokens in any value"""

    frontend_checks = """

Frontend-specific checks:
- User input rendered without sanitization (XSS risk)
- Sensitive data written to localStorage or console
- Hardcoded API endpoints, keys, or tokens
- Missing input validation on form fields"""

    backend_checks = """

Backend-specific checks:
- Hardcoded credentials, API keys, or tokens
- Missing authentication or authorization checks
- SQL queries built with string concatenation
- Sensitive data written to logs
- Missing error handling in async operations"""

    if file_type == 'terraform':
        return base + terraform_checks
    elif file_type == 'frontend':
        return base + frontend_checks
    elif file_type == 'backend':
        return base + backend_checks
    return base

def review_file_with_claude(filename, diff, file_type):
    """Send file diff to Claude for review."""

    # Using Haiku — faster and cheaper than Sonnet or Opus
    # At 100+ MRs/month, cost compounds quickly
    # Haiku handles code review quality well for this use case
    client = anthropic.Anthropic(
        api_key=os.environ['ANTHROPIC_API_KEY']
    )

    # Truncate large diffs — stay within token limits
    if len(diff) > 8000:
        diff = diff[:8000] + "\n\n[diff truncated — file too large]"

    response = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=1500,
        system=build_system_prompt(file_type),
        messages=[{
            "role": "user",
            "content": f"Review this diff for {filename}:\n\n{diff}"
        }]
    )
    return response.content[0].text

def post_review_comment(mr, review_text, filename):
    """Post review as MR comment in GitLab."""
    comment = f"### 🤖 AI Review — `{filename}`\n\n{review_text}"
    comment += "\n\n*Automated review. Human approval still required.*"
    mr.notes.create({'body': comment})

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('--project-id', required=True)
    parser.add_argument('--mr-iid', required=True)
    args = parser.parse_args()

    gl = gitlab.Gitlab(
        url=os.environ.get('GITLAB_URL', 'https://gitlab.com'),
        private_token=os.environ['GITLAB_TOKEN']
    )

    try:
        changes, mr = get_mr_changes(
            gl, args.project_id, args.mr_iid
        )
    except Exception as e:
        print(f"Failed to fetch MR: {e}")
        sys.exit(0)  # Exit 0 — don't fail the pipeline

    reviewed = 0

    for change in changes:
        filename = change.get('new_path', '')
        diff = change.get('diff', '')

        # Skip deleted files and trivial changes
        if change.get('deleted_file') or len(diff) < 100:
            continue

        file_type = detect_file_type(filename)

        # Skip binary and unknown file types
        if file_type == 'general':
            continue

        review = review_file_with_claude(filename, diff, file_type)

        if review and 'no significant' not in review.lower():
            post_review_comment(mr, review, filename)
            reviewed += 1

    if reviewed == 0:
        mr.notes.create({
            'body': "## 🤖 AI Code Review\n\n"
                    "✅ No significant issues found in changed files.\n\n"
                    "*Human review still required before merging.*"
        })

    print(f"AI review complete. Reviewed {reviewed} files.")

if __name__ == "__main__":
    main()

What Claude Actually Catches

After running this across 100+ MRs per month, the pattern is clear.

The three incident types that originally motivated this have not recurred since deployment. Hardcoded credentials get flagged at CRITICAL severity before any human reviewer sees the MR. Terraform security groups with permissive ingress rules get caught in the diff before they reach production. Tokens in documentation files get flagged immediately.

Beyond those:

Missing error handling in async backend functions
Unvalidated user input in frontend code
Terraform resources missing required tags
Debug logging that prints sensitive data
IAM policies broader than necessary

What it misses:

Claude sees the diff — not the system. It doesn't know why this code was written this way, what the business logic requires, what the rest of the codebase looks like, or whether this change conflicts with something in a file that wasn't modified. Business logic bugs, architectural issues, and context-dependent decisions still need human eyes.

The Prompt Engineering That Actually Mattered

The first version was nearly useless.

Claude posted 30+ comments on every MR. A mix of CRITICAL security findings and extremely minor style suggestions — all formatted identically. Developers started dismissing everything. Which is worse than having no AI review at all.

Three changes fixed it:

Maximum 10 findings. Hard limit in the system prompt. Forces Claude to prioritize. A review with 5 real issues beats one with 25 where 20 are noise. After this change, developers started reading the reviews instead of closing them.

Severity classification. CRITICAL/HIGH/MEDIUM/LOW on every finding. Developers can triage. CRITICAL gets fixed before merge. LOW is optional. This one change turned AI review from background noise into a signal worth acting on.

File-type-specific prompts. Terraform security checks are different from Python code quality checks. A generic prompt applied to both produces generic results for both. Separate prompts per file type — with specific checks relevant to that language and context — made findings significantly more relevant.

The system prompt is where 80% of the value lives. Spend time there.

What Still Requires a Human

Claude reviews the diff. It doesn't understand why the code was written this way. It doesn't know what the product decision was. It can't see the technical debt in the rest of the codebase that this change interacts with. It can't tell whether this feature does what the ticket actually asked for.

Human code review is still required before every merge. Always. Without exception.

What changed is what human review focuses on. Before this system, reviewers spent attention on mechanical checks — did someone remember error handling, are there obvious security gaps, is this formatted correctly. Now Claude handles that pass. Human reviewers focus on business logic, architecture, and the decisions that actually require knowing the system.

That's the right division of labor.

Results After Running This in Production

Security incidents from MR review dropped to zero. The specific things that used to slip through — committed credentials, open security groups, tokens in documentation — haven't made it through since this system went live.

Junior developers are progressing faster. Consistent, detailed, patient feedback on every MR they open. No waiting for a senior engineer's attention. No inconsistency based on who reviews on what day. The AI explains the reasoning behind every suggestion — something a tired senior engineer at the end of a long review queue often skips.

Human reviews are more focused. When the mechanical checks are handled automatically, the conversation in human review shifts to what actually needs human judgment. Less back-and-forth on obvious issues. More time on architecture and business logic.

Setting This Up in Your Environment

What you need:

GitLab with MR webhooks configured to trigger Jenkins
Jenkins with the GitLab plugin installed
Anthropic API key stored as Jenkins credential: anthropic-api-key
GitLab personal access token (api scope) stored as: gitlab-api-token
Python 3.9+ available on your Jenkins agent

Start with post { failure } in your Jenkinsfile stage and sys.exit(0) in your Python script — ensure the AI review never blocks your pipeline when it fails. Build the safety net first. Tune the prompts second.

The system prompt is your highest-leverage investment. Write specific checks for your codebase, your language stack, and your team's standards. A generic prompt gives you generic results. The specificity is what makes it useful.

Credentials committed to repositories, security groups left wide open, tokens hardcoded in documentation — these are mechanical failures. They don't require judgment to catch. They require consistent attention that human reviewers, under pressure, sometimes don't have.

That's what this system does. The judgment calls still belong to the humans.

devops #jenkins #gitlab #ai #claude #codereview #cicd #python #anthropic

I Built 20 AI-Powered DevOps Tools Because I Got Tired of Doing This Stuff Manually

Manikanta Suru — Mon, 11 May 2026 17:47:57 +0000

I've been a DevOps/SRE engineer for 10+ years.
I've managed 50+ EKS clusters at Apple scale, built OTA firmware
pipelines for 300+ EV chargers, migrated 80 applications to AWS,
and been the sole infrastructure engineer at two energy startups
where I supported teams of 30-40 engineers alone.
In all of that time, certain tasks never stopped being painful.
47 CloudWatch alarms firing at 11pm — and you have to figure out
which 3 actually matter.
A pod CrashLoopBackOff at 2am — logs open, describe output open,
trying to diagnose while half asleep.
A Terraform plan before a production apply — tired, reviewing it
manually, knowing you'll miss something.
A weekly AWS bill spike — someone asks why, you dig through Cost
Explorer for 40 minutes.
I got tired of doing all of this manually. So I built AI agents
for all of it.

What I Built
devops-ai-toolkit — 20 open source AI-powered tools across
5 sections, built with Python and Groq LLaMA 3.3.
🔗 github.com/manekanttasuru/devops-ai-toolkit
Every tool came from a real problem. None of this is theoretical.

The Tools — By Section
Kubernetes (4 tools)

Pod Failure Analyzer — diagnoses CrashLoopBackOff, OOMKilled,
Pending pods automatically from logs + describe output
Cluster Upgrade Advisor — reads your EKS version, scans for
deprecated APIs, produces a prioritized upgrade plan
RBAC Auditor — scans all roles, bindings, service accounts,
flags dangerous permissions ranked CRITICAL/HIGH/MEDIUM/LOW
Network Policy Analyzer — maps pod coverage, finds unprotected
namespaces, generates suggested NetworkPolicy YAML

AWS (4 tools)

IAM Analyzer — flags wildcards, missing MFA, old access keys,
over-permissioned roles with risk scoring
Security Group Auditor — finds open ports to 0.0.0.0/0,
orphaned groups, adds remediation commands per finding
VPC Network Analyzer — maps full topology, flags IP exhaustion,
missing flow logs, generates ASCII topology diagram
Unused Resource Hunter — finds idle EC2s, unattached EBS,
unused Elastic IPs, estimates monthly waste in dollars

Terraform (4 tools)

Security Plan Reviewer — reads terraform plan output, flags
security issues, rates CRITICAL/HIGH/MEDIUM/LOW with HCL fixes
Drift Detector — runs terraform plan, classifies drift as
INTENTIONAL/ACCIDENTAL/CONCERNING, gives per-resource recommendations
State Analyzer — scans tfstate for orphans, sensitive values,
missing tags, resource age estimation
Compliance Checker — maps your Terraform against CIS/HIPAA/SOC2
with control numbers and compliance score percentage

Monitoring (4 tools)

Dashboard Generator — takes a service name and metrics,
generates complete Grafana dashboard JSON ready to import
Log Pattern Analyzer — reads CloudWatch or local logs,
ranks error patterns by frequency and severity
Grafana Alert Router — classifies P1-P4 severity, routes to
right team, posts directly to Slack via webhook
Anomaly Detector — queries Prometheus + CloudWatch, flags
unusual patterns before they cross alert thresholds

SRE (4 tools)

Incident Runbook Generator — takes service + symptoms,
produces structured runbook with exact commands
On-Call Handoff Generator — takes current system state,
writes clean handoff brief for incoming engineer
Deployment Risk Scorer — rates LOW/MEDIUM/HIGH/CRITICAL
with go/no-go checklist per change type
Chaos Engineering Planner — generates full experiment plan
with hypothesis, steps, rollback, safety constraints

Stack
LLM: Groq API — LLaMA 3.3-70b-versatile (fast, free tier available)
Language: Python 3.9+
AWS: boto3
K8s: kubectl via subprocess
No heavy frameworks — each tool is a single Python file

Quick Start
bashgit clone https://github.com/manekanttasuru/devops-ai-toolkit
cd devops-ai-toolkit
pip install -r shared/requirements.txt
export GROQ_API_KEY=your_key_here

Run any tool — example:

cd kubernetes/pod-failure-analyzer
python main.py
Get a free Groq API key at console.groq.com

Why Groq + LLaMA
Fast enough for real-time infrastructure tooling. Free tier is
generous for experimentation. LLaMA 3.3 handles technical DevOps
context well. I use Groq in production for my other AI projects
too — MANI AI and BabyMind AI.

Every tool has a README with example output so you know what
you're getting before you run it.
If you find it useful — a star helps others find it.
If something is broken or you have ideas — open an issue.
🔗 github.com/manekanttasuru/devops-ai-toolkit