Ved Prajapati
Principal Generative AI Architect | Founder, Vedaris
Abstract
The increasing reasoning capabilities of large language models (LLMs) create opportunities to extend DevOps automation beyond deterministic pipelines toward systems capable of interpreting context, delegating tasks, evaluating outputs, and coordinating decisions. However, relying on a single autonomous agent introduces challenges in task specialization, context management, reliability, security, and workflow control.
This paper presents the architecture and implementation of an autonomous multi-agent DevOps automation platform designed to coordinate specialized artificial intelligence agents across code review, security analysis, and deployment workflows. The system uses LangGraph to provide stateful agent orchestration and inter-agent communication, the Groq API for model inference, AWS Lambda for serverless execution, Amazon DynamoDB for persistent workflow state, and Amazon EventBridge for event-driven workflow initiation.
Rather than assigning an entire DevOps lifecycle to a single general-purpose agent, the proposed architecture decomposes the workflow into specialized agents operating within a shared orchestration layer. Each agent is responsible for a defined domain and contributes its findings to the overall workflow state before subsequent actions are taken.
The architecture demonstrates how multi-agent specialization, stateful orchestration, and event-driven cloud infrastructure can be combined to create extensible AI-assisted DevOps workflows. It also identifies important production considerations including agent reliability, authorization boundaries, observability, failure recovery, human approval, model hallucination, and the appropriate boundary between probabilistic AI reasoning and deterministic automation.
Keywords: Generative AI, Agentic AI, Multi-Agent Systems, DevOps Automation, LangGraph, Large Language Models, AWS Lambda, Amazon DynamoDB, Amazon EventBridge, Cloud Architecture
- Introduction
Modern DevOps environments rely extensively on automation. Continuous integration and continuous delivery pipelines can automatically compile software, execute tests, perform static analysis, provision infrastructure, and deploy applications.
Traditional automation, however, is predominantly deterministic. A predefined condition produces a predefined action. This model is highly effective for repeatable tasks but becomes more difficult to apply when a workflow requires interpretation of unstructured information, contextual reasoning, prioritization, or decisions that cannot easily be represented through fixed rules.
Generative AI introduces another layer of automation.
Large language models can analyze source code, interpret natural-language requirements, reason over security findings, summarize technical information, and generate recommendations. Connecting these capabilities to external tools allows an LLM to participate in workflows rather than simply generate text.
A single AI agent can perform several of these operations, but increasing the number and diversity of responsibilities assigned to one agent can produce architectural problems. A general-purpose agent must maintain more context, understand multiple domains, select between more tools, and determine which objectives should take priority.
This project investigates an alternative architecture based on specialized collaborating agents.
The proposed system divides DevOps responsibilities among agents responsible for:
code review,
security analysis,
deployment coordination.
A stateful orchestration layer coordinates these agents and controls the movement of information between them.
The objective is not to replace conventional CI/CD automation. Instead, the architecture explores how agentic reasoning can complement deterministic DevOps systems in areas where contextual analysis and decision-making provide additional value.
- Research Question
This project investigates the following primary research question:
How can specialized LLM-powered agents be coordinated through a stateful, event-driven architecture to perform collaborative DevOps tasks involving code review, security analysis, and deployment decision-making?
Several secondary questions follow from this:
How should responsibilities be divided between specialized agents?
How should agents exchange information without creating uncontrolled dependencies?
How can workflow state persist across serverless executions?
Where should deterministic controls override autonomous model decisions?
How can agentic workflows be secured and observed in a production cloud environment?
What failure modes emerge when probabilistic AI components participate in deployment workflows?
- System Objectives
The platform was designed around six primary architectural objectives.
3.1 Agent Specialization
Each agent should have a clearly defined responsibility rather than requiring one general-purpose model to reason about the entire DevOps lifecycle.
3.2 Stateful Coordination
Outputs generated by one agent should be available to subsequent agents through controlled shared workflow state.
3.3 Event-Driven Execution
Workflows should be capable of starting in response to system events instead of requiring a continuously running orchestration server.
3.4 Serverless Scalability
Compute resources should be invoked when required and scale according to workflow demand.
3.5 Failure Isolation
Failure of an individual agent should not necessarily invalidate or terminate the entire workflow.
3.6 Extensibility
Additional agents and workflow stages should be introducible without requiring a complete redesign of the system.
- Architecture Overview
The system is composed of five principal components:
LangGraph — stateful multi-agent workflow orchestration.
Groq API — large language model inference used by AI agents.
AWS Lambda — serverless execution environment for workflow components.
Amazon DynamoDB — persistent storage for workflow state and execution metadata.
Amazon EventBridge — event-driven workflow initiation and integration.
At a conceptual level, the architecture follows the flow:
DevOps Event
↓
Amazon EventBridge
↓
AWS Lambda
↓
LangGraph Orchestrator
↓
Code Review Agent
↓
Security Analysis Agent
↓
Deployment Agent
↓
Final Decision / Human Approval / Deployment Workflow
Throughout execution, relevant workflow state can be persisted within Amazon DynamoDB.
This separation allows the reasoning layer, orchestration layer, execution layer, persistence layer, and event layer to remain logically distinct.
- Multi-Agent Architecture 5.1 Code Review Agent
The Code Review Agent analyzes source-code changes and identifies potential software-quality issues.
Its responsibilities can include:
reviewing modified code,
identifying potential defects,
detecting maintainability concerns,
evaluating implementation patterns,
generating structured review findings,
forwarding relevant findings to subsequent agents.
Rather than producing unrestricted natural-language output, the agent can return structured information containing fields such as severity, affected component, explanation, and recommended action.
Structured outputs make the information easier for other agents and deterministic systems to consume.
5.2 Security Analysis Agent
The Security Analysis Agent evaluates changes from a security perspective.
Potential responsibilities include:
identifying insecure coding patterns,
evaluating potentially exposed credentials or secrets,
examining authentication and authorization logic,
identifying dangerous configuration changes,
assessing security findings produced by external scanners,
assigning severity to identified risks.
The agent can consume both the original workflow context and findings generated by the Code Review Agent.
This demonstrates one of the central advantages of a coordinated multi-agent architecture: an agent does not need to independently rediscover every piece of context.
Instead, it can build upon information already generated elsewhere in the workflow.
5.3 Deployment Agent
The Deployment Agent evaluates whether the workflow is in an acceptable state to progress toward deployment.
Its inputs can include:
code-review findings,
security findings,
workflow metadata,
deployment policy,
previous agent decisions.
The Deployment Agent can produce a recommendation such as:
APPROVE
REJECT
REQUIRES_REVIEW
Importantly, an AI-generated approval should not automatically imply unrestricted production deployment.
High-risk operations should remain protected by deterministic authorization policies and, where appropriate, human approval.
This creates an important architectural boundary:
AI can reason about whether an action appears appropriate, while deterministic systems determine whether that action is actually permitted.
- Stateful Orchestration with LangGraph
A multi-agent architecture requires more than several independent LLM calls.
The system must understand:
which agent executes next,
what information is available,
what previous agents concluded,
whether a workflow should continue,
whether a workflow should branch,
how failures should be handled.
LangGraph provides a graph-based abstraction in which agents and workflow operations can be represented as nodes connected through controlled transitions.
A simplified workflow can be represented as:
START
↓
Code Review
↓
Security Analysis
↓
Decision
↙ ↓ ↘
Reject — Human Review — Deployment
↓
END
The graph therefore becomes an explicit representation of the agentic workflow rather than relying on an LLM to dynamically invent the entire execution sequence.
This improves predictability.
- Shared Workflow State
Agents require a mechanism for exchanging information.
A conceptual workflow state might contain:
workflow_id
repository
commit_id
changed_files
code_review_findings
security_findings
risk_level
deployment_recommendation
current_stage
execution_status
The Code Review Agent updates code_review_findings.
The Security Agent consumes those findings and updates security_findings and risk_level.
The Deployment Agent consumes the accumulated state and produces deployment_recommendation.
This approach avoids requiring agents to communicate through unrestricted natural-language conversations.
Instead, agents collaborate through controlled state transitions.
That distinction is important.
A multi-agent system does not necessarily require agents to behave like humans participating in a chat room. In many production systems, collaboration is more reliable when agents exchange structured state through an orchestration layer.
- Event-Driven Execution with Amazon EventBridge
DevOps workflows are naturally event-driven.
Potential events include:
source-code commits,
pull-request updates,
completed builds,
security findings,
deployment requests,
infrastructure changes.
Amazon EventBridge provides an event-routing layer capable of connecting these events to downstream processing.
An event can trigger an AWS Lambda function responsible for initiating the appropriate agentic workflow.
Conceptually:
Repository / CI/CD Event
↓
EventBridge
↓
Lambda Invocation
↓
LangGraph Workflow
This architecture reduces the requirement for permanently running orchestration infrastructure and allows individual workflows to execute when relevant events occur.
- Serverless Execution with AWS Lambda
AWS Lambda provides the execution environment for serverless components of the platform.
This model provides several architectural benefits:
event-driven invocation,
automatic scaling,
reduced infrastructure management,
isolation between executions,
integration with AWS event services.
However, serverless architecture also introduces constraints.
Agentic workflows can involve multiple model calls and potentially long reasoning cycles. Execution duration, retries, concurrency, external API latency, and state persistence therefore need to be considered when determining which workflow components should execute inside Lambda.
Long-running agent workflows may eventually require alternative orchestration or compute patterns.
The architecture should therefore treat Lambda as an execution mechanism rather than assuming every future agent workload must remain serverless.
- Persistent State with Amazon DynamoDB
Serverless compute is ephemeral.
Persistent workflow information therefore requires an external state store.
Amazon DynamoDB can maintain information including:
workflow identifiers,
execution status,
agent outputs,
timestamps,
retry information,
approval status,
final workflow results.
Separating persistent state from individual Lambda executions also enables workflows to recover from interrupted or failed execution.
A workflow that fails during security analysis, for example, does not necessarily need to repeat every previous operation if the relevant state has already been persisted.
- Model Inference
The agents use the Groq API for LLM inference.
Each specialized agent can be provided with:
a system-level role,
relevant workflow context,
task-specific instructions,
expected output structure,
constraints.
The Code Review Agent and Security Agent should not necessarily receive identical prompts.
Specialization allows prompts to be optimized around the responsibility of each agent.
This reduces the need for one extremely large system prompt containing every possible DevOps responsibility.
The architecture also keeps the orchestration layer logically separated from the inference provider.
This allows future implementations to evaluate different models or inference platforms without fundamentally changing the multi-agent workflow design.
- Agent Communication
Inter-agent communication is implemented through the shared orchestration state.
Instead of Agent A directly controlling Agent B, the process becomes:
Agent A
↓
Structured Output
↓
Shared State
↓
Orchestrator
↓
Agent B
This provides greater control over information flow.
It also creates opportunities to validate agent output before another component consumes it.
For example, a schema-validation stage can reject malformed output before it reaches the Deployment Agent.
This introduces deterministic safeguards around probabilistic model behavior.
- Reliability and Failure Handling
LLM-based systems introduce failure modes that do not normally appear in deterministic automation.
These include:
hallucinated findings,
inconsistent classifications,
malformed structured output,
contradictory agent conclusions,
unnecessary reasoning loops,
model API failures,
timeouts,
context-window limitations.
A production architecture should therefore assume that agent outputs can be incorrect.
Potential controls include:
Structured Output Validation
Agent responses should conform to defined schemas before being accepted into workflow state.
Retry Policies
Transient inference or service failures can be retried within controlled limits.
Maximum Iterations
Agent loops should have explicit execution limits.
Confidence or Severity Thresholds
Low-confidence or high-risk decisions can be escalated rather than executed automatically.
Human Approval
Production deployments and other sensitive operations can require explicit approval.
Deterministic Policy Enforcement
Security policies should not depend solely on LLM judgment.
- Security Architecture
Introducing autonomous agents into DevOps pipelines increases the importance of security boundaries.
An AI agent should not automatically receive broad infrastructure privileges simply because it participates in an automated workflow.
The system should follow the principle of least privilege.
Each execution component should receive only the permissions required for its responsibilities.
Sensitive credentials should not be embedded directly inside prompts, source code, or agent memory.
Secrets should instead be obtained through appropriate secret-management mechanisms and exposed only to components that require them.
Production deployments should also maintain a separation between:
reasoning authority
and
execution authority.
An agent may recommend deployment, but the infrastructure executing that deployment should independently verify that required policies and approvals have been satisfied.
This prevents an incorrect model decision from automatically becoming an infrastructure action.
- Observability
Traditional application monitoring is insufficient for sophisticated agentic systems.
A multi-agent platform should capture both infrastructure telemetry and AI workflow telemetry.
Useful metrics include:
workflow execution count,
agent invocation count,
inference latency,
workflow duration,
token consumption,
API failures,
retry count,
agent decision distribution,
deployment approval rate,
human escalation rate,
security finding severity,
workflow failure rate.
Tracing is particularly important.
If a deployment is rejected, operators should be able to determine:
which agent generated the relevant finding,
what input that agent received,
what output it generated,
how that output modified workflow state,
which subsequent decision was affected.
Agentic systems therefore require decision observability, not merely infrastructure observability.
- Evaluation Methodology
A rigorous evaluation of the architecture should compare the multi-agent system against a baseline.
Two configurations can be evaluated:
Baseline A — Single-Agent Architecture
One general-purpose agent receives the entire task and performs code review, security reasoning, and deployment recommendation.
Architecture B — Specialized Multi-Agent System
Separate agents perform code review, security analysis, and deployment reasoning under LangGraph orchestration.
Both architectures should process the same test scenarios.
Potential scenarios include:
clean code change,
intentionally vulnerable code,
exposed credential,
dependency vulnerability,
failed test,
unsafe infrastructure configuration,
valid deployment,
ambiguous deployment requiring human review.
- Evaluation Metrics
Several metrics can be collected.
Task Completion Rate
Percentage of workflows that reach an appropriate final state.
Security Detection Rate
Percentage of intentionally introduced security issues correctly identified.
False Positive Rate
Percentage of safe changes incorrectly classified as problematic.
Deployment Decision Accuracy
Percentage of deployment recommendations matching the expected decision.
End-to-End Latency
Time between workflow initiation and final decision.
Inference Cost
Total model usage required for each completed workflow.
Agent Invocation Count
Number of model calls required per workflow.
Human Escalation Rate
Percentage of workflows requiring manual intervention.
These measurements make it possible to evaluate whether additional multi-agent complexity produces measurable benefits.
- Expected Trade-Offs
A multi-agent system is not automatically superior to a single-agent architecture.
Specialization may improve task focus and make workflows easier to reason about, but additional agents also introduce:
additional model calls,
increased latency,
increased inference cost,
more state transitions,
additional failure points,
greater orchestration complexity.
The central engineering question is therefore not:
Can this workflow use multiple agents?
It is:
Does using multiple specialized agents produce enough improvement in reliability, specialization, or maintainability to justify the additional complexity?
This distinction is essential when designing production agentic systems.
- Human-in-the-Loop Control
Full autonomy is not appropriate for every DevOps operation.
Code analysis can tolerate a different level of autonomy than production deployment.
The architecture can therefore apply different autonomy levels according to risk.
Low risk: autonomous analysis and summarization.
Medium risk: autonomous recommendation with logged decisions.
High risk: AI recommendation followed by deterministic validation and human approval.
This creates a graduated autonomy model rather than treating AI automation as binary.
The objective is to maximize useful automation without removing accountability from high-impact operations.
- Discussion
The architecture demonstrates that multi-agent systems can be designed as structured distributed workflows rather than collections of unrestricted conversational agents.
This is particularly important in DevOps.
Production systems require predictable state transitions, security boundaries, logging, retries, failure handling, and authorization.
LangGraph provides the agent orchestration layer, while AWS services provide event-driven execution and persistent infrastructure components.
The architecture therefore separates several concerns:
LLMs provide reasoning.
Agents provide specialization.
LangGraph provides workflow coordination.
AWS Lambda provides compute execution.
DynamoDB provides persistent state.
EventBridge provides event-driven integration.
Deterministic controls provide enforcement.
This separation makes the system easier to reason about and provides clear boundaries around autonomous behavior.
- Limitations
The proposed architecture has several limitations.
First, LLM outputs remain probabilistic. Agent specialization does not eliminate hallucinations or incorrect reasoning.
Second, multi-agent workflows increase architectural complexity compared with conventional automation.
Third, additional model calls can increase both latency and cost.
Fourth, security analysis generated by an LLM should not replace established static-analysis, dependency-scanning, secret-scanning, or vulnerability-management tools.
Fifth, the effectiveness of the architecture depends heavily on prompt design, model capabilities, state representation, and evaluation quality.
Finally, production deployment requires significantly stronger governance than a prototype or research implementation.
- Future Work
Several extensions could improve the platform.
Model Routing
Different models could be selected according to task complexity, latency requirements, and cost.
Agent Memory
Persistent memory could allow agents to incorporate previous workflow outcomes and organizational context.
Automated Evaluation
Agent outputs could be continuously evaluated against expected results and historical human decisions.
Retrieval-Augmented Agents
Agents could retrieve organizational coding standards, security policies, architecture documentation, and deployment procedures before making decisions.
Enhanced Guardrails
Additional controls could constrain agent inputs, outputs, available tools, and permitted actions.
Expanded Agent Roles
Future versions could introduce specialized agents for:
testing,
incident response,
infrastructure review,
cost optimization,
compliance,
documentation,
observability.
Advanced Observability
Distributed tracing could provide end-to-end visibility across agent decisions, model calls, state transitions, and AWS infrastructure events.
- Conclusion
This paper presented an architecture for an autonomous multi-agent DevOps automation platform combining specialized AI agents with stateful orchestration and event-driven AWS infrastructure.
The system decomposes DevOps reasoning into specialized agents responsible for code review, security analysis, and deployment workflows. LangGraph coordinates agent execution and shared state, the Groq API provides model inference, AWS Lambda supplies serverless execution, Amazon DynamoDB maintains persistent workflow state, and Amazon EventBridge provides event-driven integration.
The architecture illustrates a broader principle for production agentic AI systems: autonomy should be structured rather than unrestricted.
Specialized agents can provide reasoning and domain-specific analysis, while orchestration systems define workflow boundaries and deterministic infrastructure controls retain authority over sensitive actions.
Multi-agent architectures therefore should not be viewed simply as a mechanism for increasing the number of LLMs participating in a task. Their value lies in decomposing complex reasoning into controlled responsibilities, coordinating those responsibilities through explicit state, and integrating probabilistic AI capabilities with reliable production infrastructure.
For DevOps automation, this creates a path toward systems that combine the contextual reasoning capabilities of generative AI with the reliability, security, scalability, and observability expected from modern cloud architecture.
Technology Stack
Agent Orchestration: LangGraph
Model Inference: Groq API
Compute: AWS Lambda
State Management: Amazon DynamoDB
Event Architecture: Amazon EventBridge
Primary Domain: Generative AI, Agentic AI, Multi-Agent Systems, DevOps Automation
Author
Ved Prajapati
Principal Generative AI Architect | Founder, Vedaris
AWS Certified Generative AI Developer – Professional
Stanford CS234 · MIT 6.S191 · Harvard CS50x
Top comments (1)
Putting workflow state in DynamoDB rather than in the agents is the decision that makes this recoverable, and it is worth saying why explicitly: Lambda will time out mid-workflow eventually, and a run whose state lived only in a conversation is unresumable. Externalised state also gives you the audit trail for free, which matters more here than in most agent systems because these agents touch deployments. The part I would want spelled out is idempotency across the EventBridge path. At-least-once delivery plus an agent that decides an action means the same deployment step can be proposed twice from two deliveries of one event, and dedupe cannot live inside the agent - it needs a workflow-level key checked before the effect, not after. Separating code review and security analysis from deployment is the right boundary too, since the first two are advisory and only the third can break production.