DEV Community: Ved Prajapati

Architecting Enterprise RAG Systems on AWS

Ved Prajapati — Sat, 05 Sep 2026 09:50:49 +0000

Ved Prajapati
Principal Generative AI Architect | Founder, Vedaris

Abstract

Retrieval-Augmented Generation (RAG) enables generative AI systems to ground model responses in external knowledge rather than relying exclusively on information encoded within a foundation model. While basic RAG implementations can retrieve semantically similar text and provide it to a model, production systems introduce considerably more complex architectural challenges involving document ingestion, chunking, embeddings, retrieval quality, vector search, hybrid retrieval, scalability, security, and application integration.

This paper presents the architecture of an enterprise Retrieval-Augmented Generation system designed to process multi-format knowledge sources and provide semantically relevant information to generative AI applications. The system uses Amazon Bedrock and Titan Embeddings to generate vector representations, combines semantic vector retrieval with keyword-based search through a hybrid retrieval strategy, exposes application functionality through FastAPI, and deploys application workloads on Amazon EKS with a React frontend.

The architecture emphasizes retrieval quality rather than treating RAG as a simple vector similarity problem. Document preparation, intelligent chunking, embedding generation, retrieval, ranking, context construction, and generation are treated as separate architectural stages that can be independently optimized and evaluated.

The resulting design demonstrates how cloud-native infrastructure and modern information-retrieval techniques can be combined to create scalable, production-oriented RAG applications on AWS.

Keywords: Generative AI, Retrieval-Augmented Generation, RAG, Semantic Search, Hybrid Search, Embeddings, Vector Search, Amazon Bedrock, Titan Embeddings, Amazon EKS

Introduction

Foundation models possess powerful language understanding and generation capabilities, but their knowledge is constrained by their training data and model context.

Enterprise applications frequently require information that is:

organization-specific,
recently created,
frequently updated,
proprietary,
domain-specific,
unavailable during model training.

Retrieval-Augmented Generation addresses this limitation by retrieving relevant external information at inference time and supplying that information as context to a generative model.

A simplified RAG pipeline can be represented as:

Documents

↓

Chunking

↓

Embedding Generation

↓

Vector Index

↓

User Query

↓

Semantic Retrieval

↓

Context

↓

Foundation Model

However, production RAG systems require considerably more engineering than this simplified flow suggests.

Retrieval quality depends on how documents are segmented, how embeddings represent information, how queries are interpreted, how candidate passages are retrieved and ranked, and how the final context is constructed.

This project therefore approaches RAG as an information-retrieval architecture, rather than simply connecting a vector database to a foundation model.

Research Question

The primary question explored by the system is:

How can semantic retrieval, intelligent document chunking, embeddings, and hybrid search be combined to improve the quality and scalability of enterprise Retrieval-Augmented Generation systems on AWS?

Secondary questions include:

How does document chunking affect retrieval quality?
When does semantic retrieval outperform keyword matching?
When does keyword matching recover information missed by semantic retrieval?
Can combining both approaches improve retrieval quality?
How should retrieved context be prepared for downstream generation?
How can the resulting architecture scale as document and query volume increases?

System Objectives

The architecture was designed around several objectives.

Retrieval Quality

Relevant information should be retrieved even when user terminology differs from the source document.

Exact-Match Retrieval

The system should remain capable of retrieving exact identifiers, technical terminology, names, codes, and other lexical information.

Multi-Format Ingestion

The ingestion pipeline should accommodate different forms of enterprise information rather than assuming a single document structure.

Scalability

Retrieval and application workloads should support increasing document and request volumes.

Modularity

Ingestion, embedding, retrieval, generation, APIs, and frontend components should remain logically separated.

Production Deployment

The system should operate using cloud-native infrastructure rather than remaining a local RAG prototype.

Architecture Overview

The system consists of several logical stages:

Enterprise Documents

↓

Document Processing

↓

Intelligent Chunking

↓

Amazon Titan Embeddings

↓

Vector Representation / Indexing

↓

User Query

↓

Query Embedding

↓

Semantic + Keyword Retrieval

↓

Hybrid Search

↓

Relevant Context

↓

Amazon Bedrock / Foundation Model

↓

Generated Response

↓

FastAPI

↓

React Application

The application infrastructure is deployed using Amazon EKS, providing a scalable container orchestration environment for backend and application workloads.

Multi-Format Document Processing

Enterprise knowledge rarely exists in one standardized format.

Information may be distributed across:

PDF documents,
technical documentation,
source code,
structured data,
internal knowledge bases,
operational documentation.

Before information can participate in retrieval, it must be transformed into a representation suitable for indexing.

The ingestion pipeline therefore separates document extraction from retrieval preparation.

Each source is processed into textual or structured content while retaining relevant metadata.

Metadata can include information such as:

document_id
document_type
source
section
page
timestamp
chunk_id

Maintaining this relationship between retrieved content and its source is important for traceability and downstream citation.

Intelligent Chunking

Embedding entire documents as single vectors can produce representations that are too broad for precise retrieval.

Conversely, extremely small chunks can remove the context required to understand a passage.

Chunking therefore represents an important RAG design decision.

The system divides documents into smaller retrievable units while attempting to preserve meaningful context.

Possible approaches include:

Fixed-Size Chunking

Documents are divided according to predefined token or character boundaries.

Hierarchical Chunking

Smaller child chunks provide precise retrieval while maintaining relationships with larger parent contexts.

Semantic Chunking

Document boundaries are determined according to semantic changes rather than arbitrary character counts.

Different document types may benefit from different strategies.

Source code, structured documentation and long-form prose do not necessarily have the same optimal segmentation strategy.

Embedding Generation

After chunking, textual content is converted into vector representations using Amazon Titan Embeddings through Amazon Bedrock.

An embedding represents semantic characteristics of text as a numerical vector.

Conceptually:

"How do I rotate an API key?"
↓
Embedding Model
↓
[0.018, -0.291, 0.443, ...]

Document chunks undergo the same transformation.

Similarity between a query embedding and stored document embeddings can then be used to identify semantically related information.

This allows retrieval to identify conceptual similarity even when a user's wording differs from the source material.

Semantic Retrieval

Semantic retrieval searches according to meaning rather than exact lexical overlap.

For example, a query such as:

"How can credentials be changed safely?"

may be semantically related to documentation discussing:

"secret rotation procedures"

even though the phrases contain few identical words.

The user's query is converted into an embedding and compared against indexed document vectors.

The most similar candidates are returned according to a similarity measure.

This provides a major advantage over purely keyword-based retrieval for natural-language queries.

However, semantic retrieval also has limitations.

Keyword Retrieval

Certain enterprise queries depend heavily on exact terms.

Examples include:

API identifiers,
product names,
error codes,
version numbers,
function names,
technical acronyms.

Semantic similarity may not always prioritize these exact lexical matches.

Keyword retrieval therefore remains useful alongside vector search.

Rather than treating semantic retrieval as a complete replacement for traditional information retrieval, the system incorporates both approaches.

Hybrid Search

The retrieval architecture combines:

Semantic Search

and

Keyword Search

into a hybrid retrieval strategy.

Conceptually:

User Query

↙ ↘

Vector Search Keyword Search

↘ ↙

Candidate Results

↓

Combination / Ranking

↓

Final Context

Semantic search provides conceptual matching.

Keyword search provides lexical precision.

Combining them allows the system to benefit from both retrieval paradigms.

This is particularly valuable in technical enterprise environments where queries frequently contain a mixture of natural language and exact technical terminology.

Retrieval Ranking

Initial retrieval can return multiple candidate chunks.

Not every candidate should necessarily receive equal priority.

The retrieval pipeline can therefore introduce ranking or reranking before context reaches the foundation model.

A retrieval pipeline may take the form:

Initial Retrieval

↓

Candidate Documents

↓

Relevance Ranking

↓

Top-K Context

↓

Generation

This separates candidate discovery from final relevance selection.

The distinction becomes increasingly important as knowledge bases grow.

Context Construction

Retrieving relevant information does not automatically guarantee a good generated answer.

Retrieved passages must be assembled into useful model context.

The context-construction stage must consider:

relevance,
duplication,
ordering,
available context window,
source metadata,
conflicting information.

Too little context can omit important information.

Too much context can introduce irrelevant information and consume unnecessary model tokens.

The system therefore treats context construction as an independent optimization problem rather than simply forwarding every retrieved result to the model.

Generation with Amazon Bedrock

After retrieval, relevant context is supplied to a foundation model through Amazon Bedrock.

A simplified generation request contains:

System Instructions

Retrieved Enterprise Context

User Query

The model is instructed to produce an answer grounded in the supplied information.

This architecture separates knowledge retrieval from language generation.

The foundation model therefore does not need to permanently encode the enterprise knowledge being queried.

Knowledge can instead be updated independently through the retrieval system.

FastAPI Application Layer

The system exposes application functionality through a FastAPI backend.

The API layer provides a controlled boundary between frontend clients and the underlying RAG architecture.

Potential API responsibilities include:

receiving user queries,
validating requests,
invoking retrieval,
invoking generation,
formatting responses,
handling errors,
returning source metadata.

A conceptual request may take the form:

{
"query": "How does the deployment approval process work?"
}

The backend then executes the RAG pipeline and returns the generated response with relevant metadata.

React Frontend

A React frontend provides the user-facing application layer.

Separating the frontend from the RAG backend prevents retrieval and model logic from becoming tightly coupled to the interface.

The frontend can provide:

natural-language querying,
generated answers,
retrieved sources,
document references,
loading states,
error handling.

This architecture allows alternative clients to consume the same backend APIs in the future.

Deployment on Amazon EKS

Application workloads are deployed using Amazon Elastic Kubernetes Service (EKS).

Containerization provides consistent runtime environments, while Kubernetes provides mechanisms for:

service deployment,
scaling,
workload isolation,
health management,
rolling updates,
resource allocation.

The use of EKS also separates the application architecture from individual compute instances.

As request volume increases, application components can scale independently according to workload requirements.

Scalability

RAG systems contain several independently scalable components.

These include:

document ingestion,
embedding generation,
vector retrieval,
backend APIs,
model inference,
frontend delivery.

A production architecture should avoid assuming that these components scale at identical rates.

For example, embedding generation may be intensive during ingestion but minimal during normal querying.

Retrieval infrastructure may experience consistent query load.

Generation may represent the most expensive inference stage.

Separating these components allows resources to be scaled according to their individual workload characteristics.

Security Considerations

Enterprise RAG introduces security concerns beyond conventional application security.

Retrieved information may contain proprietary or sensitive organizational data.

The system should therefore consider:

identity and authentication,
authorization,
document-level access,
encryption,
secrets management,
API protection,
network isolation,
logging and auditing.

A particularly important requirement is ensuring that retrieval does not expose documents a user is not authorized to access.

RAG security therefore must extend beyond protecting the model endpoint.

Retrieval itself must respect authorization boundaries.

Evaluation Methodology

Retrieval quality should be measured rather than assessed exclusively through visual inspection of generated responses.

A representative evaluation dataset can contain:

user queries,
expected relevant documents,
expected relevant chunks,
expected answer information.

Different retrieval configurations can then be compared.

For example:

Configuration A

Semantic retrieval only.

Configuration B

Keyword retrieval only.

Configuration C

Hybrid retrieval.

Configuration D

Hybrid retrieval with reranking.

This enables objective comparison between retrieval strategies.

Evaluation Metrics

Useful metrics include:

Precision

The proportion of retrieved chunks that are actually relevant.

Recall

The proportion of relevant information successfully retrieved.

Precision@K

The relevance of the top K retrieved results.

Recall@K

Whether expected relevant information appears within the top K results.

Retrieval Latency

Time required to retrieve relevant context.

End-to-End Latency

Total time from user request to generated response.

Groundedness

The degree to which generated answers are supported by retrieved context.

Answer Relevance

The degree to which the generated answer addresses the user's query.

Together, these metrics provide a more meaningful view of RAG quality than simply evaluating whether an answer appears fluent.

Expected Trade-Offs

Improving retrieval quality can introduce additional computational cost and latency.

Hybrid retrieval requires multiple retrieval strategies.

Reranking introduces another processing stage.

Larger candidate sets may improve recall but increase downstream processing.

Larger chunks preserve context but may reduce retrieval precision.

Smaller chunks improve precision but risk removing surrounding meaning.

Production RAG architecture therefore involves balancing:

Quality

Latency

Cost

Context

Scalability

rather than optimizing one dimension in isolation.

Observability

Production RAG systems require visibility across the entire retrieval and generation pipeline.

Useful telemetry includes:

ingestion failures,
embedding latency,
retrieval latency,
retrieved document count,
retrieval scores,
model latency,
token consumption,
API errors,
end-to-end response time.

Retrieval traces can be especially valuable when debugging poor model responses.

If an answer is incorrect, engineers should be able to determine whether the failure originated from:

document processing,
chunking,
embedding,
retrieval,
ranking,
context construction,
generation.

Without this separation, retrieval failures can easily be mistaken for model failures.

Limitations

RAG does not guarantee factual correctness.

If the correct information is absent from the source corpus, retrieval cannot provide it.

If chunking separates important information incorrectly, retrieval quality may decline.

Embeddings may fail to represent certain domain-specific relationships.

Hybrid search increases architectural complexity.

Retrieved documents may themselves contain inaccurate or contradictory information.

Foundation models may still generate unsupported statements despite receiving relevant context.

These limitations demonstrate why RAG systems require evaluation at both the retrieval layer and generation layer.

Future Work

Future improvements to the architecture could include:

Hierarchical Retrieval

Retrieve precise child chunks while providing broader parent context to the model.

Semantic Chunking

Dynamically identify document boundaries according to meaning.

Query Decomposition

Break complex user questions into smaller retrieval tasks.

Reranking

Apply dedicated relevance models to improve candidate ordering.

Metadata Filtering

Restrict retrieval according to document type, organization, user permissions, date, or other metadata.

Agentic Retrieval

Allow an AI agent to iteratively determine when additional retrieval is required.

Retrieval Evaluation Pipeline

Continuously measure retrieval quality against representative enterprise queries.

Model Evaluation

Evaluate generated responses for relevance, groundedness and correctness.

Conclusion

This paper presented the architecture of an enterprise Retrieval-Augmented Generation system with semantic and hybrid search on AWS.

The system processes multi-format enterprise information through an intelligent document-processing and chunking pipeline, generates vector representations using Amazon Titan Embeddings, and combines semantic similarity with keyword retrieval to improve information discovery.

Relevant context is provided to foundation models through Amazon Bedrock, application functionality is exposed through FastAPI, user interaction is provided through a React frontend, and application workloads are deployed using Amazon EKS.

The architecture demonstrates that production RAG should not be treated simply as:

LLM + vector database

Instead, RAG is an end-to-end information-retrieval architecture involving:

ingestion → chunking → embeddings → indexing → retrieval → ranking → context construction → generation → evaluation

The quality of the final generative AI application depends on each of these stages.

By combining semantic retrieval, keyword matching, cloud-native infrastructure, and systematic evaluation, enterprise RAG systems can provide more relevant and grounded access to organizational knowledge while retaining the scalability and operational controls required for production environments.

Technology Stack

Generative AI: Amazon Bedrock
Embeddings: Amazon Titan Embeddings
Retrieval: Semantic Search, Vector Search, Hybrid Search
Compute & Orchestration: Amazon EKS
Backend: FastAPI
Frontend: React
Primary Domain: Generative AI, RAG, Semantic Retrieval, Enterprise AI

Author

Ved Prajapati
Principal Generative AI Architect | Founder, Vedaris
AWS Certified Generative AI Developer – Professional
Stanford CS234 · MIT 6.S191 · Harvard CS50x

Architecting Multi-Agent DevOps Systems on AWS

Ved Prajapati — Sat, 05 Sep 2026 09:46:45 +0000

Ved Prajapati
Principal Generative AI Architect | Founder, Vedaris

Abstract

The increasing reasoning capabilities of large language models (LLMs) create opportunities to extend DevOps automation beyond deterministic pipelines toward systems capable of interpreting context, delegating tasks, evaluating outputs, and coordinating decisions. However, relying on a single autonomous agent introduces challenges in task specialization, context management, reliability, security, and workflow control.

This paper presents the architecture and implementation of an autonomous multi-agent DevOps automation platform designed to coordinate specialized artificial intelligence agents across code review, security analysis, and deployment workflows. The system uses LangGraph to provide stateful agent orchestration and inter-agent communication, the Groq API for model inference, AWS Lambda for serverless execution, Amazon DynamoDB for persistent workflow state, and Amazon EventBridge for event-driven workflow initiation.

Rather than assigning an entire DevOps lifecycle to a single general-purpose agent, the proposed architecture decomposes the workflow into specialized agents operating within a shared orchestration layer. Each agent is responsible for a defined domain and contributes its findings to the overall workflow state before subsequent actions are taken.

The architecture demonstrates how multi-agent specialization, stateful orchestration, and event-driven cloud infrastructure can be combined to create extensible AI-assisted DevOps workflows. It also identifies important production considerations including agent reliability, authorization boundaries, observability, failure recovery, human approval, model hallucination, and the appropriate boundary between probabilistic AI reasoning and deterministic automation.

Keywords: Generative AI, Agentic AI, Multi-Agent Systems, DevOps Automation, LangGraph, Large Language Models, AWS Lambda, Amazon DynamoDB, Amazon EventBridge, Cloud Architecture

Introduction

Modern DevOps environments rely extensively on automation. Continuous integration and continuous delivery pipelines can automatically compile software, execute tests, perform static analysis, provision infrastructure, and deploy applications.

Traditional automation, however, is predominantly deterministic. A predefined condition produces a predefined action. This model is highly effective for repeatable tasks but becomes more difficult to apply when a workflow requires interpretation of unstructured information, contextual reasoning, prioritization, or decisions that cannot easily be represented through fixed rules.

Generative AI introduces another layer of automation.

Large language models can analyze source code, interpret natural-language requirements, reason over security findings, summarize technical information, and generate recommendations. Connecting these capabilities to external tools allows an LLM to participate in workflows rather than simply generate text.

A single AI agent can perform several of these operations, but increasing the number and diversity of responsibilities assigned to one agent can produce architectural problems. A general-purpose agent must maintain more context, understand multiple domains, select between more tools, and determine which objectives should take priority.

This project investigates an alternative architecture based on specialized collaborating agents.

The proposed system divides DevOps responsibilities among agents responsible for:

code review,
security analysis,
deployment coordination.

A stateful orchestration layer coordinates these agents and controls the movement of information between them.

The objective is not to replace conventional CI/CD automation. Instead, the architecture explores how agentic reasoning can complement deterministic DevOps systems in areas where contextual analysis and decision-making provide additional value.

Research Question

This project investigates the following primary research question:

How can specialized LLM-powered agents be coordinated through a stateful, event-driven architecture to perform collaborative DevOps tasks involving code review, security analysis, and deployment decision-making?

Several secondary questions follow from this:

How should responsibilities be divided between specialized agents?
How should agents exchange information without creating uncontrolled dependencies?
How can workflow state persist across serverless executions?
Where should deterministic controls override autonomous model decisions?
How can agentic workflows be secured and observed in a production cloud environment?
What failure modes emerge when probabilistic AI components participate in deployment workflows?

System Objectives

The platform was designed around six primary architectural objectives.

3.1 Agent Specialization

Each agent should have a clearly defined responsibility rather than requiring one general-purpose model to reason about the entire DevOps lifecycle.

3.2 Stateful Coordination

Outputs generated by one agent should be available to subsequent agents through controlled shared workflow state.

3.3 Event-Driven Execution

Workflows should be capable of starting in response to system events instead of requiring a continuously running orchestration server.

3.4 Serverless Scalability

Compute resources should be invoked when required and scale according to workflow demand.

3.5 Failure Isolation

Failure of an individual agent should not necessarily invalidate or terminate the entire workflow.

3.6 Extensibility

Additional agents and workflow stages should be introducible without requiring a complete redesign of the system.

Architecture Overview

The system is composed of five principal components:

LangGraph — stateful multi-agent workflow orchestration.

Groq API — large language model inference used by AI agents.

AWS Lambda — serverless execution environment for workflow components.

Amazon DynamoDB — persistent storage for workflow state and execution metadata.

Amazon EventBridge — event-driven workflow initiation and integration.

At a conceptual level, the architecture follows the flow:

DevOps Event
↓
Amazon EventBridge
↓
AWS Lambda
↓
LangGraph Orchestrator
↓
Code Review Agent
↓
Security Analysis Agent
↓
Deployment Agent
↓
Final Decision / Human Approval / Deployment Workflow

Throughout execution, relevant workflow state can be persisted within Amazon DynamoDB.

This separation allows the reasoning layer, orchestration layer, execution layer, persistence layer, and event layer to remain logically distinct.

Multi-Agent Architecture 5.1 Code Review Agent

The Code Review Agent analyzes source-code changes and identifies potential software-quality issues.

Its responsibilities can include:

reviewing modified code,
identifying potential defects,
detecting maintainability concerns,
evaluating implementation patterns,
generating structured review findings,
forwarding relevant findings to subsequent agents.

Rather than producing unrestricted natural-language output, the agent can return structured information containing fields such as severity, affected component, explanation, and recommended action.

Structured outputs make the information easier for other agents and deterministic systems to consume.

5.2 Security Analysis Agent

The Security Analysis Agent evaluates changes from a security perspective.

Potential responsibilities include:

identifying insecure coding patterns,
evaluating potentially exposed credentials or secrets,
examining authentication and authorization logic,
identifying dangerous configuration changes,
assessing security findings produced by external scanners,
assigning severity to identified risks.

The agent can consume both the original workflow context and findings generated by the Code Review Agent.

This demonstrates one of the central advantages of a coordinated multi-agent architecture: an agent does not need to independently rediscover every piece of context.

Instead, it can build upon information already generated elsewhere in the workflow.

5.3 Deployment Agent

The Deployment Agent evaluates whether the workflow is in an acceptable state to progress toward deployment.

Its inputs can include:

code-review findings,
security findings,
workflow metadata,
deployment policy,
previous agent decisions.

The Deployment Agent can produce a recommendation such as:

APPROVE

REJECT

REQUIRES_REVIEW

Importantly, an AI-generated approval should not automatically imply unrestricted production deployment.

High-risk operations should remain protected by deterministic authorization policies and, where appropriate, human approval.

This creates an important architectural boundary:

AI can reason about whether an action appears appropriate, while deterministic systems determine whether that action is actually permitted.

Stateful Orchestration with LangGraph

A multi-agent architecture requires more than several independent LLM calls.

The system must understand:

which agent executes next,
what information is available,
what previous agents concluded,
whether a workflow should continue,
whether a workflow should branch,
how failures should be handled.

LangGraph provides a graph-based abstraction in which agents and workflow operations can be represented as nodes connected through controlled transitions.

A simplified workflow can be represented as:

START

↓

Code Review

↓

Security Analysis

↓

Decision

↙ ↓ ↘

Reject — Human Review — Deployment

↓

END

The graph therefore becomes an explicit representation of the agentic workflow rather than relying on an LLM to dynamically invent the entire execution sequence.

This improves predictability.

Shared Workflow State

Agents require a mechanism for exchanging information.

A conceptual workflow state might contain:

workflow_id
repository
commit_id
changed_files
code_review_findings
security_findings
risk_level
deployment_recommendation
current_stage
execution_status

The Code Review Agent updates code_review_findings.

The Security Agent consumes those findings and updates security_findings and risk_level.

The Deployment Agent consumes the accumulated state and produces deployment_recommendation.

This approach avoids requiring agents to communicate through unrestricted natural-language conversations.

Instead, agents collaborate through controlled state transitions.

That distinction is important.

A multi-agent system does not necessarily require agents to behave like humans participating in a chat room. In many production systems, collaboration is more reliable when agents exchange structured state through an orchestration layer.

Event-Driven Execution with Amazon EventBridge

DevOps workflows are naturally event-driven.

Potential events include:

source-code commits,
pull-request updates,
completed builds,
security findings,
deployment requests,
infrastructure changes.

Amazon EventBridge provides an event-routing layer capable of connecting these events to downstream processing.

An event can trigger an AWS Lambda function responsible for initiating the appropriate agentic workflow.

Conceptually:

Repository / CI/CD Event

↓

EventBridge

↓

Lambda Invocation

↓

LangGraph Workflow

This architecture reduces the requirement for permanently running orchestration infrastructure and allows individual workflows to execute when relevant events occur.

Serverless Execution with AWS Lambda

AWS Lambda provides the execution environment for serverless components of the platform.

This model provides several architectural benefits:

event-driven invocation,
automatic scaling,
reduced infrastructure management,
isolation between executions,
integration with AWS event services.

However, serverless architecture also introduces constraints.

Agentic workflows can involve multiple model calls and potentially long reasoning cycles. Execution duration, retries, concurrency, external API latency, and state persistence therefore need to be considered when determining which workflow components should execute inside Lambda.

Long-running agent workflows may eventually require alternative orchestration or compute patterns.

The architecture should therefore treat Lambda as an execution mechanism rather than assuming every future agent workload must remain serverless.

Persistent State with Amazon DynamoDB

Serverless compute is ephemeral.

Persistent workflow information therefore requires an external state store.

Amazon DynamoDB can maintain information including:

workflow identifiers,
execution status,
agent outputs,
timestamps,
retry information,
approval status,
final workflow results.

Separating persistent state from individual Lambda executions also enables workflows to recover from interrupted or failed execution.

A workflow that fails during security analysis, for example, does not necessarily need to repeat every previous operation if the relevant state has already been persisted.

Model Inference

The agents use the Groq API for LLM inference.

Each specialized agent can be provided with:

a system-level role,
relevant workflow context,
task-specific instructions,
expected output structure,
constraints.

The Code Review Agent and Security Agent should not necessarily receive identical prompts.

Specialization allows prompts to be optimized around the responsibility of each agent.

This reduces the need for one extremely large system prompt containing every possible DevOps responsibility.

The architecture also keeps the orchestration layer logically separated from the inference provider.

This allows future implementations to evaluate different models or inference platforms without fundamentally changing the multi-agent workflow design.

Agent Communication

Inter-agent communication is implemented through the shared orchestration state.

Instead of Agent A directly controlling Agent B, the process becomes:

Agent A

↓

Structured Output

↓

Shared State

↓

Orchestrator

↓

Agent B

This provides greater control over information flow.

It also creates opportunities to validate agent output before another component consumes it.

For example, a schema-validation stage can reject malformed output before it reaches the Deployment Agent.

This introduces deterministic safeguards around probabilistic model behavior.

Reliability and Failure Handling

LLM-based systems introduce failure modes that do not normally appear in deterministic automation.

These include:

hallucinated findings,
inconsistent classifications,
malformed structured output,
contradictory agent conclusions,
unnecessary reasoning loops,
model API failures,
timeouts,
context-window limitations.

A production architecture should therefore assume that agent outputs can be incorrect.

Potential controls include:

Structured Output Validation

Agent responses should conform to defined schemas before being accepted into workflow state.

Retry Policies

Transient inference or service failures can be retried within controlled limits.

Maximum Iterations

Agent loops should have explicit execution limits.

Confidence or Severity Thresholds

Low-confidence or high-risk decisions can be escalated rather than executed automatically.

Human Approval

Production deployments and other sensitive operations can require explicit approval.

Deterministic Policy Enforcement

Security policies should not depend solely on LLM judgment.

Security Architecture

Introducing autonomous agents into DevOps pipelines increases the importance of security boundaries.

An AI agent should not automatically receive broad infrastructure privileges simply because it participates in an automated workflow.

The system should follow the principle of least privilege.

Each execution component should receive only the permissions required for its responsibilities.

Sensitive credentials should not be embedded directly inside prompts, source code, or agent memory.

Secrets should instead be obtained through appropriate secret-management mechanisms and exposed only to components that require them.

Production deployments should also maintain a separation between:

reasoning authority

and

execution authority.

An agent may recommend deployment, but the infrastructure executing that deployment should independently verify that required policies and approvals have been satisfied.

This prevents an incorrect model decision from automatically becoming an infrastructure action.

Observability

Traditional application monitoring is insufficient for sophisticated agentic systems.

A multi-agent platform should capture both infrastructure telemetry and AI workflow telemetry.

Useful metrics include:

workflow execution count,
agent invocation count,
inference latency,
workflow duration,
token consumption,
API failures,
retry count,
agent decision distribution,
deployment approval rate,
human escalation rate,
security finding severity,
workflow failure rate.

Tracing is particularly important.

If a deployment is rejected, operators should be able to determine:

which agent generated the relevant finding,
what input that agent received,
what output it generated,
how that output modified workflow state,
which subsequent decision was affected.

Agentic systems therefore require decision observability, not merely infrastructure observability.

Evaluation Methodology

A rigorous evaluation of the architecture should compare the multi-agent system against a baseline.

Two configurations can be evaluated:

Baseline A — Single-Agent Architecture

One general-purpose agent receives the entire task and performs code review, security reasoning, and deployment recommendation.

Architecture B — Specialized Multi-Agent System

Separate agents perform code review, security analysis, and deployment reasoning under LangGraph orchestration.

Both architectures should process the same test scenarios.

Potential scenarios include:

clean code change,
intentionally vulnerable code,
exposed credential,
dependency vulnerability,
failed test,
unsafe infrastructure configuration,
valid deployment,
ambiguous deployment requiring human review.

Evaluation Metrics

Several metrics can be collected.

Task Completion Rate

Percentage of workflows that reach an appropriate final state.

Security Detection Rate

Percentage of intentionally introduced security issues correctly identified.

False Positive Rate

Percentage of safe changes incorrectly classified as problematic.

Deployment Decision Accuracy

Percentage of deployment recommendations matching the expected decision.

End-to-End Latency

Time between workflow initiation and final decision.

Inference Cost

Total model usage required for each completed workflow.

Agent Invocation Count

Number of model calls required per workflow.

Human Escalation Rate

Percentage of workflows requiring manual intervention.

These measurements make it possible to evaluate whether additional multi-agent complexity produces measurable benefits.

Expected Trade-Offs

A multi-agent system is not automatically superior to a single-agent architecture.

Specialization may improve task focus and make workflows easier to reason about, but additional agents also introduce:

additional model calls,
increased latency,
increased inference cost,
more state transitions,
additional failure points,
greater orchestration complexity.

The central engineering question is therefore not:

Can this workflow use multiple agents?

It is:

Does using multiple specialized agents produce enough improvement in reliability, specialization, or maintainability to justify the additional complexity?

This distinction is essential when designing production agentic systems.

Human-in-the-Loop Control

Full autonomy is not appropriate for every DevOps operation.

Code analysis can tolerate a different level of autonomy than production deployment.

The architecture can therefore apply different autonomy levels according to risk.

Low risk: autonomous analysis and summarization.

Medium risk: autonomous recommendation with logged decisions.

High risk: AI recommendation followed by deterministic validation and human approval.

This creates a graduated autonomy model rather than treating AI automation as binary.

The objective is to maximize useful automation without removing accountability from high-impact operations.

Discussion

The architecture demonstrates that multi-agent systems can be designed as structured distributed workflows rather than collections of unrestricted conversational agents.

This is particularly important in DevOps.

Production systems require predictable state transitions, security boundaries, logging, retries, failure handling, and authorization.

LangGraph provides the agent orchestration layer, while AWS services provide event-driven execution and persistent infrastructure components.

The architecture therefore separates several concerns:

LLMs provide reasoning.

Agents provide specialization.

LangGraph provides workflow coordination.

AWS Lambda provides compute execution.

DynamoDB provides persistent state.

EventBridge provides event-driven integration.

Deterministic controls provide enforcement.

This separation makes the system easier to reason about and provides clear boundaries around autonomous behavior.

Limitations

The proposed architecture has several limitations.

First, LLM outputs remain probabilistic. Agent specialization does not eliminate hallucinations or incorrect reasoning.

Second, multi-agent workflows increase architectural complexity compared with conventional automation.

Third, additional model calls can increase both latency and cost.

Fourth, security analysis generated by an LLM should not replace established static-analysis, dependency-scanning, secret-scanning, or vulnerability-management tools.

Fifth, the effectiveness of the architecture depends heavily on prompt design, model capabilities, state representation, and evaluation quality.

Finally, production deployment requires significantly stronger governance than a prototype or research implementation.

Future Work

Several extensions could improve the platform.

Model Routing

Different models could be selected according to task complexity, latency requirements, and cost.

Agent Memory

Persistent memory could allow agents to incorporate previous workflow outcomes and organizational context.

Automated Evaluation

Agent outputs could be continuously evaluated against expected results and historical human decisions.

Retrieval-Augmented Agents

Agents could retrieve organizational coding standards, security policies, architecture documentation, and deployment procedures before making decisions.

Enhanced Guardrails

Additional controls could constrain agent inputs, outputs, available tools, and permitted actions.

Expanded Agent Roles

Future versions could introduce specialized agents for:

testing,
incident response,
infrastructure review,
cost optimization,
compliance,
documentation,
observability.
Advanced Observability

Distributed tracing could provide end-to-end visibility across agent decisions, model calls, state transitions, and AWS infrastructure events.

Conclusion

This paper presented an architecture for an autonomous multi-agent DevOps automation platform combining specialized AI agents with stateful orchestration and event-driven AWS infrastructure.

The system decomposes DevOps reasoning into specialized agents responsible for code review, security analysis, and deployment workflows. LangGraph coordinates agent execution and shared state, the Groq API provides model inference, AWS Lambda supplies serverless execution, Amazon DynamoDB maintains persistent workflow state, and Amazon EventBridge provides event-driven integration.

The architecture illustrates a broader principle for production agentic AI systems: autonomy should be structured rather than unrestricted.

Specialized agents can provide reasoning and domain-specific analysis, while orchestration systems define workflow boundaries and deterministic infrastructure controls retain authority over sensitive actions.

Multi-agent architectures therefore should not be viewed simply as a mechanism for increasing the number of LLMs participating in a task. Their value lies in decomposing complex reasoning into controlled responsibilities, coordinating those responsibilities through explicit state, and integrating probabilistic AI capabilities with reliable production infrastructure.

For DevOps automation, this creates a path toward systems that combine the contextual reasoning capabilities of generative AI with the reliability, security, scalability, and observability expected from modern cloud architecture.

Technology Stack

Agent Orchestration: LangGraph
Model Inference: Groq API
Compute: AWS Lambda
State Management: Amazon DynamoDB
Event Architecture: Amazon EventBridge
Primary Domain: Generative AI, Agentic AI, Multi-Agent Systems, DevOps Automation

Author

Ved Prajapati
Principal Generative AI Architect | Founder, Vedaris
AWS Certified Generative AI Developer – Professional
Stanford CS234 · MIT 6.S191 · Harvard CS50x

How I Cleared AWS Generative AI Developer Professional at 16

Ved Prajapati — Fri, 04 Sep 2026 12:23:57 +0000

Most 16-year-olds in the UK are currently memorizing slide decks for high school computing exams or building toy projects on localhost.

I took a different route. Over the past three years, I engineered production systems, founded Vedaris (https://vedaris.co.uk/), and earned over 50 industry credentials, culminating on August 31, 2026, by becoming the youngest person to pass the AWS Certified Generative AI Developer - Professional.

Here is an honest breakdown of the technical stack, the architecture trade-offs, and how treating systems design as leverage opens doors traditional schooling never mentions.

The Core Milestone: Youngest AWS Generative AI Developer - Professional

The AWS Certified Generative AI Developer - Professional is not a vocabulary test on prompt engineering. It is an intensive, scenario-driven examination of production AI systems: latency constraints, token economics, zero-trust security boundaries, and enterprise governance.

The exam scenarios focus on production realities:

Architecting sub-second vector search across multi-tenant vector stores while enforcing tenant-level KMS envelope encryption.

Preventing adversarial prompt injections, model poisoning, and toxic output using Amazon Bedrock Guardrails.

Orchestrating autonomous agentic workflows using ReAct frameworks and AWS Step Functions with deterministic fallback states.

Optimizing token throughput, cache lookups, and inferencing costs across foundation models, including Anthropic Claude, Amazon Titan, and fine-tuned open weights on Amazon SageMaker.

Earning this credential at 16 as the youngest holder required moving far beyond tutorials and building directly under production constraints.

The Real Stack: Beyond Just Writing Prompts

Real-world AI systems fail if the underlying network, container orchestration, and security layers are weak. Rather than treating GenAI as an isolated API, my stack connects machine learning to enterprise infrastructure:

Cloud Architecture and Foundations: AWS Solutions Architect - Associate, AWS Well-Architected Proficient, and Intel Cloud DevOps.

Specialized Container and Cloud Networking: Certified Calico Operator - AWS Expert (Tigera) for zero-trust Kubernetes network policies, AWS Amazon EKS, and Aviatrix Multicloud Network Associate.

Hands-On Performance Sandboxes: Verified AWS Demonstrated credentials across Agentic AI, MLOps, Serverless, Application Networking, Incident Response, and Data Lakehouse architectures.

Defensive Cybersecurity and Critical Infrastructure: OPSWAT Introduction to Critical Infrastructure Protection (ICIP), Cisco Ethical Hacker and Network Defense suites, and Google Cybersecurity Professional.

Theoretical Foundations: Harvard CS50x, deep learning through MIT 6.S191, and reinforcement learning via Stanford CS234.

Technical Projects: Production Architecture

Certifications prove baseline technical literacy; production code proves capability. Through Vedaris (https://vedaris.co.uk/), the focus is building scalable, decoupled, and secure infrastructure.

Multi-Agent DevOps Automation Platform:

Engineered an autonomous multi-agent system coordinating specialised AI agents for code review, security scanning, and deployment workflows.

Implemented an inter-agent communication protocol using LangGraph and Groq API for collaborative task execution.

Deployed serverless architecture on AWS Lambda with DynamoDB state persistence and EventBridge event-driven triggers.

Enterprise RAG System with Semantic Search:

Built a retrieval-augmented generation system processing multi-format documents (PDFs, code, structured data) with intelligent chunking.

Implemented an embedding pipeline using AWS Bedrock Titan Embeddings for high-performance semantic search.

Designed a hybrid retrieval strategy combining semantic similarity with keyword matching for precision and recall optimization.

Deployed on Amazon EKS with autoscaling, monitoring via CloudWatch, FastAPI backend, and React frontend.

Systems Leverage: The Non-Standard Route

In the UK, the conventional career advice is completely linear:
GCSEs -> A-Levels -> 3-Year Bachelor's Degree -> 100k-plus pounds in debt and living costs -> Junior Graduate Scheme

That pipeline made sense when access to compute and high-level training was locked inside universities. Today, that model carries massive financial debt (Plan 5 student loans with 40-year repayment windows) and severe opportunity costs.

My roadmap is built on systems arbitrage:

Work While Learning: Completing an apprenticeship to satisfy UK Raising the Participation Age mandates while logging enterprise commercial mileage.

Alternative Entry to Level 7: Leveraging professional certifications and real-world system delivery to enter an NCSC-certified Master's (MSc in Cyber Security) via alternative entry routes, bypassing an expensive, redundant Bachelor's degree entirely.

Enterprise Direct Contracting: Operating through Vedaris to deliver high-compliance cloud and AI infrastructure directly to clients.

Advice for Young Builders

Move Past Localhost Early: Toy scripts running on your laptop do not teach you about network timeouts, IAM permission boundaries, or API throttling. Build in the cloud under production constraints.

Learn Networking and Security: An LLM wrapper is easily commoditized. An engineer who understands Calico network policies on Amazon EKS, least-privilege IAM, and threat mitigation builds things companies actually pay to protect.

Question the Conveyor Belt: Educational systems are built for predictable compliance rather than rapid iteration. Find the leverage points, learn the actual rules, and ship production systems.

Connect with me on Credly: credly.com/users/ved-prajapati
Learn more about our enterprise architecture work at Vedaris: https://vedaris.co.uk/