DEV Community: Lilupa Karu

The AI Engineering Team: Rethinking the Software Development Lifecycle on AWS

Lilupa Karu — Fri, 04 Sep 2026 06:35:36 +0000

Artificial Intelligence is changing software engineering faster than most of us expected.

And recently, I found myself asking a question that wouldn't go away:

What if AI could be more than a coding assistant? What if it could become part of the engineering team? That’s where the idea of an AI Engineering Team starts to take shape.

This wasn't just a theoretical question for me. It came from conversations, research, experiments, and—most importantly—things I was seeing firsthand at work. I started exploring what an AI-driven engineering team could actually look like, and how it could help us deliver software faster, more accurately, and with greater confidence.

A few years ago, AI-assisted development meant generating a function, completing some code, or explaining an unfamiliar snippet.

Today, AI can generate features, write tests, review pull requests, analyse logs, create infrastructure, troubleshoot failures, and interact with many of the tools engineers use every day.

So, perhaps the question has changed.

If AI can participate across the Software Development Lifecycle, why are we still treating it as just a coding assistant?

I believe we're moving towards something different:

The AI Engineering Team

Not a team of AI agents replacing engineers. Not humans disappearing from the picture. Instead, humans and AI working together as one engineering team.

Engineers focus increasingly on intent, architecture, decisions, domain knowledge, governance, trade-offs and outcomes.

AI takes on more of the analysis, coding, testing, reviewing, deployment and operational work.

The real opportunity isn't simply giving engineers better AI tools.

It's rethinking the entire Software Development Lifecycle around human–AI collaboration.

And that's what we'll explore in this article—and how AWS can provide the foundation for making the AI Engineering Team a reality.

From AI-Assisted Development to AI-Assisted Software Delivery

For a while, the story of AI in software engineering was relatively simple.

AI helped developers write code.

You gave it a prompt, it suggested a function. You pasted in an unfamiliar piece of code, it explained it. You asked for a unit test, it generated one.

The developer was still firmly in the driver's seat.

The workflow looked something like this:

And honestly, that sounds pretty good from a developer's perspective, doesn't it?

AI takes away some of the repetitive work. Developers move faster. Code gets generated in seconds instead of minutes or hours.

But there's a bigger pattern emerging.

Technology doesn't usually stop at making one step faster. It eventually starts changing the entire process and AI is no different. We're now moving from AI-assisted development towards something much bigger:

AI-assisted software delivery.

Imagine starting with nothing more than a business requirement:

"Allow customers to select a flight, reserve the available seats, and hold the booking for a defined period before completing payment."

Traditionally, turning a simple requirement into working software takes a long journey.

A requirement lands. An engineer explores the architecture, identifies impacted services and APIs, designs the solution, writes infrastructure and application code, creates tests, troubleshoots failures, raises a pull request, passes security and quality checks, deploys, and finally validates the result.

That's a lot of engineering effort between "we need this capability" and "it's working."

Now imagine AI joining us throughout that journey.

The requirement becomes the starting point.

AI analyses it, understands the architecture and APIs, proposes the solution, generates the infrastructure, code and tests, runs them, investigates failures, fixes issues, creates the pull request, performs security and quality checks, deploys to development, validates the behaviour, and summarises the outcome for the engineer.

The engineer sets the intent. AI helps turn that intent into working software.

That's where the idea of an AI Engineering Team starts to become real.

That's a fundamental shift. We're no longer talking about AI helping a developer write code. We're talking about AI participating in the software delivery lifecycle.

And this is where things get really interesting.

The human engineer doesn't disappear. The engineer moves up the abstraction level. Instead of spending most of their time asking:

"How do I implement this?"

the engineer increasingly focuses on:

"What should the system do, why should it do it, and how do we know it is doing the right thing?"

That is a very different engineering role.

It requires a broader perspective—understanding architecture, business context, domain knowledge, security, reliability, trade-offs, governance and outcomes, rather than focusing primarily on writing code.

In other words, the future engineer may need to become less of a code producer and more of a system thinker, decision maker and AI orchestrator.

And that brings us to an important question:

If AI can participate in the entire software delivery lifecycle, what should the AI engineering team actually look like?

That's where the idea of The AI Engineering Team starts to become much more interesting.

This isn't necessarily a literal architecture that every organisation should implement. It's a way of thinking about how the responsibilities of software engineering could evolve.

The New Software Development Lifecycle

Traditionally, we think about the SDLC as:

AI doesn't remove these stages. Instead, it changes who performs the work inside each stage.

Let's look at each one.

1. Requirements → AI Product Analyst

The first challenge in software development isn't writing code. It's understanding what needs to be built.

Imagine a requirement as simple as:

"Customers should be able to use a discount coupon while booking an air ticket."

Sounds easy, right?

An experienced engineer immediately sees the hidden questions: What type of coupon? Who is eligible? Where is it defined? What happens when it's invalid? What if the coupon service is unavailable? How does it affect payments, refunds and reporting?

This is where an AI Product Analyst can step in.

Instead of starting with a vague requirement, the AI can analyse existing documentation, APIs, architecture, Jira history, code, configurations and even past incidents to uncover the missing context, identify ambiguities and turn the business idea into a clear, structured engineering specification with acceptance criteria.

From “what do we want?” to “here's exactly what we need to build.”

2. Architecture → AI Architect

This is where the AI Engineering Team starts to get really interesting.

AI can already turn a requirement into an architecture diagram in seconds. But let's be honest—drawing boxes and arrows isn't architecture.

A real architect has to think about scalability, availability, security, cost, performance, failure modes, existing systems, organisational constraints, and how the solution will evolve over time.

So imagine giving an AI architecture assistant an existing AWS environment and a new business requirement.

Instead of simply producing one diagram, it could propose several options:

Option A: API Gateway → Lambda → DynamoDB
Option B: API Gateway → ECS → Aurora
Option C: EventBridge → Step Functions → Lambda → DynamoDB

But the real magic isn't the diagrams.

It's what comes next:

"Here are the trade-offs, risks, costs and operational implications of each option—and here's why I recommend this one."

AI becomes the architecture analyst, exploring possibilities and surfacing trade-offs across the enterprise and domain context.

The architect still owns the decision.

AI doesn't replace the architect. It gives the architect a much more powerful thinking partner.

3. Development → AI Software Engineer

This is where AI has already begun changing the game.

It started with:

"Write this function."

Then:

"Build this feature."

Now we're moving towards something far more powerful:

"Implement this capability within the constraints of this architecture."

That's a completely different level of intelligence.

Imagine an AI that understands your codebase, architecture, APIs, infrastructure, database schemas, coding standards, tests, security policies and observability data.

It no longer needs to answer:

"How do I write this code?"

It can start answering:

"How should this change be implemented in this system?"

That's the shift from an AI coding assistant to an AI Software Engineer.

And as AI agents become better at understanding repositories, using developer tools, running tests and iterating on their own work, engineers will increasingly move from writing every line of code to orchestrating AI to solve larger engineering problems.

4. Testing → AI Test Analyst/ Engineer

Now imagine that same AI reviewing your pull request.

Today, it might tell you that your variable name could be better or that your code can be simplified.

Useful—but what if it understood the requirement, architecture, security policies and operational context behind the change?

It could ask:

Does it meet the requirement?
Does it violate our architecture?
Does it introduce security risks?
What happens when a dependency fails or times out?
Could it unexpectedly increase AWS costs?

Consider a simple example: a Lambda function performs a high-frequency DynamoDB scan. The code works. The tests pass. The PR looks perfectly fine.

But architecturally, it could be a disaster.

An AI reviewer with knowledge of the system could catch that before it reaches production.

That's when AI code review becomes something much more powerful.

It stops reviewing code and starts reviewing engineering decisions.

5. Deployment → AI Release Engineer

Deployment pipelines have become increasingly automated, but releasing software safely still involves a surprising number of decisions.

This is another area where an AI release agent could become part of the engineering team.

Before making a release decision, the AI Release Engineer could bring together a much broader picture—understanding the scope of the change, analysing test results and deployment history, identifying affected services, checking current production health, evaluating feature flags and potential customer impact, learning from previous incidents, and assessing known risks—to determine how safely and confidently the change can be released.

And instead of simply saying "deployment successful", it could provide an engineering recommendation:

"The change affects three services and introduces a new dependency. Automated tests have passed, but the affected service has experienced elevated latency in the previous two releases. Recommend a canary deployment to 10% of traffic with a 30-minute observation window."

That's a very different type of automation. The AI isn't simply executing a pipeline. It's helping engineers make release decisions and this leads to an important principle:

AI-assisted release governance

AI can analyse the evidence, identify risks and make recommendations—and, where appropriate, even execute predefined actions.

But the organisation still defines the boundaries.

Humans decide what AI is allowed to do, under what conditions, and when human approval is required. Because ultimately, the goal isn't to remove humans from the release process.

It's to give engineers a much better understanding of the release before they make the decision.

6. Operations → AI Site Reliability Engineer

If there’s one area where I believe AI could fundamentally reshape engineering teams, it’s production operations.

Imagine this: a deployment finishes at 9:02 AM. A few minutes later, latency starts creeping up. Error rates spike. An alarm fires. An engineer gets paged.

Then the investigation begins.

Someone opens CloudWatch. Someone checks the deployment. Another digs through logs. The team starts connecting clues across metrics, traces, alarms, events and telemetry—all trying to answer three questions:

What happened? Why did it happen? And what should we do next?

Now imagine something different.

An AI SRE is already watching.

It sees the deployment, correlates it with the sudden change in latency and errors, investigates the relevant telemetry, identifies the likely root cause and recommends the next action—all before the human engineer starts investigating.

For well-defined, low-risk scenarios, it could even remediate the issue automatically within predefined guardrails. For everything else, it stops and asks a human.

AI won't replace the SRE. It will become an always-on engineering partner—watching, investigating, learning and assisting while humans remain accountable for the decisions that matter.

The future isn't just about building smarter AI agents.

It’s about building trusted agents that know when to act, when to ask, and when to stop.

AWS as the Foundation for the AI Engineering Team

If we accept the idea that AI is becoming an active member of the software engineering team, the next question naturally follows:

Where does this AI engineering team actually live?

This is where AWS becomes particularly interesting.

AWS already provides many of the building blocks needed to turn this idea into a real, production-ready engineering system.

Imagine an engineer starting with a requirement. Instead of jumping straight into an IDE and manually working through every step, they interact with an AI Engineering Hub.

That hub becomes the orchestrator.

It understands the engineer's intent and brings together the right AI agents, organisational knowledge, engineering policies and technical tools to get the job done.

At the heart of this architecture sits Amazon Bedrock, providing access to powerful foundation models. Around it, Amazon Bedrock AgentCore gives AI agents the capabilities they need to operate in production—tools, identity, memory, observability and evaluation.

These agents can take on real engineering work: analysing requirements, reviewing architecture, generating code and infrastructure, running tests, troubleshooting failures, and interacting with AWS and enterprise systems.

But here's the important part: the real value isn't any single AWS service. It's how everything comes together.

AI agents connect with organisational knowledge, engineering tools and policies to become part of the delivery lifecycle. The engineer stays in control—making architectural decisions, applying judgement, governing the process and owning the outcome—while AI takes on more of the execution.

We're not simply adding an AI coding assistant to the existing SDLC anymore. We're starting to redesign the SDLC around AI.

AI Engineering Needs Guardrails

The moment AI moves beyond generating code and starts touching real systems, everything changes.

Imagine an AI agent that can read customer data, call APIs, modify infrastructure, deploy software, and access production systems.

At that point, it's no longer just an assistant.

It's a workload with power.

And power needs guardrails.

We need the same foundations we've always relied on in distributed systems—least privilege, identity, authentication, authorisation, auditability, isolation and observability.

But AI adds a new layer of questions:

What can the agent do? What can it see? Which tools can it use? Can it change its own permissions? And what happens when it makes the wrong decision?

That's why the AI Engineering Team isn't simply an AI problem.

It's an architecture, security and governance problem.

The SDLC of the Future

This is not about eliminating humans from the loop.

It is about putting humans where human judgement creates the most value.

The Real Shift: From Coding to Intent

It’s easy to think the biggest change coming to software engineering is simply better AI models.

I don't think it is.

The bigger change is the level at which engineers will work.

For decades, we've been moving further away from the machine:

Machine Code → Assembly → High-Level Languages → Frameworks → Cloud → Infrastructure as Code → AI-Assisted Development → Intent-Driven Engineering

At every step, we've removed more of the mechanical work from engineers. AI is simply taking that journey one step further.

Instead of saying:

"I need to write a Lambda function."

An engineer might increasingly say:

"I need a secure, reliable and cost-effective way to process these events."

The intent comes first. The implementation becomes a consequence.

That's also where I see the future of engineering teams heading. I don't see ten developers being replaced by ten AI agents. I see small, highly capable teams of engineers working alongside specialised AI agents, with humans focusing on architecture, decisions, domain knowledge, governance and outcomes, while AI takes on more of the analysis, coding, testing, reviewing, deployment and operational work.

The winners won't necessarily be the organisations with the most AI. They'll be the ones that figure out how to combine humans, AI, architecture and governance effectively.

And that's the real shift.

AI isn't just becoming another developer tool. It's becoming part of the software engineering operating model.

The next generation of engineering organisations won't simply be cloud-native—they'll increasingly be AI-native.

The future of software engineering isn't humans versus AI. It's what becomes possible when humans and AI engineer together.

AWS AI-DLC: Rethinking the SDLC in the Age of AI

Lilupa Karu — Wed, 04 Feb 2026 09:54:19 +0000

Lately, my day-to-day world in software engineering has shifted more toward architecture and leadership than hands-on delivery. But over the Christmas break, I had the chance to get closer to the ground again—offering some friendly guidance to help build a SaaS product from scratch. No legacy systems, no existing processes—just a blank canvas and a big idea.

A former colleague of mine, Edwin, who was deep in the build, asked a familiar question: “Where should I start? Do we do this the same way we always have—Agile, sprints, ceremonies?” I paused, thought it through, and said, “Honestly, no mate. You should try AI-DLC.” He laughed and replied, “I know it has something to do with AI… but what exactly is it?”

That question led to several long conversations, mapping AI-DLC onto his project and rethinking what modern software development could look like when AI isn’t just a supporting tool, but a first-class participant in how software is designed, built, and operated. This article is my attempt to share that thinking—for anyone ready to step outside the old playbook and explore what’s next.

What Is AWS AI-DLC?

AWS AI-DLC (AI-Driven Development Lifecycle) is an AI-native approach to software development that fundamentally rethinks how systems are designed, built, and operated. Instead of treating AI as a supporting tool used at isolated stages, AI-DLC embeds intelligence across the entire lifecycle—from ideation and requirements to implementation, testing, deployment, and operations. AI becomes an active participant in delivery, continuously assisting with decisions, execution, and optimisation.

Crucially, AI-DLC preserves a strong human-in-the-loop model. Engineers and architects define intent, apply domain context, validate outcomes, and retain ownership of production systems, while AI accelerates execution, reduces manual effort, and enforces consistency at scale. The result is a development lifecycle where humans focus on strategy and judgment, and AI handles orchestration—enabling teams to deliver faster, with higher quality and confidence.

AI-DLC deliberately keeps humans in the driver’s seat, with explicit approval gates for critical decisions, while AI accelerates the work that happens in between.
That’s why AI-DLC isn’t vibe coding. It delivers speed without sacrificing control.

How do we use AI-DLC?

Just as you interact with ChatGPT by asking a question, you can initiate an AI-DLC workflow by simply stating your intent. AI-DLC follows a simple yet powerful pattern—one that consistently repeats across every phase of the development lifecycle.

I want to start an AI-DLC project for an event management application.

From there, the AI takes over the orchestration. It automatically identifies whether the project is greenfield (new) or brownfield (existing), guides you through the appropriate lifecycle phases, asks clarifying questions to refine requirements, seeks explicit approval before advancing, and continuously tracks progress throughout the journey.

AI-DLC Core Framework

AWS has formally introduced the AI-DLC methodology and its core philosophy here. Let’s explore the key components that define it.

Inception Phase

This is the planning and definition stage where the project kicks off.

Roles: Product Owner, Developers, and AI.
Key Ritual: Mob Elaboration (Collaborative defining of requirements).
Process:
- Starts with a high-level Intent (Goal).
- Breaks down the intent into smaller working pieces (labeled Unit 1, Unit 2... Unit n).
Artefacts (Outputs):
- PRFAQs (Press Release / Frequently Asked Questions)
- User Stories
- NFRs (Non-Functional Requirements).
- Risks.

Construction Phase

This is the building and testing stage where the "Units" are turned into functional software components.

Roles:
- For Construction: Developers and AI.
- For Testing: Product Owner, Developers, and AI.
Key Rituals: Mob Construction and Mob Testing
Process:
- Involves iterative design and coding cycles.
- Transforms "Units" into "Bolts" (Bolt 1, Bolt 2... Bolt n).
Artefacts (Outputs):
- Domain Design, Logical Design, and Codes.
- Deployment Units that are: Secured, Instrumented, De-Risked, Tested, and Packaged.

Operation Phase

This is the deployment stage where the software goes live.

Roles: Product Owner, Developers, and AI.
Process: The finalized "Bolts" are moved into the production environment.
Artefacts (Outputs):
- Deployment Units active in the Production Environment.

The Five Critical Artefacts of AI-DLC

Imagine you’re building a modern event management platform—one that allows organisers to create events, sell tickets, manage capacity in real time, and control entry seamlessly on event day. This is how AI-DLC’s five core artefacts come together in practice.

Intent: The North Star

What it is
Intent captures the why—the business goal or outcome that anchors everything that follows. It is the single source of truth that AI uses to guide decomposition, prioritisation, and decision-making.

Example Intent

“Build a scalable event management application that enables organisers to create and manage events, sell tickets securely, control venue capacity, and handle fast, fraud-free entry on event day.”

This Intent becomes the guiding compass for AI throughout the lifecycle, ensuring every feature, design choice, and deployment decision traces back to this core objective.

Unit: The Self-Contained Value Block

What it is
A Unit is a cohesive, independently valuable slice of functionality. AI decomposes the Intent into Units that deliver measurable outcomes and can be built, tested, and deployed in isolation.

Example Units
From the event management Intent, AI may derive Units such as:

Event Creation & Configuration
Ticket Sales & Payments
Capacity & Availability Management
Event Day Entry & Check-In
Reporting & Analytics

Each Unit is loosely coupled, allowing teams to move fast without waiting on other components—perfect for parallel development and incremental releases.

Bolt: The High-Velocity Execution Cycle

What it is
Bolts represent the smallest execution loop in AI-DLC. They focus on rapid, tangible progress—measured in hours or days rather than traditional multi-week sprints.

Example Bolts (for the “Event Day Entry & Check-In” Unit)

Day 1: Implement QR code–based ticket validation
Day 2: Add offline check-in support for poor connectivity
Day 3: Introduce real-time entry count and capacity alerts

AI plans these Bolts to maximise delivery speed while keeping scope sharply focused, with humans validating quality and user experience.

Domain Design: The Business Logic Blueprint

What it is
Domain Design captures the heart of your business logic using domain-driven design principles—independent of infrastructure or deployment concerns.

Example Domain Model (Event Management)

Entities: Event, Ticket, Attendee, Venue
Value Objects: TicketType, CapacityLimit, EntryStatus
Aggregates: EventSession (enforcing capacity rules)
Domain Events: TicketPurchased, CapacityReached, AttendeeCheckedIn
Repositories: EventRepository, TicketRepository

AI then extends this into Logical Design, recommending architectural patterns (for example, event-driven processing for entry scans) and documenting decisions in Architecture Decision Records (ADRs) for human review and approval.

Deployment Units: Ready-to-Ship Deliverables

What it is
Deployment Units are fully operational, production-ready packages—combining application code, configuration, infrastructure definitions, and automated tests.

For the event platform, this could include:

Ticketing and payment services
Entry scanning APIs
Real-time capacity monitoring components
CI/CD pipelines and infrastructure templates

Each Deployment Unit is validated for scalability, reliability, and security—ensuring the system performs flawlessly even during peak event-day traffic.

Closing Thoughts

Edwin and I caught up again on a warm midsummer evening, beers in hand, somewhere in town. By then, his project was already past the halfway mark. What stood out wasn’t just the progress, but how smoothly it was moving. No long pauses, no painful rework—just steady momentum. Choosing AI-DLC over the usual Waterfall or even Agile approach paid off. Features were landing fast, working software was in users’ hands, and milestone after milestone quietly ticked itself off.

If I were starting a new software product today, I wouldn’t think twice—I’d use AI-DLC. Not because it’s trendy, but because it matches where the industry is clearly heading. Humans still set the vision, apply judgment, and take ownership of outcomes. AI simply accelerates the journey from idea to reality. To me, AI-DLC feels like the natural evolution of SDLC—one that leans into the future instead of trying to hold it back.

Enterprise Bedrock Agent for Retail Customer Success

Lilupa Karu — Fri, 18 Jul 2025 06:01:58 +0000

Amazon Bedrock has quickly become one of the most talked-about platforms in the software world—a buzzword echoing across boardrooms, tech meetups, and cloud architecture discussions alike. Given its growing popularity, there’s little need to cover the basics here. Instead, let me take you into a real-world conversation that brought this technology into a sharp, business-focused context.

Just a few months ago, I caught up with a friend of mine—a successful retail business owner—who casually brought up his curiosity about integrating generative AI into his operations.

“What an incredible time to embrace it,” I responded, sensing the potential spark.

He leaned in and shared his vision:

“I want a smart solution, something like a conversational agent that can understand a customer’s buying needs, streamline the sales process, and—most importantly—if the exact product isn’t available, it should intelligently recommend alternatives based on intent, price, and market trends.”

That’s when it hit me: this wasn’t just a chatbot—this called for an agentic AI system. A solution powered by GenAI that could reason, adapt, and take action with business context in mind.

We ended up spending the afternoon diving deep into possibilities over two strong black coffees. By the time the mugs were empty, we had mapped out a high-level solution tailored to his business needs—bridging customer intent, product intelligence, and market awareness in one streamlined flow.

Later that evening, I realized the conversation was worth documenting—not just from a solution standpoint, but through the lens of business value and architectural thinking. This article is the result—a walk through that thought process, focusing on how agentic GenAI systems with AWS Bedrock can unlock real, transformative outcomes.

Business Problem

A premium bags and shoes retailer, operating across both physical stores and online channels, faces a recurring challenge: the inability to meet customer expectations when a specific product is unavailable. Whether it's an item that's out of stock or a product that doesn't quite match the customer's style or preferences, the result is the same—missed sales opportunities and (and I guess) disappointed shoppers.

This issue arises frequently at the store level and in the digital storefront. Customers often walk away or abandon their carts simply because no suitable alternatives are presented in real time. The lack of intelligent, personalized recommendations at the moment of decision-making is leading to lost revenue and a decline in customer satisfaction.

How Agentic AI Solve This Problem

Enter Generative AI and Agentic AI—transformative technologies that can revolutionize how retailers respond to product availability challenges.

By leveraging Amazon Bedrock, businesses can deploy GenAI-powered agents capable of deeply understanding customer intent, context, and preferences in real time. When a specific product is unavailable, the agent doesn’t simply show generic alternatives—it intelligently curates suggestions that align with the customer’s style, color preferences, price range, and even previous purchase history.

These agents go beyond basic chatbot interactions. They think and act like digital sales associates, capable of reasoning across a product catalog, identifying patterns, and making personalized recommendations that feel human and thoughtful. They can explain why an alternative product is a good match, handle follow-up questions naturally, and even suggest complementary items—just like an expert salesperson would.

For example, if a customer is searching for a specific designer handbag that’s out of stock, the agent can dynamically recommend visually similar models, limited-edition pieces, or items trending with similar customer profiles—all within the same conversation.

This approach not only salvages potentially lost sales but also elevates the customer experience, making shoppers feel understood and supported. The result: increased conversion rates, stronger brand loyalty, and a modern retail experience that meets the expectations of today’s digitally-savvy consumers.

High-level Flow

To bring the GenAI-powered agent to life, during the thought process of the architecture I decided to utilise a combination of AWS Bedrock, Amazon OpenSearch, AWS API Gateway, Lambda and a few other key AWS services to enable a scalable, secure, and intelligent experience.

Architecture

Architecture Components

Customer Interaction Layer

Customer Chat Interface

Description: This is the front-end application (web, mobile, smart kiosk) where customers send their queries. I have not focused in detail on these front-end applications in this context, as our primary focus must center on the Agentic AI solution.
Role: Provides the user experience for interaction with the AI agent.

Amazon API Gateway

Description: A fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
Role: Acts as the single entry point for all customer queries. It handles request routing, authorization, rate limiting, and acts as a proxy to the back-end Lambda functions.
Security: Integrates with Amazon Cognito for user authentication and authorization.

Query Handler Lambda Function

Description: A serverless compute service that executes code in response to events.
Role: Receives the raw customer query from API Gateway. It can perform initial validation or pre-processing of the query before passing it to the next stage.
Scalability: Automatically scales with the number of incoming requests.

Core AI & Data Processing Layer

Query Ingestion SQS Queue

Description: A fully managed message queuing service that enables us to decouple and scale microservices, distributed systems, and serverless applications.
Role: Provides asynchronous processing of customer queries. This prevents the API Gateway from timing out for longer processing queries and decouples the ingestion from the processing, improving resilience.

Product Availability Check Lambda Function

Role: Consumes messages from the SQS queue. It queries the DynamoDB: Product Catalog/Inventory to check if the requested product is available. If unavailable, it extracts relevant keywords or product identifiers to initiate the search for alternatives.

Product Catalog/Inventory DynamoDB

Description: A fast and flexible NoSQL database service for single-digit millisecond performance at any scale.
Role: Stores the real-time product catalog and inventory information. This allows for quick lookup of product availability.

OpenSearch Query Handler Lambda

Role: If a product is unavailable, this function constructs a query for Amazon OpenSearch Service. It uses the extracted keywords and potentially context from the initial query to perform both semantic and vector searches.

Amazon OpenSearch Service

Description: A managed service that makes it easy to deploy, operate, and scale OpenSearch clusters. It supports both keyword-based search (semantic) and vector search (for similarity).
Role: Stores vectorized product descriptions, attributes, and potentially customer reviews. It's crucial for identifying relevant product alternatives based on semantic similarity to the unavailable product or the customer's desired attributes.
Scalability: Easily scales to handle large product catalogs and high query volumes.
Security: Deployed within a VPC for network isolation, and access controlled via IAM policies.

Amazon Bedrock Knowledge Base

Description: A fully managed capability in Amazon Bedrock that enables you to connect foundation models (FMs) to your company data sources for Retrieval Augmented Generation (RAG).
Role: This acts as the external knowledge source for the Bedrock Agent. The prepared contextual product data is ingested here. When the Bedrock Agent receives a request, it can query this Knowledge Base to retrieve relevant product information to augment its response. This is crucial for grounding the GenAI model's responses in factual, up-to-date product data, reducing hallucinations.

Amazon Bedrock Agent

Description: A new capability within Amazon Bedrock that allows you to create generative AI applications that can automate multi-step tasks by seamlessly connecting with company systems, APIs, and data sources.
Role: This is the core orchestrator of the GenAI solution.
- It receives the user query (potentially enhanced by initial processing).
- It understands the intent ("product unavailability, find alternatives").
- It orchestrates calls to the Bedrock Knowledge Base (via RAG) to get relevant product alternatives.
- It uses a Foundation Model (FM) (e.g., Anthropic Claude, Amazon Titan) to craft a natural, personalised, and persuasive response incorporating the retrieved product recommendations.
- Memory Retention: Bedrock Agents can maintain conversational context across interactions, leading to more human-like and continuous dialogues.
- Code Interpretation: Can be used for complex analytical queries or dynamic content generation if needed.

Response Formatter Lambda Function

Role: Receives the generated response from the Bedrock Agent. It can perform post-processing such as formatting the response for the chat interface, adding specific links, or integrating with other downstream systems. It then sends the final response back to the customer via API Gateway.

Data Storage & Pre-processing

Product Data Source Bucket

Description: Object storage built to retrieve any amount of data from anywhere.
Role: Serves as a data lake for storing raw product data, images, descriptions, and other unstructured or semi-structured information that will be used to generate embeddings for OpenSearch. It can also store chat logs for analytics.

Security, Monitoring & Management

Amazon Cognito

Description: A service that provides authentication, authorization, and user management for web and mobile apps.
Role: Manages user identities and provides secure authentication for the chat interface, integrating with API Gateway for authorization.

AWS IAM

Description: Enables us to securely control access to AWS services and resources.
Role: Defines granular permissions for all AWS services and resources. Each Lambda function, Bedrock Agent, and other services will operate under IAM roles with the principle of least privilege, ensuring they only have access to the resources they need.

AWS KMS

Description: Makes it easy for us to create and manage cryptographic keys and control their use across a wide range of AWS services and in the applications.
Role: Used for encryption at rest for sensitive data stored in DynamoDB, S3, and OpenSearch Service, ensuring data confidentiality.

Amazon VPC

Description: Enables us to launch AWS resources into a virtual network.
Role: Provides network isolation for services like OpenSearch Service and potentially Lambda functions connecting to internal resources, enhancing security.

Amazon CloudWatch

Description: A monitoring and observability service.
Role: Collects logs, metrics, and events from all AWS services in the architecture (Lambda, API Gateway, Bedrock, OpenSearch, DynamoDB). This is critical for:
- Traceability: End-to-end logging of requests and responses helps trace the flow of information and debug issues.
- Monitoring: Real-time metrics for performance, errors, and usage.
- Alerting: Setting up alarms for anomalies (e.g. increased error rates, high latency) via Amazon SNS.
- CloudWatch Logs Insights: For interactive analysis of log data.

Amazon Bedrock Guardrails

Description: A feature within Amazon Bedrock that allows us to implement safety policies and filters for generative AI applications.
Role: Ensures that the GenAI model's responses are safe, on-topic, and do not contain harmful, offensive, or off-brand content. It's crucial for maintaining brand reputation and compliance.

DevOps & MLOps

GitHub and GitHub Actions

Description: Services for Continuous Integration and Continuous Delivery (CI/CD).
Role: Automate the build, test, and deployment of the Lambda functions, Bedrock Agent configurations, and other infrastructure changes. This ensures rapid, consistent, and reliable deployments.

Amazon SNS

Description: A fully managed messaging service for application-to-application (A2A) and application-to-person (A2P) communication.
Role: Sends alerts based on CloudWatch alarms, notifying relevant teams of operational issues or security incidents.

Amazon CloudWatch Dashboards/Logs Insights

Description: Visualisation and query tools within CloudWatch.
Role: Provides centralised dashboards for monitoring the health and performance of the entire solution, and allows for deep dives into logs for troubleshooting and analysis.

Enterprise Grade Considerations

Scalability: All chosen AWS services (Lambda, API Gateway, SQS, DynamoDB, OpenSearch, Bedrock) are inherently scalable and managed by AWS, eliminating the need for manual server provisioning or scaling.
Security
- Authentication & Authorization: Amazon Cognito and IAM roles with least privilege.
- Data Encryption: AWS KMS for encryption at rest and SSL/TLS for encryption in transit.
- Network Isolation: VPC for internal services like OpenSearch.
- Content Moderation: Amazon Bedrock Guardrails to prevent harmful or off-brand content.
- PII Handling: Implementation of PII redaction or masking if sensitive customer data is processed.
Traceability: Comprehensive logging with CloudWatch Logs and metrics, allowing for detailed auditing and debugging. Chat history persistence in DynamoDB also aids in traceability and understanding user interactions.
Reliability & High Availability: Services are designed for high availability by default (e.g. multi-AZ deployments for DynamoDB, OpenSearch, and Bedrock). SQS provides decoupling for resilience.
Cost Optimisation: Pay-as-you-go model for most services, and serverless compute (Lambda) means you only pay when code runs.
Observability: Integrated monitoring through CloudWatch and its various features.

There are countless enhancements that can be made to this solution, unlocking endless business possibilities. On a recent call, my friend mentioned that his development team had just delivered the first version of what they called an "intelligent agent" for their e-commerce website. I was thrilled to hear that and immediately said, "Mate, don’t just call it an intelligent agent—give it a name! I truly believe these AI applications will soon become part of human society, and just like us, they deserve names!"

Effortless API Scaling: Unlock the Power of AWS AppSync

Lilupa Karu — Thu, 26 Dec 2024 10:25:21 +0000

AWS AppSync Journey

During the challenging times of the COVID-19 pandemic in 2020, I embarked on architecting my first AWS AppSync enterprise solution with GraphQL APIs. The project involved integrating several data sources, including a legacy system. After completing the architecture design and undergoing a thorough peer review, our engineering teams began building the solution, with AWS AppSync as the core component. The project was a success, delivering a robust solution that brought smiles to everyone involved. At the time, I had the intention of writing an article about the experience, but, much like the pandemic itself, that plan faded into the background.

Fast forward to a few months ago, I had a coffee catch up with Brett, an ex-colleague who had been part of that AppSync project. As we reminisced about the incredible time we spent on the project, I was inspired to revisit AWS AppSync and finally write this article. Reflecting on the challenges we overcame and the happy outcomes we achieved, I owe it to Brett's recollection of those moments to share this journey.

Since AWS AppSync is a serverless implementation of GraphQL, a brief overview of GraphQL is necessary before proceeding.

What is GraphQL

The fundamental concept of a GraphQL API is that all API functionality is accessed through a unified query language (GQL) via a single endpoint. Instead of making multiple requests to different endpoints to retrieve various data needed for building a software application, a single request can be issued to a GraphQL API, returning all the necessary data at once. This approach reduces the complexity of modern applications and enhances user experience by enabling faster load times.

The system providing data to a GraphQL API is rarely just a single application. In my experience, I’ve worked with nearly a dozen applications where GraphQL played a key role, but only one of those had a single data source. Typically, the backend consists of multiple microservices or data sources, with the GraphQL API layer responsible for aggregating data from various applications and delivering a unified response to the API requester. This GraphQL layer must efficiently parse the incoming requests, identify the appropriate data sources for each part, and seamlessly combine the results to generate the final response.

Components in GraphQL

GraphQL Types
GraphQL Types define the structure of entities by specifying their fields and attributes. Below is a list of the various types available.

Scalar Type
Object Type [In the Object type, there are three special root operation types: Query, Mutation, and Subscription]
Input Types
Enumeration Type
Union and Interface Type
Lists and Non-Null

Schema
The schema is treated as a contract between the server and the client. It consists of a collection of types and fields, along with a defined set of operations available for interacting with the data.

type Project {
    id: ID!
    name: String!
    client: String!
    tasks: [Task!]
  }

type Task {
    title: String!
    description: String
    duration: Int
  }

Query
Queries serve as entry points on a GraphQL server, providing read access to data sources.

  query {
    project (id: 1) {
      name
      client
    }
  }

Mutations
GraphQL Mutations function as entry points on a GraphQL server, allowing write access to data sources. Whenever data needs to be modified—whether creating, updating, or deleting—it is done through GraphQL mutations.

type Mutation {
  addProject(name: String, client: String): Project
}

Resolvers
Resolvers are built-in functions in GraphQL that connect types or fields defined in the schema to the corresponding data in the data sources.

AppSync Deep Dive

With a basic understanding of GraphQL, attention can now be turned to AWS’s serverless offering of GraphQL. AWS AppSync, a fully managed serverless service, facilitates real-time data queries, synchronisation, and communication. Through AppSync, AWS provides a GraphQL-as-a-Service solution, enabling the seamless creation of scalable and resilient GraphQL APIs in the cloud. Request parsing and resolution, as well as integration with other AWS services such as AWS Lambda, NoSQL and SQL data stores, and HTTP APIs, are all handled to retrieve backend data for the API.

Overview and Architecture

Developers are granted access to their data via this managed GraphQL service, which offers several advantages over traditional gateways. GraphQL promotes declarative coding and integrates effectively with modern tools and frameworks such as React, React Native, iOS, and Android. Subscriptions can be utilised to implement live updates, push notifications, and other features.

When GraphQL subscription operations are invoked, AWS AppSync automatically establishes and maintains a secure WebSocket connection. Data can then be distributed in real-time to subscribers from a data source, while connection and scaling requirements are managed.

With AWS AppSync, serverless GraphQL and Pub/Sub APIs allow secure querying, modification, or publication of data using a single endpoint.

GraphQL APIs

AWS AppSync-built GraphQL APIs empower frontend developers to seamlessly query multiple databases, microservices, and APIs through a single GraphQL endpoint. This simplifies data interaction and accelerates application development.

Pub/ Sub APIs

AppSync Use Cases

Gateway for microservices
AppSync serves as a powerful gateway for microservices, allowing you to access and aggregate data from various sources with ease.

Real Time Collaboration
AppSync enables scalable, real-time collaboration by broadcasting data from the backend to all connected clients (one-to-many) or facilitating communication between clients (many-to-many). This is ideal for chat applications, shared documents, or live dashboards.

Polyglot data access
With AppSync, you can interact with diverse data sources—including SQL databases in Amazon Aurora Serverless, NoSQL tables in Amazon DynamoDB, search indexes in Amazon OpenSearch Service, REST endpoints via Amazon API Gateway, and serverless backends using AWS Lambda—using a single GraphQL call.

Offline Delta Sync
AppSync integrates with Amplify DataStore, an on-device persistent storage engine, to enable offline-first capabilities. Data is automatically synchronised between mobile/web apps and the cloud using GraphQL, ensuring a seamless experience even in limited connectivity scenarios.

Let’s Build Something Together

Why just read when you can build? Let’s dive into a practical example to explore how AWS AppSync works in action.

For this demo, we’ll build a simple coffee ordering app—a favorite theme of mine because, let’s face it, life without coffee isn’t life at all.

Building the Coffee Order App

Here’s what the app does:

GraphQL Schema for Orders Customers can place coffee orders by providing their name, mobile number, and preferred coffee type. Orders are submitted through the createOrder mutation and stored in a DynamoDB table via a Lambda function.

Querying Orders
Coffee shop owners can retrieve new orders from the database using the listOrders GraphQL query. This query also utilises the same Lambda function for data retrieval.
Infrastructure and Frontend
The backend is built entirely with AWS CDK, ensuring it adheres to Infrastructure as Code (IaC) best practices. On the frontend, I’ve created a simple React application that fetches new coffee orders via AppSync’s GraphQL API.

The complete application is available on GitHub for you to explore and adapt.

AWS CDK-based AppSync APIs: Vitinya CDK
React Front-end web app: Vitinya Web

As for the app’s name, Vitinya, it’s inspired by the Vitinya Pass in Bulgaria’s Balkan Mountains—a place I’ve yet to visit but remains high on my bucket list. By naming the app after this scenic destination, I’ve given myself a constant reminder to plan my Eastern European holiday soon!

Until Next Time

That’s all for now. Stay tuned for more articles and hands-on examples in the beautiful AWS ecosystem. Until then, happy AWSing!

Event-Driven Magic: Exploring AWS EventBridge

Lilupa Karu — Sat, 20 Jul 2024 11:36:46 +0000

I kept thinking about what Werner Vogels said in his speech at AWS re:Invent in December 2022,

the world is event-driven

Even though I’ve involved on a few event-driven architectures before, his words made me reconsider how we use event-driven architectures.

The result? I’ve developed some impressive reference architectures utilising modern cloud-based tools and services. Let’s begin by reviewing the fundamentals.

Event-driven architecture is all about events. Simply put, an event is a change in state. Picture it like this: swiping a credit card at a payment machine is an event. Our world is full of these actions and events, and they play a big role in how everything functions. Think of events as signals that indicate when something changes in a system, like saving information to a database, uploading a file into storage, or a sensor being triggered on an IoT device.

A key point to remember about events is their immutability; once they occur, they cannot be altered. This concept mirrors everyday life — once a moment passes, it is unchangeable. For instance, I wished for the All Blacks to win RWC 2023. Regardless of how much I wanted it, there was no way to go back and change the outcome once the event had taken place and was recorded in history.

Event Driven Architecture Components

Event Producer: These entities generate events based on specific actions or triggers within the system and are solely aware of the event router, not the event consumers.

Event Broker: The event broker is responsible for receiving and filtering events, ensuring that each event is directed to the correct consumer. It serves as a mediator, enabling the smooth transmission of events throughout the architecture.

Event Consumer: Event consumers either subscribe to particular events or monitor event streams, reacting to events based on their individual needs or initiating further actions within the system.

Event-Driven Architecture Patterns

There’s much discussion surrounding event-driven architecture patterns across various categories. Below are some of the most popular patterns:

Communication Patterns: publish/subscribe, point-to-point, event streaming, request/reply

Consumption Patterns: guaranteed delivery, hierarchical topics, event filtering, backpressure, and push

Event Generation Patterns: ECST, event sourcing, CQRS, CDC

Governance Patterns: event catalog, event APIs, intermediated, disintermediated, access control, and authorization

Let’s talk about EventBridge

Let's keep our attention on the main topic. We should delve into what EventBridge is and explore the reasons behind its growing popularity.

EventBridge, introduced in July 2019, is a serverless service that leverages events to seamlessly connect components of software applications. This facilitates the creation of modern, scalable, and loosely coupled event-driven applications.

Its robust positioning within this architectural domain stems from its seamless integration capabilities with numerous other AWS services, including Step Functions, SQS, SNS, and Lambda functions, all of which inherently complement event-driven architectures.

In the middle of the illustration, we have AWS EventBridge, a serverless event bus. EventBridge is designed with a pub-sub architecture pattern in mind, allowing external sources to ingest events as ‘publishers.’ These sources can range from bespoke applications and various SaaS applications to a wide array of AWS services.

Events

An event is a real-time change in a system, data, or environment. This change can occur in your application, an AWS service, or a SaaS partner service. While events can have similar structures, they must all include the top-level fields: detail, source, and detail-type. Custom events must, at a minimum, contain these fields.

Custom Event Sample

{
    "version": "0",
    "id": "665f15b9-bbbb-ffff-0000-ee92777062fb",
    "detail-type": "product.update",
    "source": "your source app",
    "account": "123456789012",
    "time": "2024-07-16T05:24:42Z",
    "region": "us-east-1",
    "resources": [],
    "detail": {
        "action": "product update"
    }
}

Rules

Once the events are ingested into EventBridge, rules redirect the events to appropriate targets based on their content. EventBridge has a soft limit of 300 rules, which can be increased via a support ticket. These rules can be connected to multiple targets, such as AWS services like Lambda functions, Step Functions, or SQS queues.

Event Schema

EventBridge is capable of handling event schemas, which represent the structure of events. A schema typically includes details such as the title and type of each data element. For example, on an online survey website, a schema might include fields for the survey name, description, reviewer’s name, user ID, and reviewer’s comments and ratings. These attributes are defined with specific types, such as the name being a string and the user ID being a GUID, etc.

Schema Registry

A schema registry serves as a centralised repository for managing a collection of schemas. It allows you to search, locate, and track various schemas utilised and generated by your applications, ensuring consistency and compatibility across your data infrastructure.

Schema Discovery

Schema discovery automates the identification and registration of schemas, streamlining their integration into your registry. When enabled for an EventBridge event bus, schema discovery automatically records the schema of each event sent to the bus in the registry. If an event’s schema changes, schema discovery promptly creates a new version in the registry. After a schema is registered, you can generate a code binding for it, facilitating seamless interaction with the schema.

Pipes

EventBridge buses facilitate the routing and distribution of events to services using the publish/subscribe pattern, while EventBridge Pipes manage direct integrations between services through the point-to-point pattern. Both patterns are essential for constructing event-driven architectures. EventBridge Pipes allow for direct integrations without needing specialised knowledge. These pipes consist of a Source and a Target, and an enrichment step can be employed to enhance the payload data before it is sent to the target.

EventBridge Use Cases

There are numerous use cases for event-driven architecture where we can effectively apply AWS EventBridge. Let’s explore some of these use cases.

SaaS Product Integration

Imagine your organisation has a CRM system that is a SaaS partner with AWS EventBridge. The architecture below can be used to integrate this CRM with other business activities. The CRM system can send events to the EventBridge whenever there are changes in the system, data, or environment. Based on the rules defined in the EventBridge, messages are distributed to the appropriate tasks. For example, when a new customer registers in the CRM, the customer banking integration Step Function subscribes to the event and performs the relevant tasks.

Bespoke System Integration

When developing a bespoke software application that requires integration with external entities in an event-driven manner, the following architecture would be ideal.

Consider a scenario where a customer requests custom-made furniture. The request is received by the company’s sales system, which then communicates the specifications to the factory. Throughout the various phases, the customer needs to be kept informed, and once the furniture is ready, the designated courier company must be notified for pickup and delivery. This integration can be handled more reliably and efficiently using an event-driven architecture with AWS EventBridge, rather than a polling architecture.

After diving into a multitude of theories, it’s great to get some hands-on experience. Part two of this article, featuring a near real-world EventBridge integration, will be released soon. Stay tuned!