"Quick show of hands. Who here has tried to scale AI adoption across a large organization and found it much harder than the blog posts make it sound?"
This opening from Lilia Abaibourova perfectly captured the reality many organizations face. While individual developers might successfully use AI coding assistants, scaling that success across thousands of engineers requires something fundamentally different. In DEV323, Lilia (Principal Product Manager at Prime Video) and James Hood (Principal Software Engineer at AWS) shared their journey from isolated AI experiments to company-wide AI Native transformation.
What makes this session particularly valuable is the honest discussion of both successes and failures, the grassroots movement that sparked adoption, and the organizational support needed to sustain it. This wasn't a polished success story - it was a real transformation with real challenges.
Watch the full session:
From AI-Assisted to AI Native: What's the Difference?
Lilia drew a critical distinction that many organizations miss. AI-assisted development means using tools like autocomplete and chatbots in isolated, ad hoc ways - pasting context between tools throughout your workflow. The impact is on individual productivity for isolated tasks.
AI Native development is fundamentally different:
- AI tooling embedded throughout each step of the software development lifecycle
- Every role enabled with AI - PMs, developers, designers, not just engineers
- Agents owning multi-step workflows end to end, not just autocomplete
- Context embedded and shared throughout tooling and infrastructure
- Organizational productivity transformation, not just individual gains
As Lilia emphasized: "We reimagine how we build software" rather than just accelerating existing processes.
The Accidental Movement: From Skeptic to Power User
James Hood's journey began as a self-described "AI skeptic" caught in a vicious cycle: flashy demos would spark curiosity, he'd experiment on real problems, the tools would fail, and skepticism would deepen. Sound familiar?
His breakthrough came when he realized that LLMs have emergent behaviors based on how you prompt them. By combining better tools with specific techniques, he became what he calls a "gen AI power user."
The Two-Day Feature Story
James shared his first major success: building a feature into Amazon Q Developer CLI in a codebase he'd never seen, written in a language he didn't know, going from concept to pull request in two days and production launch in seven days.
His key insight: Software development is a process. You can't give an AI (or a human) a vague idea and expect perfect implementation. You need to go through research, requirements clarification, design, implementation planning, and then implementation.
James used AI to "speed run" through this process:
Research the Codebase (5 minutes):
- Asked Q CLI to analyze the codebase structure
- Focused on the specific component he needed to modify
- Had it document slash command implementation details
Clarify Requirements (24 questions, ~35 minutes):
- Used a specific prompt pattern to turn the agent into a partner
- Agent asked clarifying questions one at a time
- Sometimes James didn't have answers - the agent provided options
- Iterative refinement until requirements were clear
Design Solution (~20 minutes):
- Generated detailed design document based on planning docs
- Reviewed and refined the design
- Ready for implementation planning
Implementation Plan (created 14-prompt sequence):
- Converted design into step-by-step prompts for code generation
- Each step was self-contained and meaningful
- Generated a to-do list to track progress
Implementation (remainder of 2 days):
- Executed prompts sequentially, testing along the way
- Used human judgment throughout - not "vibe coding"
- Leveraged two decades of programming experience to review and guide
James emphasized: "I was following along as it was implementing. I was going back and forth with the agent, questioning aspects of the code and using my human judgment and experience to make sure we were headed in the right direction the whole time."
Sparking the Grassroots Movement
This success led James to create an internal Slack channel in March 2025: amazon-buildergenai-power-users. It grew to tens of thousands of members - a massive grassroots movement of builders experimenting, sharing techniques, and learning from each other.
James described this as the "AI native virtuous cycle": curiosity leads to experimentation, which leads to sharing (both successes and failures), which maintains healthy skepticism while fueling more curiosity.
Organizational Support: Three Pillars of Transformation
Lilia took over to explain how Prime Video provided organizational support to scale this grassroots energy. Transformation requires both bottom-up energy and top-down support.
Pillar 1: Access and Infrastructure
Prime Video built an AI enablement stack that goes far beyond coding assistance:
Tools Across the SDLC:
- Kiro (AI native development IDE with CLI interface) - gaining traction with PMs and designers for prototyping, not just developers
- Amazon Quick Suite (business intelligence and AI platform) - for researching requirements, learning from past experiments, analyzing data
- Custom agents - test agents for hundreds of device types, operational agents for incident diagnosis and troubleshooting
Shared Context and Capability Layer:
- Model Context Protocol (MCP) as the "context backbone"
- Knowledge bases connecting AI tools to business and technical content
- Context flows from PM specs → development (via Kiro) → operational agents
- No more pasting context between tools
Infrastructure Stack:
- Foundation: AWS (IAM, compute, observability)
- Model layer: Amazon Bedrock with largest model selection, AgentCore, Amazon SageMaker
- Context layer: MCP servers and hosts, knowledge bases
- Experience layer: Mix of off-the-shelf tools (Kiro) and custom agents
The key was balancing central investments (code repository access, issue tracking) with team-specific solutions (Prime Video catalog metadata) and local customization (prompts, steering files).
Pillar 2: Culture and Learning
Learning and experimentation became part of everyday culture through the AI native learning flywheel:
Safe Sandboxes:
- Demo days and hackathons for AI experimentation
- Automating non-AI-native workflows
- Building test workflows and on-call operations
Formalized Mechanisms:
- AI champions program from early adopters
- AI bar raising program from principal engineering community
- Quick start guide codifying best practices and techniques
- Spec-driven development and live architecture specs
Embedded Rituals:
- AI reflection sections in team retros
- AI learning time in quarterly reviews
- Sharing both successes and challenges
Role-Specific Learning Paths:
- Engineering: Spec-driven development workshops, agent workflow creation
- Product/Design: Writing, prototyping, coding workshops (50 PMs coded for the first time!)
- Data teams: Natural language query interfaces
- Leadership: MCP for goal reporting and strategic document queries
Every role learned MCP to access relevant knowledge bases and tools. As Lilia noted: "If you accelerate only one part of the process of one role and one team, then you're just moving the bottleneck somewhere else."
Pillar 3: Trust and Rigor
Early pitfalls taught important lessons:
Code Review Overload:
- AI-generated multi-file changes created bottlenecks
- Solution: Emphasized ownership principle (review every line), built code review agents with human oversight
- Result: Code review velocity restored
Policy and Process Friction:
- Autonomous agents needed data access
- Review processes built for humans, not agents
- Solution: Partnered with security/privacy to modernize processes and automate governance
Guardrails at Every Layer:
- Infrastructure: Capacity controls, IAM policies
- Model/Capability: Prompt templates, fallback mechanisms, rigorous evaluation sets
- Context: Authentication controls for MCP servers (remote and local)
- Experience: Permissions based on SSO and service-to-service authentication
The safe path became the easy path.
Measuring What Matters
Lilia was transparent about measurement evolution:
Adoption Metrics:
- Initially obsessed with pure adoption numbers
- Evolved to tool-specific adoption per SDLC phase and role
- Still monitored but not the only focus
Velocity and Efficiency:
- DORA metrics (industry standard)
- Meaningful increases in deployment velocity and code review velocity
- Critical: Controlled for incident rates and rollback rates
Use Case-Specific Metrics:
- Efficiency gains for custom agents
- Time savings for specific workflows
- Domain-appropriate measurements
Qualitative Insights:
- Regular surveys and interviews
- Leadership listening sessions
- Builder feedback as best source of insight
One engineer shared: Previously spent three weeks minimum on design documentation and stakeholder alignment. With AI tooling, writing the document takes three days, freeing time to focus on stakeholder discussions and integration use cases.
The Demo: Strands Agent SOPs
James concluded with a powerful demo of Strands Agent SOPs, open-sourced less than two weeks before re:Invent.
What are Agent SOPs?
Standard Operating Procedures for agents - detailed step-by-step instructions in a specific Markdown format that helps agents repeatedly follow processes.
The Format:
- Title and overview
- Parameters section
- Steps with constraint sections using RFC 2119 keywords (MUST, SHOULD, MAY)
- Examples section (LLMs work well with examples)
- Troubleshooting section
This format strikes the perfect balance between following the same procedure every time while leveraging agent flexibility. Internally, they call the behavior "determin-ish-tic."
Creating SOPs:
Rather than writing SOPs by hand, you chat with an agent that has the SOP format as a steering file. James demonstrated:
- Creating a simple name poem generator SOP through conversation
- Modifying it to add optional parameters
- Creating a real-world GitHub issue triage SOP that searches for unlabeled issues, applies labels, and adds comments
The Full Development Workflow:
James showed the complete prompt-driven development (PDD) SOP that automates the entire process he described earlier:
- Codebase summary generation
- Requirements clarification through guided questions
- Design document creation
- Implementation plan generation
- Step-by-step implementation
All available as downloadable, reusable SOPs - no copying and pasting prompts from blog posts.
Key Takeaways
For Individual Developers:
- Software development is a process - use AI to speed through research, requirements, design, and planning
- Leverage human judgment throughout - this isn't "vibe coding"
- Share learnings (successes and failures) to fuel the virtuous cycle
- Maintain healthy skepticism while staying curious
For Organizations:
- Grassroots energy is essential but insufficient - provide organizational support
- Build infrastructure that enables every role, not just developers
- Make learning and experimentation part of everyday culture
- Build trust through rigor - guardrails at every layer
- Measure adoption, velocity, and outcomes while listening to builders
- The safe path must be the easy path
For AI Native Transformation:
- It's not about tools - it's about reimagining how you build software
- Context sharing across tools and teams is critical
- Every role must evolve, not just engineering
- Balance bottom-up energy with top-down support
- Start with safe sandboxes, formalize what works, embed in rituals
As James concluded: "I don't think that you can succeed in AI native transformation without this kind of grassroots energy and movement and excitement from your builder community. However, transformation has to come both from - you can't rely purely on bottoms-up and you can't rely purely on top-down. It's a mix of both."
Resources:
About This Series
This post is part of DEV Track Spotlight, a series highlighting the incredible sessions from the AWS re:Invent 2025 Developer Community (DEV) track.
The DEV track featured 60 unique sessions delivered by 93 speakers from the AWS Community - including AWS Heroes, AWS Community Builders, and AWS User Group Leaders - alongside speakers from AWS and Amazon. These sessions covered cutting-edge topics including:
- 🤖 GenAI & Agentic AI - Multi-agent systems, Strands Agents SDK, Amazon Bedrock
- 🛠️ Developer Tools - Kiro, Kiro CLI, Amazon Q Developer, AI-driven development
- 🔒 Security - AI agent security, container security, automated remediation
- 🏗️ Infrastructure - Serverless, containers, edge computing, observability
- ⚡ Modernization - Legacy app transformation, CI/CD, feature flags
- 📊 Data - Amazon Aurora DSQL, real-time processing, vector databases
Each post in this series dives deep into one session, sharing key insights, practical takeaways, and links to the full recordings. Whether you attended re:Invent or are catching up remotely, these sessions represent the best of our developer community sharing real code, real demos, and real learnings.
Follow along as we spotlight these amazing sessions and celebrate the speakers who made the DEV track what it was!
Top comments (0)