DEV Community: Kento IKEDA

What Changes and What Stays the Same for SRE with AWS Frontier Agents

Kento IKEDA — Mon, 13 Apr 2026 20:13:46 +0000

On March 31, 2026, AWS made DevOps Agent and Security Agent generally available — the first two of the autonomous AI agents announced at re:Invent 2025 under the "Frontier Agents" brand. A 2-month free trial is included, after which pay-as-you-go pricing kicks in.

https://aws.amazon.com/blogs/mt/announcing-general-availability-of-aws-devops-agent/

https://aws.amazon.com/blogs/machine-learning/aws-launches-frontier-agents-for-security-testing-and-cloud-operations/

The official announcements highlight numbers like "up to 75% MTTR reduction" and "penetration testing compressed from weeks to hours." The question that matters more is: how does this change the day-to-day work of an SRE team? Feature overviews are already plentiful, so this article focuses on what shifts to agents and what stays with humans.

What Is a Frontier Agent?

AWS announced three Frontier Agents at re:Invent 2025: Kiro Autonomous Agent (software development), DevOps Agent (operations), and Security Agent (security). Of these, DevOps Agent and Security Agent are now GA. Kiro Autonomous Agent remains in preview.

AWS defines Frontier Agents as systems that "work independently to achieve goals, scale massively to tackle concurrent tasks, and run persistently for hours or days." Frankly, that description could apply to existing AI agents like Claude Code or Devin. What AWS emphasizes is delivering "complete outcomes" rather than assisting with individual tasks, but this feels like a difference of degree, not kind.

In practice, it's probably best to think of them as domain-specialized autonomous agents — deeply integrated with DevOps and security workflows. "Frontier" is more of a marketing brand than a technical category: "AWS's first-party, domain-specific agent products" is a fair characterization.

What matters isn't the naming — it's how these agents affect SRE work.

What DevOps Agent Does and Doesn't Do

AWS describes DevOps Agent as an "always-available operations teammate." However, since it requires human approval for fixes and can't make business decisions, the reality is closer to an "always-on SRE apprentice" — it investigates and proposes, but can't decide or execute. Here's where that boundary lies.

What the Agent Does

Imagine an alert fires at 2 AM. Traditionally, the on-call engineer wakes up from a Datadog alert, opens their laptop, checks dashboards for metric anomalies, digs through logs, cross-references deployment history, and identifies the root cause. DevOps Agent automates this entire initial investigation.

Specifically, it correlates metrics and logs from monitoring tools (CloudWatch, Datadog, Dynatrace, New Relic, Splunk, Grafana), code repositories (GitHub, GitLab, Azure DevOps), and CI/CD deployment histories to build hypotheses like "this code change introduced in this deployment correlates with this metric anomaly." Investigation progress is shared via the web console and Slack, where you can ask follow-up questions or redirect the investigation.

The GA release adds Azure and on-premises environments as investigation targets. On-premises tools connect via MCP (Model Context Protocol), enabling consistent investigation across multicloud and hybrid setups.

Beyond incident response, DevOps Agent also provides proactive improvement recommendations — analyzing historical incident patterns to identify gaps in alert coverage, test coverage, code quality, and infrastructure configuration.

It's worth noting that Datadog's Bits AI SRE offers quite similar capabilities: autonomous alert investigation, source code analysis, and deployment correlation. The key difference is that DevOps Agent can simultaneously span multiple observability tools (Datadog + CloudWatch + Splunk, etc.) and include Azure and on-premises environments via MCP. If your organization is entirely within the Datadog ecosystem, Bits AI SRE may be sufficient. If you have multiple tools or a multicloud setup, DevOps Agent's cross-platform analysis adds value. More on this in the "How Does the Relationship with Existing Tools Change?" section.

The GA release also introduced "Learned Skills" and "Custom Skills." Learned Skills let the agent learn from your organization's investigation patterns and tool usage, improving accuracy over time. Custom Skills let you add organization-specific investigation procedures and best practices, configurable per incident type (triage, root cause analysis, mitigation). Code Indexing also enables code-level fix suggestions based on repository understanding.

What the Agent Doesn't Do

This is the critical part. DevOps Agent investigates and proposes — executing fixes is up to humans.

It can generate fix proposals and work with Kiro or Claude Code to produce fix code, but applying changes to production requires human approval. This is intentional — AWS has made the design decision that "an agent that modifies production without approval won't be trusted."

The other thing the agent doesn't do is business judgment. "This incident has major customer impact, so we need a company-wide response." "It's Friday night — let's apply a workaround and do the root cause fix Monday." These decisions require human context. Identifying the technical root cause and deciding how to respond are separate jobs. DevOps Agent handles the former; humans own the latter.

What Security Agent Does and Doesn't Do

Security Agent is the security counterpart — an "always-on penetration tester and security reviewer." It has three main capabilities: on-demand penetration testing, design document security review (Design Review), and PR security review (Code Review).

What the Agent Does

Penetration Testing

Traditionally, penetration testing meant "once or twice a year, for the most critical applications only, outsourced to specialists, taking weeks." Cost and time constraints leave most of the application portfolio untested. As the Security Agent FAQ notes, "most organizations limit manual penetration testing to their most critical applications and conduct these tests periodically." And even tested applications become partially unverified the moment new code is deployed.

Security Agent changes this structure. You create an "Agent Space," connect your GitHub repository, and the agent reads source code, architecture documents, and design docs to understand the application's structure before running automated penetration tests against endpoints.

The key difference from simply running a scanner: Security Agent validates discovered vulnerabilities by actually sending payloads to confirm exploitability. Reports include reproduction steps, dramatically reducing false positives. Per the official features page, testing covers OWASP Top 10 vulnerability types plus business logic flaws. According to the GA announcement blog, LG CNS reported significant false positive reduction, over 50% faster testing, and roughly 30% cost reduction.

Design Review

This capability reviews architecture documents and design docs from a security perspective before any code is written. It checks against AWS best practices and your organization's custom security requirements. Catching issues at the design stage avoids costly rework after implementation.

PR Review

Pull Request-level security review is the third capability. As of GA, it supports GitHub PRs — Security Agent automatically reviews PRs for security issues when they're created. You can configure it to check custom security requirement compliance, common security vulnerabilities, or both.

PR security checks aren't new — many organizations already have Claude Code or Codex review PRs with security instructions via CLAUDE.md, or have SAST tools in their CI/CD pipeline. Security Agent's difference is operational: security requirements are defined once in the console and automatically applied across all repositories. This removes the overhead of maintaining per-repository md files, but it doesn't enable something technically impossible before. Of the three capabilities, penetration test automation is where the real differentiation lies.

Design reviews are free up to 200/month, and code reviews up to 1,000/month. Only penetration testing is paid ($50/task-hour) — more on this in the "Cost Structure" section.

What the Agent Doesn't Do

Security Agent automates "discovery and validation" — "judgment and response" remain human territory.

Security policy decisions are human work. "Which risks to accept and which to address," "how to interpret compliance requirements" — these are outside the agent's scope. For example, "fixing this vulnerability requires a breaking API change, but we need to consider the impact on a major customer's release schedule" is a business trade-off that requires human judgment.

Social engineering (tricking employees into granting access) and vulnerabilities that can only be discovered by understanding the entire business workflow are also difficult to cover with automated testing alone. While the official documentation says business logic flaws are included in the test scope, the agent doesn't fully replace a human penetration tester's judgment of "what this operation means in the context of this business flow." Security Agent's strength is "broad, frequent, systematic testing" — complementing, not replacing, "deep, creative testing" by human experts.

How Does the Relationship with Existing Tools Change?

"Will Datadog or PagerDuty become unnecessary?" Short answer: no. The relationship changes, not the need.

Here's how the human steps change, using a late-night alert as an example:

Previously humans handled all 5 steps; after adoption, human steps drop to 2. Red = previously all-human work, green = shifts to the agent, blue = remains human.

Monitoring Tools (Datadog / CloudWatch, etc.)

DevOps Agent doesn't "replace" these tools — it uses them as "data sources." GA supports integrations with CloudWatch, Datadog, Dynatrace, New Relic, Splunk, and Grafana. This isn't about canceling Datadog and switching to DevOps Agent — it's about DevOps Agent analyzing the metrics and logs that Datadog collects.

You might wonder: "Could we drop Datadog, consolidate on CloudWatch, and use DevOps Agent / Security Agent to save costs?" DevOps Agent handles incident investigation and improvement recommendations — it doesn't include day-to-day monitoring features like APM, RUM, distributed tracing, or dashboards. Datadog's value extends beyond incident response, so a simple swap doesn't work. That said, there is overlap between Datadog's Bits AI SRE and DevOps Agent's incident investigation capabilities, so whether you need to pay for both is worth evaluating.

In fact, the quality of your monitoring setup directly affects DevOps Agent's effectiveness. The agent can only analyze what your tools collect. Sparse metrics and logs mean less accurate analysis. The direction isn't "adopt the agent so we can invest less in monitoring" — it's "invest in monitoring to maximize the agent's effectiveness."

Incident Notification and On-Call Management (PagerDuty / Datadog On-Call, etc.)

Alert routing and on-call management roles remain unchanged. DevOps Agent starts investigating the moment an alert fires, completing initial investigation before the notified human even logs in. On-call scheduling, escalation, and incident lifecycle management continue to be handled by tools like PagerDuty or Datadog On-Call. The GA release added PagerDuty integration as well.

What changes is "the first thing the on-call engineer does." Instead of "open the dashboard and check metrics," it becomes "read the Agent's investigation results shared in Slack." Per the official GA blog, Zenchef (a restaurant technology platform) submitted an issue to DevOps Agent during a hackathon and had the root cause identified in 20–30 minutes — an investigation that would normally take 1–2 hours, completed while the engineers stayed focused on the hackathon.

GitHub (Security Agent PR Review)

Security Agent automatically posts security review comments on GitHub PRs. Developers can review and address findings without leaving the PR interface. Merge decisions remain human. Details and differentiation points are covered in the "What Security Agent Does and Doesn't Do" section above.

Cost Structure

Frontier Agents have a distinctive pricing model, especially DevOps Agent's tie-in with AWS Support plans.

DevOps Agent: Support Credits

DevOps Agent costs $0.0083/agent-second (roughly $30/hr). The official pricing page shows usage examples: a small team (10 incident investigations/month, 8 minutes each) at ~$40/month, and an enterprise (500 incidents, 10 Agent Spaces) at ~$2,300/month.

On top of this, per the pricing page, AWS Support customers receive monthly credits based on the prior month's Support spend:

Support Plan	Credit Rate
Unified Operations	100% of prior month's Support spend
Enterprise Support	75% of prior month's Support spend
Business Support+	30% of prior month's Support spend

For example, an organization paying $15,000/month for Enterprise Support would receive $11,250/month in DevOps Agent credits. If usage stays within that, the incremental cost is zero.

Behind this credit structure is the relationship with TAM (Technical Account Manager) in Enterprise Support. Traditionally, Enterprise Support customers get a TAM who provides architecture reviews and operational guidance. The Enterprise Support page now presents TAM and DevOps Agent side by side — TAM handles strategic guidance, DevOps Agent handles 24/7 automated investigation and improvement proposals. DevOps Agent is positioned as an "extension" of Support, not a replacement, which explains why credits come from Support spend.

Security Agent: Penetration Testing Cost Transformation

Security Agent penetration testing is billed at $50/task-hour (design and code reviews have free tiers as mentioned above). Per the official FAQ, a 2-month free trial is included post-GA.

Traditional third-party penetration testing typically costs hundreds of thousands of yen (tens of thousands of dollars) per engagement, taking weeks. This "high unit cost × low frequency" structure transforms into "$50/task-hour × high frequency."

The implication: it becomes economically viable to expand penetration testing coverage. Organizations that could only test their most critical applications due to cost constraints can now continuously test across their entire portfolio.

How Does SRE Team Management Change?

Frontier Agents adoption has implications for how SRE teams operate.

The Nature of On-Call Pain Changes

DevOps Agent's biggest impact is automating initial investigation for late-night alerts. Per the official GA blog, WGU (Western Governors University) deployed it to production during preview, reducing estimated 2-hour investigations to 28 minutes.

Traditional on-call pain comes from being woken up and having to start investigating from scratch in a less-than-ideal state. Opening dashboards, hunting for metric anomalies, pulling related logs, cross-referencing deployment history — this alone can take 30 minutes to an hour.

After DevOps Agent adoption, this "starting from zero" disappears. The on-call engineer's first action becomes "read the Agent's findings." The agent presents "this code change in this deployment correlates with this metric anomaly — here's the proposed fix" as the starting point.

However, a new kind of pressure may emerge: "Is the root cause the Agent identified actually correct? Are there blind spots?" The tension between trusting the agent's output and risking an incorrect fix, versus distrusting it and re-investigating from scratch. Worth discussing as a team before it happens.

Required Skill Sets Change

The shift goes from "investigate from scratch when alerted" to "review the Agent's findings and judge whether anything was missed." It's similar to when CI/CD became standard — the emphasis moved from "memorize manual deployment procedures" to "design pipelines and make judgment calls when they fail." As automation advances, the ability to audit automated output and handle exceptions that automation can't becomes more important than hands-on execution skills.

For less experienced team members, learning design becomes necessary. Previously, "investigating incidents yourself" was the primary way to build incident response skills. With agents handling initial investigation, these "learn by doing" opportunities shrink.

Possible approaches include "form your own hypothesis before looking at the Agent's findings," "review Agent output with intentionally injected errors," or "regular incident response drills without the Agent." The same thinking as understanding manual deployment procedures even when you rely on CI/CD.

Can You Reduce Headcount?

For SRE team managers, this question is unavoidable. Can Frontier Agents adoption mean fewer people?

In the short term, "same headcount, broader coverage" is more realistic. Agent-handled initial investigations free up SRE team time. Whether that freed time goes to "headcount reduction" or "proactive improvements that were previously backlogged (SLO reviews, chaos engineering, architecture improvements)" is the real question. The latter likely delivers more organizational value.

SRE teams perpetually stuck in reactive incident response mode (the "firefighter" state) is a common challenge. Frontier Agents adoption is a catalyst for accelerating the shift from "firefighter" to "fire prevention engineer."

Constraints and Caveats

Reading the Numbers

AWS's published figures — "up to 75% MTTR reduction," "up to 80% faster investigations," "94% root cause accuracy" — are all preview-period customer-reported values, explicitly qualified with "up to." Whether your environment sees similar results depends on application complexity, monitoring maturity, and incident characteristics. Treat them as reference points and validate in your own environment.

WGU's "estimated 2 hours → 28 minutes" and LG CNS's "over 50% faster testing" are also results from specific situations. This article cites these numbers as material for understanding implications, not as guarantees that generalize.

Region Limitations

Both DevOps Agent and Security Agent are available in the same six regions at GA: US East (N. Virginia), US West (Oregon), Europe (Frankfurt / Ireland), and Asia Pacific (Sydney / Tokyo). Tokyo region availability is a plus for teams in Japan.

However, per New Claw Times analysis, DevOps Agent inference processing occurs in US regions regardless of the selected region. Organizations with strict GDPR or data residency requirements should verify this.

Free Trial Limitations

Both agents include a 2-month free trial, but DevOps Agent has monthly caps. Per official pricing, the trial period allows up to 10 Agent Spaces, 20 hours of incident investigation, 15 hours of prevention evaluations, and 20 hours of on-demand SRE tasks per month. Excess usage incurs standard charges. Sufficient for pre-production evaluation, but watch the limits for large-scale PoCs.

Multicloud Support Reality

DevOps Agent supports AWS, Azure, and on-premises. On-premises connection uses MCP, requiring access configuration for target tools. "It doesn't just see other environments without setup." Note that DevOps Agent does not explicitly support Google Cloud environments at GA.

Security Agent's penetration testing supports per the official GA announcement "AWS, Azure, GCP, other cloud-providers, and on-premises" — it can test any reachable endpoint regardless of cloud provider.

Security Agent: GitHub Only

As mentioned earlier, Security Agent's PR review (Code Review) only supports GitHub at GA. Organizations primarily using GitLab or Bitbucket need to factor in this constraint.

Summary: Not "Replacement" but "Redesigning the Division of Labor"

Frontier Agents don't "eliminate" SRE work — they "partition" it.

Work that shifts to agents centers on pattern recognition and correlation analysis: detecting anomalies across metrics and logs, matching against historical incidents to form hypotheses, systematically scanning code for vulnerabilities. This "intellectual labor that demands volume and speed" is where agents excel.

Work that stays human centers on judgment and decision-making: whether to apply a fix, how to assess business impact, how much risk to accept, how to evolve the architecture. These are context-dependent, requiring organizational knowledge and business priority judgment — outside the agent's scope.

For SRE team management, this is an opportunity to redesign team skill composition and on-call structure around this new division of labor. Not "agents mean we need fewer people," but "agents handle more of the routine, so humans can focus on work that demands greater judgment."

Architecture Layers That S3 Files Eliminates — and Creates

Kento IKEDA — Thu, 09 Apr 2026 06:56:45 +0000

On April 7, 2026, AWS made Amazon S3 Files generally available. It lets you mount S3 buckets as NFS v4.1/v4.2 file systems from EC2, EKS, ECS, and Lambda.

Launching S3 Files, making S3 buckets accessible as file systems | AWS News Blog

aws.amazon.com

There are already plenty of setup guides and first-look posts. This article focuses on something different: what becomes unnecessary and what becomes possible in your architecture.

If you use S3 regularly and are wondering "this sounds big, but how does it actually affect my architecture?" — this is for you.

The Problem S3 Files Is Solving

Let's start with a shared understanding.

Say an ML team needs to preprocess training data. The raw data is in S3. They want to use pandas. While pd.read_csv("s3://my-bucket/data.csv") works, under the hood boto3 issues GET requests and loads data into memory. Writing results back requires PUT. This is fundamentally different from open("./data.csv").

At scale, this becomes an architectural problem. Many organizations operate pipelines like this:

Copy from S3 to EFS/EBS, process, write results back to S3. This "middle copy layer" exists solely to bridge the I/O model gap between object storage and file systems. Maintaining sync scripts, managing consistency during copies, and provisioning EFS — all of this overhead comes from that gap.

S3 Files aims to eliminate this gap entirely.

From the application's perspective, S3 data appears as a local directory. pd.read_csv("/mnt/s3files/data.csv") reads from S3 behind the scenes, and df.to_csv("/mnt/s3files/result.csv") automatically commits changes back.

The full technical overview is in the official documentation.

docs.aws.amazon.com

Why This Isn't Just Another Mount Feature

If "mount S3" sounds familiar, you might be thinking of Mountpoint for Amazon S3 or Google Cloud's Cloud Storage FUSE (gcsfuse). S3 Files has a fundamentally different architecture.

The Difference from FUSE-Based Tools

FUSE-based tools emulate file system behavior on top of S3's API. In Mountpoint for Amazon S3, for example, overwriting a file means deleting the old object and PUTting a new one. Partial file writes — a basic file system operation — aren't supported. Directories don't actually exist, leading to inconsistencies with empty directories.

S3 Files doesn't emulate. It connects EFS (Elastic File System), a real NFS file system, to S3. The file system side provides real NFS semantics, and the S3 side remains real S3 objects. Two distinct systems coexist with an explicit synchronization layer between them.

This matters in practice: appending to a WAL (Write-Ahead Log) or editing part of a config file works with byte-level writes on the file system side, periodically synced to S3 as whole objects. With FUSE, these operations require re-PUTting the entire object.

What "Stage and Commit" Actually Does

Andy Warfield, VP and Distinguished Engineer at AWS, describes the sync model as "stage and commit" in his post on All Things Distributed, explicitly noting it's "a term borrowed from version control systems like git" (official documentation uses "synchronization" instead).

File system changes are like working directory changes in Git. They aren't immediately reflected in S3 — instead, they're batched and committed as S3 PUTs approximately every 60 seconds. In the other direction, when objects are updated in S3 (e.g., via PutObject from another application), the official documentation states changes are reflected in the file system "typically within seconds." DevelopersIO's hands-on testing measured approximately 30 seconds.

Amazon S3 Files が GA — S3 バケットをファイルシステムとしてマウント、EFS と比較してみた | DevelopersIO

2026年4月提供開始のAmazon S3 Filesは、S3バケットをNFS v4.2でマウント可能にする新サービス。EC2/Lambda/EKS/ECSから利用でき、既存レガシーアプリケーションのコード変更なしでS3を活用できます。

dev.classmethod.jp

If both sides modify the same file simultaneously, S3 is the source of truth. The file system version is moved to a lost+found directory, with a CloudWatch metric indicating the conflict.

This is a deliberate tradeoff: not a real-time shared file system, but one that tolerates tens of seconds of delay in exchange for preserving both file and object semantics without compromise.

According to Warfield's post, the team initially tried to make the boundary between files and objects invisible, but every approach forced unacceptable compromises on one side or the other. They ultimately decided to make the boundary itself an explicit, well-designed feature. His post is essential reading for understanding the "why" behind S3 Files.

S3 Files and the changing face of S3 | All Things Distributed

Andy Warfield writes about the hard-won lessons dealing with data friction that lead to S3 Files

allthingsdistributed.com

Architecture Layers That Disappear

Here's the core of this article: what specific architectural patterns does S3 Files make unnecessary?

1. S3 → EFS/EBS Staging Pipelines

Consider a daily retraining pipeline for a recommendation model. Purchase logs accumulate in S3, and preprocessing involves data cleansing → feature generation → format conversion.

Previously, every time an EC2 or SageMaker Processing Job starts, it first downloads data from S3 to EBS. For 100GB of training data, depending on instance network bandwidth, the download alone takes several minutes. After processing, results are uploaded back to S3, and the EBS volume is cleaned up. Of the four steps — download → process → upload → cleanup — only "process" is the actual work.

With S3 Files, you mount the S3 prefix (e.g., s3://ml-data/purchase-logs/) and your processing script reads and writes /mnt/s3files/purchase-logs/ directly. Download, upload, and cleanup steps disappear.

Note: if a downstream job needs to read results via the S3 API immediately, the ~60-second commit delay matters. If both jobs use the same mount point, this isn't an issue. For S3 API consumers, design around S3 event notifications or explicit waits.

2. Lambda's "/tmp Download" Pattern

Consider a Lambda function that generates thumbnails when images are uploaded to S3. The traditional implementation:

# Traditional: Download → Process → Upload
import boto3
from PIL import Image

s3 = boto3.client('s3')

def handler(event, context):
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']

    download_path = f'/tmp/{key.split("/")[-1]}'
    s3.download_file(bucket, key, download_path)

    img = Image.open(download_path)
    img.thumbnail((128, 128))
    thumb_path = f'/tmp/thumb_{key.split("/")[-1]}'
    img.save(thumb_path)

    s3.upload_file(thumb_path, bucket, f'thumbnails/{key}')

With S3 Files mounted:

# S3 Files: Operate directly on mounted paths
from PIL import Image

def handler(event, context):
    key = event['Records'][0]['s3']['object']['key']

    img = Image.open(f'/mnt/s3files/{key}')
    img.thumbnail((128, 128))
    img.save(f'/mnt/s3files/thumbnails/{key}')

You don't even need to import boto3. The same code you'd write for local development works as-is.

Beyond code simplicity, Lambda functions are freed from /tmp capacity constraints (default 512MB, max 10GB). For functions referencing multi-GB ML models, cold start download time directly impacted latency. S3 Files pre-fetches files below a configurable threshold (default 128KB) alongside metadata, and fetches larger files on demand. Warfield calls this "lazy hydration" in his post — you can start working immediately even with millions of objects in the bucket.

docs.aws.amazon.com

3. Self-Managed EFS + S3 Sync

If your organization uses S3 as a data lake but needs EFS for real-time processing or interactive analysis, you likely have DataSync, Step Functions, or cron scripts bridging the two. Maintaining this sync logic — detecting new objects, identifying diffs, retry on failure, consistency during sync, cleanup of stale EFS files — is a significant operational burden.

S3 Files replaces this with managed synchronization. Per the official documentation, import from S3 runs at up to 2,400 objects/second, and export to S3 uses ~60-second batch windows. Unused file data is automatically evicted from the file system cache (configurable from 1 to 365 days, default 30) but never deleted from S3. File system storage costs scale with your active working set, not your total dataset.

docs.aws.amazon.com

4. Adapter Layers for Legacy Applications

Log aggregation tools watching /var/log/, build systems reading from /src/, config management tools writing to /etc/ — these applications assume open() / read() / write() and rewriting them for the S3 SDK is often impractical.

Previously, "put files on EFS, back up to S3 as needed" was the pragmatic solution. S3 Files lets you keep S3 as primary storage while applications access it via NFS mount. POSIX permissions and file locking (flock) are supported, making migration possible with a mount point change and zero code changes.

New Architecture Patterns

What becomes practically feasible for the first time?

1. Two-Tier Read Optimization

S3 Files uses a two-tier architecture internally. The first tier, "high-performance storage," caches small, frequently accessed files with sub-millisecond to single-digit millisecond latency per the official documentation. The second tier is S3 itself — reads of 1MB or larger are streamed directly from S3 even if data is cached locally, because S3 is optimized for throughput. Notably, these large reads incur only S3 GET request costs with no file system access charge.

Official performance specifications:

Metric	Value
Max read throughput per client	3 GiB/s
Aggregate read throughput per file system	Terabytes/s
Max read IOPS per file system	250,000
Aggregate write throughput per file system	1–5 GiB/s (varies by region)
Max write IOPS per file system	50,000

docs.aws.amazon.com

For context: EBS gp3 provides 125 MiB/s baseline throughput, scalable to 2,000 MiB/s (~2 GB/s) with additional provisioning. io2 Block Express maxes out at 4 GB/s. S3 Files delivers comparable read throughput without any volume provisioning.

From spec values alone: reading a 100GB dataset sequentially takes ~13 minutes at gp3 default (125 MiB/s) versus ~33 seconds at S3 Files maximum (3 GiB/s). Actual throughput depends on workload and instance type, but the order-of-magnitude difference matters. And since 1MB+ reads are billed at S3 GET rates only, heavy sequential reads essentially incur no file system charges.

2. Large Reference Data in Lambda

Previously, Lambda functions using large reference data had three options: container images with embedded models (max 10GB, rebuild on every model update), EFS mounts (requires VPC, tends to increase cold starts), or S3 downloads to /tmp (max 10GB, download time added to cold starts). S3 Files is a fourth option: mount the S3 prefix, read model files via the file system. Model updates require only an S3 upload — no Lambda redeployment needed.

Unlike EFS mounts, the backend is your standard S3 bucket, so S3-native features like versioning, lifecycle policies, and cross-region replication work as-is.

3. AI Agent Access to S3 Data

Coding agents like Claude Code, Codex, Kiro, and Cursor use file system operations as their primary data access method: ls to list files, cat to read, editor to modify and save. It's the Unix toolchain.

Of course, agents can access S3 through other means — running aws cli commands, calling S3 APIs via MCP servers or Skills/Powers, generating boto3 code. But all of these are indirect compared to file operations and add reasoning steps. To search S3 logs, a file system lets you write grep -r "ERROR" /mnt/s3files/logs/ in one line, while the S3 API requires listing objects, downloading individually, and searching locally.

With S3 Files mounting the bucket, this indirection disappears. To the agent, S3 data is just another directory under /mnt/s3files/.

Warfield's post describes AWS engineering teams using Kiro and Claude Code hitting the problem of agent context windows compacting and losing session state. With S3 Files, agents write investigation notes and task summaries to shared directories, and other agents read them. When sessions end, state persists on the file system for the next session.

File locking (flock) supports mutual exclusion across agents and processes. However, S3 API access bypasses file locks — if you write from both the file system and S3 API simultaneously, locking won't protect you.

Constraints and Decision Criteria

S3 Files isn't universal. Key constraints to evaluate:

Commit Interval: ~60 Seconds (By Design)

Writes take ~60 seconds to appear as S3 objects. If job B reads via S3 API immediately after job A writes via the file system, job B may see stale data.

This isn't just a limitation — it's a cost optimization. Per the official documentation, consecutive writes to the same file are aggregated within the 60-second window and committed as a single S3 PUT, reducing S3 request costs and versioning storage overhead.

Sync throughput per the official performance specification: S3 → file system at up to 2,400 objects/s and 700 MB/s; file system → S3 at up to 800 files/s and 2,700 MB/s.

No "commit now" API exists at GA. Warfield mentions this as an area for future improvement. Workarounds: pass data between jobs via the file system (same mount point), or trigger downstream jobs via S3 event notifications.

Rename Costs

S3 has no native rename. File system renames are implemented as copy + delete internally. Per the official performance specification, renaming a directory of 100,000 files completes instantly on the file system, but takes several minutes to reflect in the S3 bucket. During that window, the file system shows the new path while S3 still has the old keys. S3-side request costs (100K CopyObject + 100K DeleteObject) are also non-trivial.

Buckets Exceeding 50 Million Objects

Warfield's post warns about mounting buckets with more than 50 million objects (this figure doesn't currently appear on the official quotas page). Consider mounting a specific prefix to narrow the scope.

VPC Requirement

Mount targets live inside a VPC. Lambda functions and EC2 instances must connect from subnets in the same AZ as the mount target. Per the official documentation, supported compute services are EC2, Lambda, EKS, and ECS. On-premises or cross-cloud resources are not in the supported list.

Namespace Incompatibilities

Some S3 object keys can't be represented as POSIX filenames: keys ending with /, keys containing POSIX-invalid characters, or path components exceeding 255 bytes. See the official quotas page for the full list.

This is intentional. Per Warfield's post, the team chose to pass through the vast majority of keys that work in both worlds and emit events for incompatible ones rather than silently converting them.

Versioning Required

S3 Files requires S3 bucket versioning. For existing buckets, evaluate the storage cost impact (old versions are retained) and compatibility with existing lifecycle rules.

Decision Flowchart

What to Review in Your Existing Architecture

First, inventory your pipelines for "copy from S3, process, write back to S3" patterns. Batch processing and ML preprocessing pipelines with EBS/EFS staging layers are prime candidates for replacement.

Second, consider how storage choices change for new projects. "Put it in S3 now, access it as a file system later" is now a viable strategy, reducing the urgency of early "object vs. file system" decisions.

Third, audit Lambda functions that explicitly download to / upload from /tmp. Functions handling large reference data or sharing data across invocations are worth evaluating.

S3 started 20 years ago as an object store. With Tables, Vectors, and now Files, it has expanded how data can be accessed. S3 Files removes one more architectural constraint imposed by storage choices. It won't apply to every workload, but for organizations where "the data is in S3 but the tools need a file system" — and that's a lot of organizations — the impact is significant.

AI-Powered Test Case Review with MagicPod MCP Server and Claude

Kento IKEDA — Mon, 30 Mar 2026 03:55:38 +0000

Test Automation has Advanced. Is Review Also Part of the Loop?

The number of teams automating E2E tests with no-code tools like MagicPod is increasing. The hurdle for creating test cases has certainly lowered. However, who ensures the quality of the created test cases, and how?

For source code in product development, there is a place where code management and review are integrated, such as GitHub Pull Requests. On the other hand, no-code test automation tools often lack an equivalent environment, leaving the establishment of a review system to the team's will and ingenuity. Often, the creator of a test case remains the only person who understands it deeply. It is worth pausing to consider whether this state is leading to individual dependency (siloing).

In this article, I will introduce a mechanism for AI review of test cases by combining the official MCP server provided by MagicPod with Claude. I have summarized it in a reproducible form, from setup to actual review.

Target Audience of This Article

Teams operating test cases in MagicPod but without a functioning review process.
QA engineers, developers, and PdMs who feel there are challenges with test quality.
Those interested in MCP servers and AI agents but lacking a specific image of how to use them.

To ensure that even those unfamiliar with terminal operations can reproduce this, I have also included GUI-based procedures.

Overview: What to Do and How

The overall structure of the mechanism is simple:

Connect the MagicPod MCP server to Claude Desktop.
Prepare a Skill file that defines the review criteria.
Simply instruct "Review this," and the AI will fetch, analyze, and output a report of the test cases.

The MagicPod MCP Server is an official module for operating various MagicPod functions from AI agents (Claude, Cursor, Cline, etc.). It is published on GitHub as MIT-licensed OSS, and no additional costs are incurred on the MagicPod side.

As of March 2026, there are mainly four things you can do via the MCP server:

Execute tests via Web API.
Retrieve test execution information (statistical information, etc.).
Refer to help pages to suggest usage or troubleshooting.
Create and edit test cases in natural language via Autopilot.

In the AI review described in this article, we use the "retrieval of test case information" among these. It only reads test cases and does not make changes to existing tests. Note that test creation, editing, and execution are only supported in cloud environments, and not in local PC environments—a constraint to keep in mind. All you need is a MagicPod contract and a Claude subscription.

Furthermore, the official help also introduces how to identify unstable locators using the MCP server. This content is also included in the review criteria of this article, showing that MagicPod also envisions improving test case quality via MCP.

Important Note: The official help explicitly states that user information is not used for machine learning via the MCP server, and MagicPod does not retain the entered prompt information. The Web API token is written in the MCP server's config file but is designed not to be passed to the AI agent.

Why Explain with Claude Desktop?

In this article, I explain the procedure using Claude Desktop because it is an environment that is easy to introduce even for those not used to CLI or terminals. Terminal operation is limited to just one edit of a configuration file; everything else is completed within the chat UI. It can be set up via GUI, and the MCP connection status can be checked on the screen, making it suitable as a first step for those encountering MCP servers for the first time.

If You Use Other Tools

Since the MagicPod MCP server complies with MCP (Model Context Protocol), it can be used from any AI tool that supports MCP. The review criteria and prompts in this article are designed to be general-purpose and tool-independent.

If you use Cursor

Cursor has a mechanism called Project Rules, and you can place the Skill file from this article directly as a Rule. An article by Hacobu ("Trying to create an AI review mechanism with Cursor × MagicPod MCP Server") is a helpful preceding case for Cursor users. Refer to Cursor's official documentation for how to set up the MCP server.

If you use Claude Code

Claude Code also supports MCP servers. You can achieve equivalent operation by writing the contents of the Skill file in CLAUDE.md or specifying the MCP server with the --mcp-config option. This might be easier for those comfortable with the terminal.

If you use ChatGPT

Since September 2025, ChatGPT has supported MCP servers in Developer Mode. However, it only supports remote servers (SSE / streaming HTTP) and does not support local execution (stdio). Since the MagicPod MCP server is a stdio method launched via npx locally, you need to use something like ngrok to create a tunnel to connect directly from ChatGPT. In terms of ease, Claude Desktop or Cursor are simpler to set up.

Other MCP-compatible tools (Cline, Windsurf, etc.)

If you add the MagicPod MCP server to the MCP server configuration file (equivalent to claude_desktop_config.json), it will work with any tool. The review criteria prompts are plain text, so there is no need to convert them to tool-specific formats.

Note: Although the steps in this article assume Claude Desktop, the design of the review criteria and the content of the Skill file are the essence of this mechanism. Choose the tool based on your preference.

Setup (5 minutes)

The following steps are for the first time only. Once the setup is complete, the AI will autonomously execute reviews according to the Skill file.

Prerequisites

Have a MagicPod account and access to the project to be reviewed (Free trial is also possible).
Claude Desktop must be installed (Mac / Windows).
Must be subscribed to Claude Pro or higher.

Step 1: Get MagicPod API Token

Access https://app.magicpod.com/accounts/api-token/, issue a token, and copy it.

Step 2: Install Node.js (if not already installed)

npx is required to run the MCP server. Please install the LTS version from https://nodejs.org/.

Step 3: Edit Claude Desktop Configuration File

Open Claude Desktop.
Select Claude → Settings... from the menu bar.
Click Developer on the left menu → Edit Config.
The folder containing the configuration file will open in Finder (or Explorer on Windows). Open claude_desktop_config.json with a text editor.
Replace the content with the following and save:

{
  "mcpServers": {
    "magicpod-mcp-server": {
      "command": "npx",
      "args": ["-y", "magicpod-mcp-server", "--api-token=PASTE_YOUR_TOKEN_FROM_STEP_1_HERE"]
    }
  }
}

Replace PASTE_YOUR_TOKEN_FROM_STEP_1_HERE with the API token string you copied in Step 1. The token should directly follow --api-token= (e.g., --api-token=abc123def456).

Step 4: Restart and Confirm Connection

Completely quit Claude Desktop and restart it.

Note: To check the MCP after restarting, look under the + → Connectors below the chat input field. It's okay if magicpod-mcp-server is displayed. If it doesn't appear in an already open chat, try opening a new chat.

As a functional check, type the following into the chat:

Are you connected to MagicPod? Please tell me the list of available organizations and projects.

If the organization name and project list are returned, the setup is complete.

Implementing the Skill File

Once setup is finished, the next thing to do is place the Skill file.

What is a Skill File?

A Skill file is a Markdown file that teaches an AI agent how to perform a specific task. Based on the Agent Skills Open Standard, the same SKILL.md format can be used across multiple tools like Claude Code, Cursor, Gemini CLI, and Codex CLI.

The Skill file provided in this article combines the following into a single file:

Target organization and project names for review.
Review criteria (what to check, importance levels).
Output format (findings for each test case + summary).
Report generation (sharing formats for Slack/Confluence).

Placement Method

Tool	Placement Location
Claude Desktop / claude.ai	Create a project and add the file as Knowledge
Claude Code	Place at `.claude/skills/magicpod-review/SKILL.md`
Cursor	Place in `.cursor/rules` (Loaded as Project Rules)
Other MCP Tools	Add to each tool's rule/knowledge location

Copy the full text of the Skill file at the end of this article.
Save it with the filename magicpod-review.md.
Rewrite {Organization Name} and {Project Name} in the file.
Add it to the placement location for your tool according to the table above.

If you have multiple projects, rewrite the "Basic Settings" section of the Skill file as follows:

## Basic Settings

- Target Organization: MyOrganization
- Target Projects (Review all projects in order if none specified):
  - ProjectA (Browser)
  - ProjectB (Mobile App)
  - ProjectC (Browser)

For Claude Desktop:

Create a new project from "Projects" (e.g., "MagicPod Review").
Select "Files" and upload magicpod-review.md.
Open a new chat within that project.

Contents of the Skill File: Designing Review Perspectives

The core of a Skill file is the definition of review perspectives. Here, based on the 10 perspectives from the MagicPod official blog post, 10 Ideas for Test Automation Review Perspectives, we have categorized them into those that are easy for AI to detect and those that require human judgment.

This classification is a crucial point that determines the accuracy of the Skill file. While an AI will function even if you simply ask it to check everything, the results will be a mix of irrelevant comments and useful suggestions, making it time-consuming to filter through them. By deciding in advance what to leave to the AI and what should be reviewed by a human, the reliability of the output increases, bringing the quality closer to a level where review results can be shared with the team as they are.

AI Detectable Criteria

Criterion	What to detect	Importance
Missing Assertions	Test cases with zero verification commands	High
Fixed Wait Times	Use of "Wait X seconds" type commands	Medium
Unstable Locators	XPath index dependency (e.g., `div[2]/button[1]`)	Low
Naming Issues	Empty test case names or description fields	Medium
Mechanical Names	Auto-generated names like "Button Element (1)"	Medium
Abandoned Steps	Steps left as "Temporarily Disabled"	Low
Long Tests	Over 200 steps	High
Unused Shared Steps	Repeated patterns not using shared steps	Medium

Criteria requiring human judgment (AI only provides information)

Criterion	Information provided by AI
Validity of Assertions	List of assertions in use (Team decides if appropriate)
Dependencies	List of test cases not using session restart

Note: MagicPod Web API's human_readable_steps does not include step line numbers and cannot distinguish whether a step is disabled. Identifying the specific location of a finding must be based on step content.

Are These 10 Perspectives Sufficient?

To conclude, they function well for a primary screening in AI reviews, but they are not a silver bullet. Based on the results of reviewing over 60 test cases across multiple projects, here are the strengths and limitations of these 10 perspectives.

Fields well-covered: Structural issues

Among the 10 perspectives, issues such as missing assertions (Perspective 1), naming deficiencies (Perspective 4), overly long tests (Perspective 7), and underutilized shared steps (Perspective 8) can be identified by looking at the structure of the test case. AI excels at this type of pattern matching and was able to detect these with high accuracy during actual reviews. In particular, empty description fields were found in many test cases, and it can be said that this single perspective alone made the review worthwhile.

Blind spots: Validity of test design

On the other hand, as mentioned in the official blog as other perspectives, consistency with test design documents and the validity of test data are not included in these 10 perspectives. These are difficult to judge within a MagicPod review alone, as they require cross-referencing with test design documents or product specifications. Therefore, this is currently a heavy burden for AI reviews via MCP.

Findings outside the 10 perspectives

In this review, points were raised that were not included in the original 10 perspectives. These were all observations regarding test cases created to verify MagicPod's behavior, but it is noteworthy that the AI picked up on such remnants.

Broken steps (UI element: not configured). Left incomplete without being configured.
Test data containing passwords in plain text.
A mixture of tests for checking MagicPod behavior and production regression tests within the same project.

These are examples where the AI picked up additional issues from the context without being bound by the list of perspectives in the prompt. In other words, while specifying the 10 perspectives in the prompt, there is room for the AI to expand upon them autonomously. I feel that it is important for the practical operation of AI reviews to not restrict the perspectives too strictly and to leave some margin for the AI.

In summary, the 10 perspectives are well-balanced for covering test case implementation quality and can be recommended as a baseline for AI reviews. Reviews that require test design validity or product-specific domain knowledge remain the role of humans.

How to Use

If you have already finished placing the Skill file, the usage is simple.

Basics: Running a Review

Just type one of the following into the chat:

Review this

Check the test cases

Since the organization name, project name, viewpoints, and output format are all defined in the Skill file, the AI will autonomously perform the following steps with just that one phrase: "Retrieve test case list -> Analyze each step -> Check according to viewpoints -> Output according to the format."

Note: Please execute this from the chat within the project where the Skill file is located. If you type "Review this" in a regular chat (outside of the project), the AI will ask "What would you like me to review?" because the Skill file will not be loaded.

Sharing the Report

Once the review results are available, you can also generate reports for sharing.

Summarize for Slack

Create a report in a format that can be pasted into Confluence

The Skill file includes templates for both Slack mrkdwn and standard Markdown formats, ensuring the output matches your intended destination.

In environments where the Slack MCP connector is connected, entering "Summarize for Slack" will present you with options such as:

Post directly to a channel (posts immediately if a channel name is specified)
Save as a draft (for when you want to review the content before posting)
Output only the mrkdwn text (for manual copying and pasting)

If the Confluence or Jira MCP connectors are connected, you can also create Confluence pages or Jira tickets directly from the chat. Even in environments without active MCP connectors, text output for copying and pasting is always available.

The following is an example of a report output for Slack (project names and other details have been replaced with placeholders).

🔍 MagicPod Test Case AI Review Results
Date: 2026/03/28 | Target: SampleOrg (4 projects / Over 60 cases)

🔴 High: 3 cases 🟡 Medium: Many 🟢 Low: 0 cases
━━━━━━━━━━━━━━━━━━━━━━━
🚨 Top 3 Priority Improvement Actions

1. Description Field Entries - Medium
The description field is empty in many of the target test cases.

2. Organization of Behavior Verification Tests - Medium
Tests used to verify behavior during initial setup still remain in the old folder.

3. Replacement of Fixed Wait Times - Medium
Wait 3 seconds and Wait 5 seconds are used extensively throughout ProjectD.
━━━━━━━━━━━━━━━━━━━━━━━
1. ProjectA_Web (22 cases)
- Description field is empty: Almost all cases
- Mechanical UI element names: Area(8), Area(119), etc.
- Regression tests have substantial assertions and appropriate comment separation.

2. ProjectB_Web (23 cases)
- Description field is empty: All cases
- Shared steps not utilized: Screen verification patterns are identical in 10 or more cases.
- Naming conventions are well-consistent across all test cases.

(Details for the other 2 projects are in the thread below 👇)

The report features a three-tier structure: Priority Improvement Actions, followed by Project Summaries, and then Details in Threads. This format allows the team to understand exactly what to do first at a glance. By also including positive points (💡), I have ensured the report provides encouragement rather than just pointing out issues.

Customization Examples

Review while excluding the old folder.

Review only ProjectB_Web.

Show the differences from the last time.

The Skill file defines rules to handle these types of instructions, so it works simply by adding conditions in natural language.

Results of the Actual Implementation

We executed AI reviews on more than 60 test cases across 4 projects. The following are the results.

Most Common Finding: Numerous Empty Description Fields

A large number of the target test cases had empty description fields. This is likely a common situation across many teams. When only the test case name is provided, the intent is not conveyed effectively, which leads to lower accuracy in reviews, handovers, and AI utilization.

While MagicPod features an AI summarization function, it summarizes the actions of the steps. The underlying purpose, such as what specifically needs to be verified in the test, must be written by a human.

Typical Findings in Learning Test Cases

Test cases created at the beginning of projects to verify MagicPod's behavior were still present. Although they were stored in an old folder separate from production regression tests, the AI review accurately detected issues with them.

A test named Test 1. The purpose was unclear, and the description field was empty.
Missing assertions. The test only performed login operations without a step to verify that the login was successful. It was just running through operations without guaranteeing anything.
UI element names like Area(119). Mechanical names automatically generated by MagicPod remained, making it impossible for a third party to identify what the element was.
Neglected unconfigured steps. In one test case, a broken step labeled UI element: not configured was left behind.

All of these were remnants from when MagicPod's behavior was being tested. In the production regression tests, naming conventions were unified and assertions were properly implemented. However, the AI's ability to point out that these traces of experimentation remaining in the project could become noise during bulk execution was very useful.

These are the types of issues that anyone would notice if they performed a review, but if no review is conducted, they remain neglected forever. The value of an AI review lies in mechanically picking up these problems that are obvious upon inspection but lack someone to look at them.

Discoveries Outside the Defined Perspectives

Furthermore, some findings emerged that were not included in the ten initial perspectives.

Frequent use of fixed wait times. In one project, wait 3 seconds and wait 5 seconds were scattered across multiple test cases.
Visualization of quality differences between projects. By reviewing across four projects, it became immediately clear that the rate of description completion varied significantly by project.

While following the list of perspectives, the AI sometimes picks up additional issues from the context. This behavior is similar to that of a human reviewer and can be seen as an advantage of not strictly limiting the prompts.

Detecting Positive Aspects

AI reviews were also effective for detecting good practices, not just pointing out flaws.

Test cases where sections were appropriately divided using comments (//).
Configurations where shared steps were utilized, keeping the login process DRY (Don't Repeat Yourself).
Projects where test case naming conventions were unified across all test cases.

Reports that only contain criticisms can lower team motivation. Including notes on what is well-done increases the overall acceptability of the report.

Limitations and Proper Usage

AI review is not a silver bullet. It is necessary to recognize the following limitations in advance.

What AI is good at:

Structural issues that can be detected via pattern matching (naming conventions, presence of assertions, number of steps)
Bulk analysis and comparison across projects
Consistency in maintaining the same check standards no matter how many times it is executed

What AI is not good at:

Semantic-level judgment, such as whether this assertion is appropriate for the purpose of this test
Checking consistency with test design documents (when those documents are outside of MagicPod)
Team-specific context (e.g., there is an unavoidable reason for this fixed-time wait)

In practice, a realistic approach is a two-tier structure: use AI review for self-review or primary screening, and then have a human review only the points that require human judgment. A workflow is becoming widespread where an AI review runs first on a code Pull Request, and humans focus on making decisions after reviewing the summary. The same structure can be applied to test case reviews.

Summary

The reason test case reviews become dependent on specific individuals often stems from the lack of a formal review system. When review criteria are vague and reviewers lack sufficient time, the reviews themselves eventually stop happening.

The MagicPod MCP server combined with AI offers a solution to this problem, providing consistent primary screening rather than a perfect review. In this experiment, the fact that many test case description fields were empty is something a human reviewer could also point out. What the AI did was simply look at them.

If you are already using MagicPod, setup takes only five minutes, and it takes about ten minutes even including the placement of the Skill file. The only additional cost is the fee for Claude. Why not start by typing review into one of your projects and seeing what kind of feedback you get?

Appendix: Full Text of the Skill File

The review perspectives, output formats, and report sharing features explained in this article are all included in this single file. Please replace {Organization Name} and {Project Name} with your own environment and add the file to the location mentioned earlier.

# MagicPod Test Case AI Review Skill

## About This File

Placing this file in Claude Desktop's Project Knowledge or CLAUDE.md enables automated review of test cases via the MagicPod MCP server.

## Prerequisites

- The MagicPod MCP server must be connected.
- You must know the target Organization Name and Project Name.

## Basic Settings

- Target Organization: {Organization Name}
- Target Project: {Project Name}
- Review Output Language: English

## How to Execute a Review

Execute the review when the user gives any of the following instructions:
- "Review this"
- "Check the test cases"
- "Please do a MagicPod review"

### Execution Steps

1. Retrieve the list of test cases for the target project via the MagicPod MCP server.
2. Retrieve step details for each test case.
3. Check them according to the review perspectives below.
4. Output the results according to the specified format.

## Review Perspectives

Reference: https://magicpod.com/blog/testcase-review-idea/

The following perspectives are categorized into "Automatic Detection" and "Human Judgment Support." 
For automatic detection, always point out any relevant findings. 
For human judgment support, present the information and leave the final decision to the user.

### Automatic Detection (Point out if applicable)

#### Perspective 1: Missing Assertions
- Detect test cases that do not contain any verification commands (assert-type or "verify"-type).
- Tests consisting only of operations do not guarantee anything.
- Severity: High

#### Perspective 2: Use of Fixed-Time Waits
- Detect locations using "Wait X seconds" or "Fixed-time wait" commands.
- Recommend replacing them with condition-based waits (e.g., "Wait until element is displayed").
- If their use is unavoidable, a reason should be left in the comments.
- Severity: Medium

#### Perspective 3: Unstable Locators
- Detect index-dependent locators in XPath (e.g., //div[@id='root']/div[2]/button[1]).
- Only individually created locators are subject to check (AI self-healing targets have lower priority).
- Severity: Low (Since these are naturally discovered if run daily).

#### Perspective 4: Deficient Test Case Names or Descriptions
- Test case name is empty or does not follow naming conventions.
- Description field is empty (the purpose of the test is not described in text).
- Severity: Medium

#### Perspective 5: Remaining Mechanical UI Element Names
- Automatically generated names like "Button element (1)" or "Text input (2)" remain as-is.
- Meaningful names like "Email address input field" or "Login button" are ideal.
- Severity: Medium

#### Perspective 6: Neglected Disabled Steps
- Detect steps that remain in a "Temporarily disabled" state.
- Unnecessary steps hinder the understanding of the test's intent.
- Severity: Low

#### Perspective 7: Overly Long Tests
- Detect test cases exceeding 200 steps.
- Negatively affects readability, stability, and maintainability.
- Severity: High

#### Perspective 8: Underutilization of Shared Steps
- Detect suspected cases where the same operation patterns are hard-coded across multiple test cases.
- Typical examples include login processes or test data initialization.
- Severity: Medium

### Human Judgment Support (Information presentation only)

#### Perspective 9: Validity of Assertions
- List the assertions (verification commands) used in each test case.
- Leave it to the user to judge whether they are appropriate for the objective.
- Identify whether URL verification, image diff, element value verification, visibility check, or title check is used.

#### Perspective 10: Dependencies Between Test Cases
- List test cases that do not use session restart.
- If dependencies exist, check if that fact is noted in the description field.

### Constraints on API Specifications

The `human_readable_steps` in the MagicPod Web API has the following constraints:
- Step line numbers are not returned: When pointing out issues, identify them by step content rather than "Step X."
- It is unclear if a step is disabled: Add a note that the detection accuracy for Perspective 6 is limited.

## Output Format

### Findings per Test Case

```

markdown
## Test Case: {Test Case Name}

| # | Perspective | Finding | Severity |
|---|-------------|---------|----------|
| 1 | Missing Assertions | No verification commands exist | High |
| 4 | Naming Deficiency | Description field is empty | Medium |


```

### Summary (Always output at the end of the review)

```

markdown
## Review Summary

- Total Test Cases: X
- Cases with Findings: Y
- Cases without Findings: Z

### Tally by Perspective
| Perspective | Count |
|-------------|-------|
| Missing Assertions | X |
| Fixed-Time Wait | X |
| ... | ... |

### Top 3 Priority Improvement Actions
1. ...
2. ...
3. ...


```

## Setup Guide (No terminal required)

### Step 1: Obtain MagicPod API Token
1. Access https://app.magicpod.com/accounts/api-token/
2. Copy the token string.

### Step 2: Install Node.js (Only if not already installed)
1. Access https://nodejs.org/
2. Download and install the LTS version.
3. This enables `npx`, which is required to run the MCP server.

### Step 3: Edit Claude Desktop Configuration File
1. Open Claude Desktop.
2. Select Claude > Settings... from the menu bar.
3. Select "Developer" from the left menu.
4. Click the "Edit Config" button. The folder containing the config file will open in Finder/Explorer.
5. Replace the content with the following and save:

```

json
{
  "mcpServers": {
    "magicpod-mcp-server": {
      "command": "npx",
      "args": ["-y", "magicpod-mcp-server", "--api-token=PASTE_TOKEN_HERE"]
    }
  }
}


```

Note: If other MCP servers are already configured, add this entry inside the `mcpServers` object.

### Step 4: Restart Claude Desktop
1. Completely quit Claude Desktop (Cmd+Q).
2. Start it again.
3. Connection is successful if the MCP tool icon appears below the chat input field.

### Step 5: Verify Operation
Enter the following in the chat:
```

plaintext
Are you connected to MagicPod? Please tell me the list of available organizations and projects.


```

### About Costs
- MagicPod MCP server itself: Free (MIT licensed OSS)
- Required: MagicPod main subscription + Claude Pro subscription
- For review purposes (read-only), no additional MagicPod costs are incurred.


## Report Generation and Sharing

After the review is complete, generate a report for sharing if the user gives any of the following instructions:
- "Make a report" / "Summarize this"
- "I want to share this on Slack" / "I want to post this to Confluence"
- "Summarize for sharing"

### Format Based on Destination

#### For Slack Posting

Output using Slack mrkdwn syntax. Assuming long texts will be split into threads, the main body should contain only key points.

```

markdown
*🔍 MagicPod Test Case AI Review Results*
Execution Date: YYYY-MM-DD

*Target*
• Project: {Project Name}
• Test Cases: X

*Summary*
• Findings: Y / No findings: Z
• 🔴 High: X  🟡 Medium: X  🟢 Low: X

*Top 3 Priority Improvement Actions*
1. ...
2. ...
3. ...

Details in thread 👇


```

Post detailed findings for each test case in the thread.

#### For Confluence / Documentation

Output a full report in Markdown format. This can be used as-is with Confluence's Markdown paste feature.

```

markdown
# MagicPod Test Case AI Review Report

## Basic Information
- Execution Date: YYYY-MM-DD
- Target Organization: {Organization Name}
- Target Project: {Project Name}
- Total Test Cases: X
- Review Perspectives: Based on MagicPod Official Blog Top 10 + Additional Perspectives

## Summary

| Metric | Value |
|--------|-------|
| Test Cases with Findings | Y (XX%) |
| Test Cases without Findings | Z (XX%) |
| High Severity Findings | X |
| Medium Severity Findings | X |
| Low Severity Findings | X |


```

```

markdown
## Tally by Perspective

| # | Perspective | Count | Severity |
|---|-------------|-------|----------|
| 1 | Missing Assertions | X | High |
| 2 | Fixed-Time Wait | X | Medium |
| 3 | Unstable Locators | X | Low |
| 4 | Naming Deficiency (Name/Desc) | X | Medium |
| 5 | Mechanical UI Element Names | X | Medium |
| 6 | Neglected Disabled Steps | X | Low |
| 7 | Long Tests (Over 200) | X | High |
| 8 | Underutilization of Shared Steps | X | Medium |
| + | Security (Plaintext passwords, etc.)| X | High |
| + | Broken Steps (not configured) | X | High |

## Top 3 Priority Improvement Actions

1. **[Action Name]** — [Reason and Expected Effect]
2. **[Action Name]** — [Reason and Expected Effect]
3. **[Action Name]** — [Reason and Expected Effect]

## Good Practices

(Include good practices detected by AI. Provide praise as well as improvements.)

## Details by Test Case

(Expand the list of findings for each test case here.)

## Supplement: Basis for Review Perspectives

This review is based on the perspectives from the MagicPod official blog "Reviewing Automated Tests? 10 Review Perspective Ideas" 
(https://magicpod.com/blog/testcase-review-idea/), reconstructed in a format suitable for AI detection.


```

### Instructions for Sharing Execution

Handling cases where the user specifies a concrete sharing destination:

#### "Post to Slack"
1. If Slack MCP server is connected: Post directly to the channel using `slack_send_message`.
2. If not connected: Output text in Slack mrkdwn format and prompt the user to copy-paste.

#### "Post to Confluence"
1. If Atlassian MCP server is connected: Create a page using `createConfluencePage`.
2. If not connected: Output in Markdown format and prompt the user to use Confluence's Markdown paste.

#### "Make a Jira ticket"
1. Propose creating Jira tickets for findings with High severity.
2. After user approval, create them using `createJiraIssue`.

### Report Customization

Adjust the report if the user specifies the following:
- "Exclude the 'old' folder" -> Generate report only for production tests.
- "Summarize across projects" -> Generate a cross-project summary.
- "Show only the good points" -> Generate a "praise" report rather than an improvement report.
- "Show the difference from last time" -> Compare with the previous report (saved in Knowledge).

```

`

When Software Development Common Sense Flips: The Law of Decreasing Generation Costs

Kento IKEDA — Sat, 14 Mar 2026 07:07:10 +0000

CLAUDE.md, .cursor/rules, Kiro Specs, Devin Playbooks. The past year has seen an explosion of instruction files for AI. While every new tool comes with a new name, doesn't what they are doing feel like deja vu?

Isnt this a requirements definition document? Isnt this onboarding material? Isnt this a runbook?

Anyone involved in software development for a long time likely feels the same way. At the same time, you might feel that some practices previously considered common sense are starting to become a hindrance in modern development.

This article explores the true nature of this deja vu and sense of misalignment.

To state the conclusion first: the design philosophy of AI coding tools is a reinvention of practices built over 20 years of human team development. In this process, part of software development common sense is structurally flipping. There appears to be a simple law to categorize what flips and what remains universal.

What are Instruction Files Reinventing?

First, let's organize the facts.

If we map the ways to provide instructions in various AI tools to what humans have done in team development for years, it looks like this:

AI Tool Concept	Human Team Development Equivalent
Kiro Specs	PRD / Requirements Definition
CLAUDE.md	Onboarding Documentation
.cursor/rules	Coding Conventions (.eslintrc, etc.)
Devin Playbooks	Operating Procedures / Runbooks
Hooks (PreToolUse, etc.)	Git hooks / CI Pipelines
Skills	Internal Common Libraries
Plugins	Templates distributed via npm/pip

Many of you have likely noticed this correspondence.

The important part is what comes next. In human team development, there was a long period of trial and error regarding the design of what to provide. Giving only coding conventions doesn't help a new member get moving. Giving only a PRD doesn't convey design intent. Giving only a procedure manual doesn't allow for exceptional judgments. The knowledge that a person becomes effective only when provided with layers of why this design was chosen, what must not be touched, and criteria for judgment when stuck has been accumulated over many years.

The instruction design for AI tools is exactly a reinvention of this knowledge.

Furthermore, the differences in design philosophy among tools can be translated into differences in what kind of organization provides what to a new member first.

Kiro is like a company that starts with specifications. Let's structure the requirements and clarify acceptance criteria before starting implementation. The design of having three stages—requirements.md, design.md, and tasks.md—aims for a middle ground between pre-definition and iteration. (Note: Kiro provides Requirements-First and Design-First workflows; here, we focus mainly on the Requirements-First flow).

Claude Code is like a company that emphasizes verbalizing implicit knowledge. The guideline to write persistent context that cannot be inferred from code is spreading as a best practice for CLAUDE.md, which is the essence of onboarding materials. It is the idea of organizing technical stacks, reasons for past design decisions, and hidden pitfalls that cannot be read from the code itself.

Cursor is like a company that gives rules and lets people work freely. You write project rules flatly in .cursor/rules and leave the rest to the agent's judgment. Because the degree of freedom is high, the quality of the rules directly affects the output quality. Conversely, since the ability to write rules serves as leverage for the team's output quality, the operational design of rules (who updates them and when) becomes implicitly important.

Devin is like a company that makes people follow procedures. It is a design where humans control the task execution flow by explicitly defining steps in Playbooks.

None of these is the single correct answer; rather, they represent design differences based on the nature of the team and the task.

Predecessors Hit the Same Walls

Now for the main topic.

Looking at the failure patterns occurring in AI tool instruction design, we notice they are structurally identical to the failures repeated in human team development over the last 20 years.

Collapse Without Documentation

Using Claude Code without writing CLAUDE.md and repeatedly re-instructing it because it's not right is almost the same structure as the silos created when starting a project without design documents.

In human teams, if the basis for design decisions is not documented, a state of it's unknown unless you ask that person is born. For AI, it is even simpler: context disappears once you cross sessions. As a result, you repeat the same explanation, yet get a different output than before. This is the very challenge stated over a decade ago: without documentation, implicit knowledge evaporates.

The efforts of Kiro to structure requirements.md in EARS format and Claude Code to persist prerequisite knowledge via CLAUDE.md are architectural answers to this evaporation.

Paralysis by Information Overload

Adding ten MCP servers only to have the response quality collapse is another common pattern.

Defining too many tools crowds the context window, and the agent begins to skim the results. As pointed out in Anthropic's article on Code execution with MCP, direct calls to MCP tools can cause token consumption to explode. One example mentioned is a case where transferring a transcript of a two-hour sales meeting from Google Drive to Salesforce could consume an additional 50,000 tokens.

https://www.anthropic.com/engineering/code-execution-with-mcp

This is the same structure as inviting a new hire to 30 Slack channels on their first day and causing them to freeze from information overload. Both humans and AI have limits to the amount of information they can process, necessitating a design that provides necessary information at the appropriate granularity. The evolution of Claude Code Skills, Cursor Rules, and Kiro Powers as mechanisms to provide only necessary information when needed is a response to this problem.

Chaos Without Specifications

Kiro Specs' design philosophy of writing specifications first is an answer to the classic failure pattern of starting development without specs and seeing the project go up in flames in the latter half.

Since AI generates code probabilistically, it will fill in the blanks of ambiguous specifications on its own. You won't know if that matches your intent until you verify the output. If you run with ambiguous specs, a game of whack-a-mole style bug fixing begins. This is the same for both human and AI development.

The Inevitability of Automation

Forcing automatic formatting or linting via Hooks upon saving files is a reenactment of the history where we moved from relying on human goodwill for reviews and style consistency to automating it with CI.

Ten to fifteen years ago, themes like Jenkins implementation, test automation, and continuous integration were frequent at web conferences. Discussions on securing quality through systems became popular, leading to build servers, the cultivation of automated testing cultures, and the establishment of deployment pipelines. The rapid development of Hooks and Skills in the context of AI coding today is exactly a reenactment of this history.

The Law: What Happens When Generation Costs Drop?

While the summary so far is that history repeats itself, another point is that some common sense is flipping.

To explain this flip, I propose a law:

When generation cost decreases, the center of gravity for value shifts from the product to the intent.

The cost of writing code has dropped dramatically. Instruct an AI, and hundreds of lines of code appear in seconds. Let's consider the structural consequences of this change.

It is the same structure as the movement of bottlenecks in the Theory of Constraints (TOC). When the cost of one process drops sharply, the bottleneck moves to another process. Ten years ago, implementation man-hours were often the bottleneck, so it was rational to invest in technologies that improved code quality, such as test automation, refactoring, and coding conventions.

Now, writing code itself has become cheap. Consequently, the bottleneck moves to what to make it write. The process of verbalizing specifications, recording design decisions, and organizing context—in other words, the process of refining intent—is becoming the bottleneck for productivity.

Note that this law relies on the current asymmetry where generation costs have dropped, but verification costs have not decreased at the same rate. If AI-driven code review or formal verification advances significantly in the future, verification costs may also drop dramatically, moving the bottleneck to yet another process. In that sense, this is not a permanent law of the universe but a law explaining current structural dynamics.

Using this law, let's categorize development common sense into what flips and what remains universal.

What Flips: Refining the Product (Code)

These are things considered a given for any good developer over the past 20 years, but for which the ROI is changing in the AI era.

Note that these haven't become meaningless; rather, they have changed from things you should always invest in to things you judge based on the situation.

DRY Principle (Flip Degree: Medium to High)

DRY (Don't Repeat Yourself) is a design principle to eliminate code duplication and centralize changes. This was a rational investment when humans were maintaining the code. Duplication leads to missed updates and becomes a hotbed for bugs.

However, in the context of AI generating code, abstraction for the sake of DRY can sometimes become a risk.

Introducing abstraction layers for unification increases the chance of the AI misreading the context. If the responsibility of a common function is unclear, the AI might use that function for unintended purposes or call it in inappropriate places. Concrete, self-contained code is often less likely to be misunderstood by AI.

Of course, DRY remains important in core parts of the codebase that humans will maintain long-term. The criteria for judgment have changed: we must now discern whether this code is a core component for long-term maintenance or a peripheral one intended for regeneration.

Refactoring (Flip Degree: Medium)

Refactoring is the process of improving internal structure without changing external behavior. It has long been valued as a means of paying off technical debt.

As code generation costs drop, the calculation changes. Which is faster and more reliable: carefully refactoring existing code over a week, or clarifying specifications and having it regenerated? The latter is increasingly becoming a realistic option.

However, this doesn't mean refactoring is unnecessary. Regeneration is only effective within the scope where specifications can be clearly verbalized. Code containing implicit specifications accumulated over years of operation—such as edge case handling not written in docs or performance tuning results—risks being lost during regeneration.

The axis of investment has shifted from always refactor to refactor or regenerate in this case.

Test Coverage (Flip Degree: Low to Medium)

The value of writing tests itself does not change. What has changed is the ROI of humans writing test code by hand. What is flipping here is the allocation of investment toward metrics, not the design principle.

If you leave identifying test cases and generating test code to AI, comprehensive tests appear in seconds. What humans should do here is design the test strategy—judging what should be tested—rather than implementing the test code.

Spending time on determining whether a test verifies a truly meaningful specification is becoming higher ROI than spending time just to raise coverage numbers.

Naming and Readability (Flip Degree: Low)

The degree of flip for investment in readability is smaller compared to other items, but judgment is beginning to change depending on who is reading. If the code is for humans to read and understand, beautiful naming and clever comments remain important.

On the other hand, in code primarily read and written by AI, type information and intent explanations in JSDoc can improve AI understanding more than beautiful naming intended for humans. The definition of readability itself is changing depending on the reader.

What is Universal: Refining the Intent

While some things flip, others have always been important and are even more worth investing in now. According to the law, these are things related to the definition of intent and judgment—processes AI cannot do.

Verbalizing Specifications and Intent

The ability to accurately verbalize what to make and why is actually increasing in value in the AI era.

It is no coincidence that Kiro Specs is designed to have you write specifications first. If you throw ambiguous specs at an AI, it will probabilistically fill in the blanks, resulting in code that sort of works but you don't know if it meets the specs. The effort to verify and fix this often exceeds the effort of writing specifications from the start.

Starting development without a PRD leads to chaos later. This lesson from 10 years ago is perfectly valid for AI development. In fact, because AI generation speed is fast, ambiguous specs can lead to a large amount of rework in a short time, arguably making the importance of specifications greater than before.

Recording Design Decisions

The culture of ADR (Architecture Decision Records)—why this configuration was chosen, what alternatives existed, and why they were rejected—is being rediscovered in the context of CLAUDE.md.

The guideline for CLAUDE.md to write persistent context that cannot be inferred from code is exactly the recording of design decisions. Information such as packages/shared/src/legacy/ is a compatibility layer for old APIs. You will want to refactor it, but do not touch it (three external systems depend on it, scheduled for removal in 2026 Q3) cannot be inferred no matter how closely an AI reads the code.

To have AI write correct code, we must provide the background of these design decisions as context. This is the same structure as onboarding a new human member; the verbalization and persistence of prerequisite knowledge is universally important across eras.

Review and Verification Ability

The value of the ability to read and judge is rising relative to the ability to write.

In the era when CI was just about if the build passes, it's okay, the quality gate was the success or failure of the build. Now, the phase where humans judge if the AI output meets the requirements is becoming the quality bottleneck.

Reviewing AI-generated code differs in quality from reviewing human-written code. AI can produce inconsistent errors and it is not rare for it to lie convincingly. To spot library version mismatches, calls to non-existent APIs, or logic that ignores instructions, one needs an understanding of the entire codebase and a deep understanding of specifications.

Since review is becoming the process that determines quality, investment here is universally important.

Organizing Context

CLAUDE.md, Steering, Rules. Though the names differ, they all represent the act of organizing prerequisite knowledge to provide to the AI.

We are in an era where the most cost should be spent on organizing preconditions before writing code. This is a direct consequence of the law: when generation cost drops, the bottleneck moves to intent. Code can be regenerated as many times as you like, but if the preconditions are wrong, it will never be correct no matter how many times you regenerate it.

Categorizing by the Law

If we look at the summary so far along one axis, the criteria for categorization are simple:

What Flips = Investment in processes AI can now do cheaply (Refining the product)
What is Universal = Investment in processes AI still cannot do (Refining the intent)

As generation costs drop, the ROI of the former decreases, and the ROI of the latter increases. This can be called a structural law that does not depend on specific tools or languages.

Predicting the Next from History

Combining this law with patterns from software development history, we can see several problems that have not yet fully materialized but are almost certain to come in the context of AI coding. While each of these themes is deep enough to be explored in its own separate article, I will only outline their structures here.

The Problem of Document Obsolescence

The issue where no one updates the internal Wiki and a new member causes an accident using old procedures has occurred repeatedly in technical organizations.

The same will happen with CLAUDE.md or Kiro Specs. Code is generated and updated by AI, but the maintenance of instruction files is done by humans. This asymmetry will eventually become a problem. A state where the code is up-to-date but the instruction file is a lie is the classic problem of code works but docs are wrong. Handling this issue may become a primary axis in future tool competition.

Kiro's Agent Hooks is a mechanism that can trigger agents based on events such as file changes, allowing for the configuration of tasks like automatic document updates. However, as pointed out in the SDD (Spec-Driven Development) tool comparison article published on martinfowler.com, drift between specification and implementation remains a challenge. Addressing this issue may become a key factor in the future competition among tools.

https://martinfowler.com/articles/exploring-gen-ai/sdd-3-tools.html

The Return of It Works on My Machine

Before Docker, deployments often broke due to differences in development environments. It works on my machine hindered team development due to lack of reproducibility.

A similar problem is appearing in AI tool instruction design. On GitHub, dotfiles repositories like My Claude Code Settings or My AI Agent Setup are surging, creating a culture of sharing personally optimized CLAUDE.md, skills, and hooks. While healthy for individual productivity, It works with my CLAUDE.md could eventually become an obstacle in team development.

The AI Version of the DevOps Problem

DevOps was born out of the disconnect between those who build and those who run. In the future, the same wall may arise in the structure of humans operating code in production that was written by AI.

Lisanne Bainbridge's 1983 paper Ironies of Automation pointed out that as automation becomes more sophisticated, operators lose manual skills and become unable to respond to exceptions. This insight, repeatedly confirmed in aviation and process control, also applies to software development. As AI writes more code, humans understand the details of the codebase less, and humans may become unable to handle failures that the AI cannot handle.

https://en.wikipedia.org/wiki/Ironies_of_Automation

The Orchestration Problem of Multi-Agents

With the decentralization of microservices, how to maintain consistency between services became a major challenge. The same could happen with Agent Teams. Running 10 agents in parallel might decrease efficiency due to the overhead of sharing context. This is the Agent version of Brooks's Law—adding manpower to a late software project makes it later—proposed by Fred Brooks in The Mythical Man-Month in 1975.

https://en.wikipedia.org/wiki/Brooks%27s_law

Conclusion

The design philosophy of AI coding tools is a reinvention of practices built over 20 years of human team development. Some parts of history repeat, while some common sense flips.

The law for categorization is simple: When generation cost decreases, the center of gravity for value shifts from the product to the intent.

Refining code—DRY, refactoring, naming, coverage—is changing from things you should always invest in to things you judge based on the situation. Refining intent—verbalizing specifications, recording design decisions, review ability, organizing context—remains as important as ever, and its weight as a bottleneck has increased.

And if you know the patterns of history, you can see through new tools and concepts, recognizing them as that thing from 10 years ago. If you can see through them, you can apply the lessons left by your predecessors as they are.

Problems that look new may boil down to known patterns when viewed structurally. Thinking this way, there might not be that many truly unknown challenges after all.

Why Kiro Looks Unassuming: Organizing Design Philosophy Differences in the Age of Claude Code and Cursor

Kento IKEDA — Mon, 23 Feb 2026 23:44:15 +0000

As AI-assisted development becomes the norm, AI coding tools are rapidly shifting from simple "code completion" to "autonomous agent development." This evolution is fast becoming the standard.

By early 2026, major tools have all evolved toward a direction where "agents autonomously write, test, and fix code"—exemplified by Claude Code’s Agent Teams, Cursor’s parallel Subagents, GitHub Copilot’s Coding Agent, and Windsurf’s parallel Cascade.

In this landscape, using Kiro might honestly feel a bit underwhelming.

However, if we discuss this "unassuming" nature without breaking down which layer of development it addresses, Kiro risks being dismissed as merely a "weak IDE."

This article is intended for developers who use Claude Code or Cursor daily. For those already reaping the benefits of agents, I want to organize the differences in the problems Kiro is trying to solve.

Premise: What Makes 2026 AI Coding Tools Strong?

First, let’s align our understanding of the starting point.

The assessment that "Claude is superior in terms of overall raw capability" is likely based on practical observation rather than emotion. However, by early 2026, it’s fair to say that we are no longer in a situation where "only Claude is strong." The current reality is that major tools are evolving rapidly in distinct directions.

Claude Code: Reasoning Power x Autonomous Loops x Multi-Agents

Claude Code’s strength can be described in three layers.

First is its massive context processing capability. With a context window of up to 1 million tokens (beta), it can fit design docs, implementation code, tests, logs, and diff histories into a single reasoning space. This moves the needle from "local optimization" toward "semi-global reasoning," making cross-dependency analysis and side-effect detection possible within realistic timeframes.

Second is the autonomous execution loop. Partially automating the loop of Implementation → Testing → Error Reading → Root Cause Hypothesis → Fixing changes the very structure of productivity by expanding human cognitive bandwidth.

Third is Agent Teams, released alongside Opus 4.6 in February 2026. A lead agent decomposes tasks, and multiple workers execute them in parallel within independent contexts. Sequential work that used to take hours now completes in minutes through parallelization. This is particularly effective for large-scale refactoring or feature implementations spanning multiple modules.

Furthermore, with structured instructions via CLAUDE.md or AGENTS.md, tool integration through MCP, and context continuity between sessions via an automated memory system, it has evolved from a mere reasoning engine into an agent development platform.

Cursor: Parallel Subagents and Proprietary Models

Cursor changed significantly with version 2.0. By integrating the proprietary coding model "Composer," it achieved four times the generation speed of previous versions. Even more noteworthy are the asynchronous Subagents. Multiple sub-agents run in parallel without blocking the parent agent, forming a tree structure where sub-agents can spawn further sub-agents. Test results in review articles have reported that migrating a router in an 8,000-line Next.js app was reduced from 17 minutes (sequential) to 9 minutes (parallel).

https://myaiverdict.com/cursor-ai-review/

With Background Agents, an agent can now work on an independent branch and open a Pull Request while the developer is busy with other tasks. It also features process control mechanisms like Plan Mode, Rules, and Hooks.

GitHub Copilot: Issue-Driven Autonomous Agents

GitHub Copilot’s Coding Agent creates a Draft Pull Request autonomously in the background simply by assigning an Issue to @copilot. Operating within the GitHub Actions environment, it automates everything from branch creation to commits and opening PRs. It can share project-specific instructions via AGENTS.md and link tools via MCP. It is designed to iterate on automatic fixes based on feedback from PR reviews.

Windsurf: Parallel Cascade and SWE-1.5

Windsurf released parallel multi-agent sessions in Wave 13. Using Git worktrees, multiple Cascade agents can operate simultaneously within the same repository without branch conflicts. SWE-1.5 demonstrated performance rivaling frontier models on SWE-Bench-Pro and was offered for free to all users for three months following its December 2025 release. Workflow automation via Cascade Hooks has also been added.

Common Trends

The trend here is clear: all major tools are competing on "autonomy and parallelism of reasoning." Processing more files, faster, with less human intervention. This is the dominant competitive axis for AI coding tools in early 2026.

As long as we evaluate based on this axis, Kiro currently occupies a late-comer position.

That is exactly why we need to precisely understand what Kiro is actually optimizing for.

What is Kiro Optimizing For?

While major tools optimize for "autonomy and parallelism of reasoning," Kiro optimizes for "state management of the development process."

This may sound abstract, so let’s look at Kiro’s specific mechanisms.

Spec: A Mechanism to Fix Specifications as Structure

The core feature of Kiro is Spec (Specification-Driven Development). From natural language prompts, it generates three Markdown files in stages:

requirements.md: Describes user stories and acceptance criteria using the EARS (Easy Approach to Requirements Syntax) notation. EARS was originally developed by Rolls-Royce for airworthiness regulation analysis of jet engines. it eliminates ambiguity through a structured format: WHEN [condition] THE SYSTEM SHALL [behavior].
design.md: Documents technical architecture, sequence diagrams, and implementation considerations.
tasks.md: Breaks down the implementation plan into discrete tasks, explicitly tracing each task back to specific requirements in requirements.md.

Crucially, these three files are not independent documents; they are a linked structure. Changing requirements.md updates the design via "Refine" in design.md, which then maps new tasks to requirements through "Update tasks" in tasks.md. It is designed so that the impact of a spec change propagates structurally.

Steering: A Mechanism to Turn Implicit Knowledge into Persistent Context

In many projects, critical information exists only as implicit knowledge:
This constraint stems from audit requirements; this cache design assumes future scalability; this async process requires order guarantees; this API depends on an external contract...
Much of this is scattered across code, reviews, or verbal explanations, rarely maintained as a formal structure.

Kiro’s "Steering" fixes these as Markdown files under .kiro/steering/. By default, it generates three files: product.md (product goals and business context), tech.md (tech stack and constraints), and structure.md (project structure and naming conventions).

Steering files have three inclusion modes:

always: Included in all interactions.
fileMatch: Included only when matching specific file patterns.
manual: Included only when explicitly referenced.

This allows for controlling context window consumption while ensuring the AI receives the necessary premises. It is also designed for team deployment, allowing priority control between global and workspace Steering, and distribution via MDM like Jamf.

Hooks: Systematizing Process Automation

Agent Hooks are triggers that automatically execute predefined agent actions in response to events like file creation, saving, or deletion. You can systematize routine tasks in an event-driven manner—such as automatically generating tests upon saving a file, syncing documentation when code changes, or running security checks.

What This All Means

Looking at Spec, Steering, and Hooks together, it’s clear that Kiro isn’t just about generating spec docs. It’s about clarifying requirements (requirements.md), fixing design constraints (design.md + Steering), task decomposition and traceability (tasks.md), impact propagation during changes (Refine/Update tasks flow), persisting implicit knowledge (Steering), and automating routine processes (Hooks).

In short, Kiro’s approach is not about making reasoning faster or stronger, but about clarifying the premises upon which reasoning depends and managing the development process as a state.

Reasoning Engines and State Management Belong to Different Layers

This is the fundamental difference.

LLMs are probabilistic reasoning models. They generate the most plausible output based on the given context. Whether you parallelize with Agent Teams or build tree structures with Subagents, each agent is performing "reasoning within a given context."

"State management" here refers to a mechanism that explicitly maintains invariants, manages dependencies as a structure, and can recalculate the scope of impact when changes occur.

While LLMs can refer to context, they only reason "within the provided scope." Any premise outside that context is treated as if it doesn’t exist.

For example, consider a case where you change a rate-limiting specification. If you give the change to Claude Code, Agent Teams can simultaneously fix code, update tests, and modify documentation. However, if the consistency with future multi-tenant plans or audit requirements isn't in the context, they won't be part of the reasoning. Parallelization doesn't solve this.

The issue here isn't the model's intelligence or speed, but "where the premises are held."

Kiro fixes constraints in EARS format in requirements.md, documents design decisions in design.md, and persists project-specific premises in Steering. When specs change, the impact propagates structurally from requirements to design to tasks.

While the major tools of 2026 compete on "how fast, autonomously, and in parallel reasoning can run," Kiro competes on "how structurally the context required for reasoning can be maintained." They are competing on different layers.

Isn't "Existing Tools + SDD" Enough?

This is the strongest counterargument, and it has gained even more weight from late 2025 into 2026.

SDD is No Longer Unique to Kiro

Specification-Driven Development (SDD) is no longer an approach exclusive to Kiro.

GitHub released "Spec Kit" as open source. It’s a CLI tool that allows the same requirements → design → tasks flow as Kiro to be used across multiple agents like Claude Code, Cursor, GitHub Copilot, Gemini CLI, and Windsurf. It also supports requirement definition via EARS.

Furthermore, each tool has begun to incorporate its own process control mechanisms. Claude Code has instruction structuring via CLAUDE.md and AGENTS.md. Cursor has Rules and Plan Mode. GitHub Copilot has AGENTS.md and custom agent definitions. Windsurf has Rules and Workflows.

The argument that "if you want SDD, just use GitHub's Spec Kit with your tool of choice" or "process control can be done to some extent in any tool" is now quite realistic.

What Kiro Still Provides

So, what is Kiro’s differentiator?

While GitHub's Spec Kit provides the "SDD workflow" in an agent-agnostic way, the synchronization between specifications and implementation remains a human responsibility. If you change a requirements.md generated by Spec Kit, the propagation of that impact to design.md or tasks.md is not automated.

While Rules or AGENTS.md in other tools structure instructions for agents, they aren't mechanisms for managing the three-layer structure of specs, design, and tasks in a linked fashion within a single IDE.

Kiro’s edge lies in the "seamless integration of Spec, Steering, and Hooks within the IDE." The flow of requirements.md change → design.md Refine → tasks.md Update → Task execution → Auto-verification via Hooks is completed within a single environment. Spec files are Git-manageable and can be reviewed and shared by the team.

This is the difference between the "concept of SDD" and the "operation of SDD." While the concept can be realized with Spec Kit, the cost of maintaining it as a daily operation depends heavily on the degree of tool integration.

Addressing the "Expert Engineer + Claude Code" Ultimate Combo

Let's go further: "Isn't an expert engineer writing specs themselves and having Claude Code's Agent Teams implement them the ultimate setup?"

That is indeed a powerful configuration, but the question is where the responsibility for state management lies.

In this setup, judgments about "which invariants exist," "which design constraints are vital," and "which changes are breaking" remain as implicit knowledge on the human side. This rarely causes issues in short-term projects. However, as spec changes accumulate, members rotate, or maintenance occurs six months later, the degradation of implicit knowledge is inevitable.

Kiro provides concrete mechanisms against this degradation. EARS format in requirements.md fixes acceptance criteria without ambiguity, each task in tasks.md traces back to a requirement number, and Steering persists project premises in a Git-manageable form. These are versioned, reviewable, and accessible even if the person in charge changes.

Kiro’s Current Limitations

To be fair, Kiro has clear limitations.

The synchronization between Spec and implementation code isn't fully automated. As noted in reviews on Martin Fowler’s site, drift between specs and implementation remains a challenge. There are also reports of Spec granularity not fitting project scale—the "sledgehammer for a small nail" problem where four user stories and sixteen acceptance criteria are generated for a tiny bug fix.

https://martinfowler.com/articles/exploring-gen-ai/sdd-3-tools.html

Kiro’s default (Auto mode) uses routing based on the Sonnet 4 series, but Pro users and above can select Opus 4.6. It also features parallel execution via custom Subagents and an Autonomous Agent (preview). The gap in model performance and agent features is closing fast; Kiro’s differentiation will likely remain in its integration of process state management via Spec, Steering, and Hooks, rather than on the axis of "reasoning power and parallelism."

Kiro's pricing ranges from Free (50 credits/month) to Power ($200/month), using a credit-based usage model.

Why Does Kiro Still Look Unassuming?

The evaluation criteria for AI coding tools in 2026 are clear.

How many minutes did it take to implement with Agent Teams? How many files were modified in parallel with Subagents? How many hours after assigning an Issue to a Coding Agent was a PR opened? How many bugs were crushed simultaneously with parallel Cascade?

On this axis, speed and autonomy are what look attractive. The impact of "Agent Teams modified 5 modules simultaneously" is overwhelmingly larger as an experience than "Requirements were structured in EARS notation."

Conversely, Kiro’s value lies in preventing accidents: suppressing spec deviation, preventing missed dependencies, and making design constraints explicit. Generally, "the absence of accidents" is difficult to appreciate.

As a result, Kiro isn't weak; it just stands in a "position that is hard to evaluate."

Conclusion: So, What Should You Do?

Let’s translate this into a practical decision-making framework.

First, we must acknowledge that in early 2026, AI coding tools are evolving rapidly in reasoning autonomy and parallelism. In terms of raw implementation speed on this axis, Kiro is not in the same league.

So when do you need a Kiro-like approach? The criterion is "what is the primary bottleneck in this project?"

When to Maximize Agent Reasoning Power

The more the following conditions apply, the more rational it is to stick with Claude Code, Cursor, or GitHub Copilot:

Limited scope
Relatively stable specifications
PoC or exploratory phases
Small team with shared implicit knowledge

In these cases, the bottleneck is "implementation speed." You gain more by maximizing agent reasoning and generation than by fixing the process.

The Threshold for Process State Management

On the other hand, things change when you see these signs:

Frequent specification changes
Non-functional requirements introduced late in the game
Handovers or long-term maintenance are expected
Inconsistencies between tests and specs begin to increase
The labor to verify agent output starts to exceed the labor of implementation

At this point, the bottleneck shifts to "consistency maintenance." Here, where you fix the state matters more than the agent's reasoning capability.

Having requirements fixed in EARS in requirements.md, tasks traced back to requirements in tasks.md, and project premises persisted in Steering becomes a safety net for your future self or your successor six months down the line.

Whether you adopt Kiro as an IDE or incorporate Kiro’s philosophy into your toolchain via GitHub's Spec Kit is a separate decision. The important thing is to "recognize the existence of the layer called process state management."

Final Summary

Claude Code is powerful. Cursor is fast. GitHub Copilot is deeply integrated with GitHub. Windsurf excels in parallelism. These are undeniable strengths. But that strength lies in "autonomy, speed, and parallelism of reasoning and implementation."

Kiro is not flashy, but its value lies in the "structural fixation of process state."

If you compare them simply without understanding this difference, Kiro will always lose. But if you change the evaluation axis, the view changes.

In 2026, all AI coding tools are running toward "making reasoning faster, stronger, and more autonomous." Kiro cannot win on that competitive axis. However, phases dominated by reasoning and phases dominated by consistency maintenance costs are different things. The moment the latter becomes dominant, Kiro’s fixation of process state through Spec, Steering, and Hooks begins to show its worth.

It’s not about superiority; it’s about a difference in layers. Recognizing that difference and choosing what your project needs might be the most realistic solution for 2026.

Beyond Assistance: The Executive Power of "Agent Plugins for AWS"

Kento IKEDA — Wed, 18 Feb 2026 04:41:45 +0000

Released by AWS Labs on GitHub in February 2026, Agent Plugins for AWS is a "plugin library that grants executable skill sets to AI agents," distinguishing itself from simple code completion or natural language assistance.

In addition to being officially supported as a Claude Code plugin, this OSS can be installed and used via the official marketplace in Cursor. It supports the entire workflow end-to-end—from AI-driven design assistance, recommendations, and cost estimation to IaC generation and deployment to AWS.

awslabs / agent-plugins

Agent Plugins for AWS equip AI coding agents with the skills to help you architect, deploy, and operate on AWS.

Agent Plugins for AWS

Important

Generative AI can make mistakes. You should consider reviewing all output and costs generated by your chosen AI model and agentic coding assistant. See AWS Responsible AI Policy.

Agent Plugins for AWS equip AI coding agents with the skills to help you architect, deploy, and operate on AWS. Agent plugins are currently supported by Claude Code and Cursor.

AI coding agents are increasingly used in software development, helping developers write, review, and deploy code more efficiently. Agent skills and the broader agent plugin packaging model are emerging as best practices for steering coding agents toward reliable outcomes without bloating model context. Instead of repeatedly pasting long AWS guidance into prompts, developers can now encode that guidance as reusable, versioned capabilities that agents invoke when relevant. This improves determinism, reduces context overhead, and makes agent behavior easier to standardize across teams. Agent plugins act as…

View on GitHub

In that sense, existing CLI-based agents are capable of similar actions if we only consider the aspect of "execution." However, Agent Plugins differs in that it goes beyond mere task automation; it is provided as a capability layer that systematizes the design process.

This article explores "what it achieves," "why it is important," and "what value it creates."

Why It’s More Than Just a "Completion Tool"

Previously, development support using LLMs typically followed these patterns:

Providing natural language advice in response to questions like, "How should I implement this architecture on AWS?"
Humans designing and coding based on that advice.
Deployment and testing performed manually by humans.

In this flow, the AI's role is centered on "design assistance," while the actual application to the environment depends on humans.

In contrast, "Agent Plugins for AWS" is a suite of plugins that allows AI agents to proactively handle everything from Design → Recommendations → Cost Estimation → IaC Generation → Deployment. This is qualitatively different from automation tools that simply suggest command completions or trigger a CLI via natural language.

Differences from CLI Automation: Why It's Not Just "Terminal Operations"

Many might think that with current tools like Claude Code or various AI CLIs, one can already:

Generate IaC
Execute the AWS CLI
Proceed to deployment without manual intervention ...provided that agent mode is enabled with appropriate permissions. Many are likely already practicing this.

If we look solely at "whether it can be executed," traditional CLI-based agent environments can perform similar tasks.

So, what is the differentiator for "Agent Plugins for AWS"?
The difference lies not in "executability," but in "at which layer the capability is integrated."

1. Ad-hoc Inference vs. Structured Capability

In CLI-based automation, the agent reasons based on a prompt each time to generate and execute necessary commands. Design decisions and service selections tend to rely on the model's internal knowledge and the immediate context.

In contrast, Agent Plugins for AWS is characterized by explicitly defining the AWS design workflow itself—Analyze, Recommend, Estimate, Generate, Deploy—as extended capabilities of the agent. This is not just a sequence of commands to execute a task; it is a capability that stages the entire design process.

In other words, it represents a shift from a model that "executes operations thought up on the spot" to one that "internalizes the AWS design process as a structured capability."

While mechanisms like Claude's Skills or Kiro Powers can indeed grant agents additional specialized knowledge or scripts, they act as modules to enhance behavior or knowledge for specific domains. They do not systematize the entire design workflow.

"Agent Plugins for AWS" takes a fundamentally different role by officially packaging the sequence from design to execution and providing it as a capability integrated with live data (pricing, documentation, etc.) via MCP.

2. Inference-Centric vs. Live Data-Connected

While recommendations and generation are possible via CLI execution alone, those judgments tend to rely on the model's internal training data.

Agent Plugins connects via MCP servers to:

awsknowledge (Official documentation)
awspricing (Real-time pricing)
aws-iac-mcp (IaC best practices)

This structure ensures that design recommendations and cost estimates are tied to the latest official information and real-world data. The difference is not "whether it can be done," but "whether the information sources backing the judgment are systematically integrated."

3. Operational Automation vs. Domain Capability Expansion

CLI automation is primarily about making "operations" more efficient.
Agent Plugins grants the agent specific AWS domain knowledge, such as:

Service selection logic
Cost evaluation flows
IaC output patterns

This can be viewed as an attempt to "expand the design capability within the AWS domain" rather than just automating command execution.

Positioning in the Capability Stack

Structurally, Agent Plugins is positioned in the following layer:

While CLI automation optimizes the "CLI / API Execution Layer," Agent Plugins adds an "AWS Domain Capability" intermediate layer above it. This design elevates the agent's capability stack rather than just adding another tool.

Organizational Perspective

CLI automation improves individual efficiency. Agent Plugins standardizes the design workflow. This difference may seem small but becomes significant at the organizational level:

Reproducibility of designs
Consistency in cost evaluation
Uniformity of IaC output
Reviewable rationales for recommendations

Agent Plugins contributes to the "standardization of processes." Therefore, its differentiator is not "replacing the CLI," but "stacking AWS-specific capabilities on top of the CLI." If CLI automation is the "execution foundation," Agent Plugins is the "capability extension layer" built upon it.

Basic Structure and Workflow of Agent Plugins

Agent Plugins for AWS is a collection of plugin modules that grant AWS-related functionalities to AI agents. According to the README, the goal is to provide skills that allow AI coding agents to assist with everything from AWS design and deployment to operations.

The 5-Step Workflow

Analyze: Analyzes source code and project structure to identify frameworks, dependencies, and data stores.
Recommend: Suggests appropriate AWS service configurations and provides the reasoning.
Estimate: References real-time pricing via the AWS Pricing MCP server to estimate the cost of the recommended setup.
Generate: Converts the design into IaC (CDK or CloudFormation).
Deploy: Reflects and executes the generated IaC in the AWS environment after user approval.

This establishes a circuit for the AI agent to proactively drive the workflow from design to implementation and deployment.

Real-Data Integration Powered by MCP Servers

The Model Context Protocol (MCP) server is the crucial underlying mechanism supporting the utility of Agent Plugins. MCP is a standardized protocol for connecting AI models to external data sources and tools. AWS-side MCP servers provide official documentation, pricing, and best practices.

Welcome to Open Source MCP Servers for AWS

Get started with open source MCP Servers for AWS and learn core features.

awslabs.github.io

Key MCP Servers

MCP Server	Role
awsknowledge	AWS documentation, architecture guides, best practices.
awspricing	Real-time AWS pricing information.
aws-iac-mcp	IaC (CDK/CloudFormation) best practices.

This allows the agent to refer to the latest live data rather than relying solely on the model's internal knowledge.

Real-World Value

1) Cloud Migration & Architecture Design Support

In traditional cloud migration, humans had to handle multiple phases: analyzing current setups, selecting services, decision-making based on costs, designing IaC, and deploying.
By simply instructing an agent using Agent Plugins in natural language, much of this is automated.

"I want to optimize this project for an AWS serverless architecture and deploy it."
This single instruction can lead to recommendations, cost comparisons, IaC, and execution, significantly reducing manual effort and ensuring design accuracy.

2) Formalizing Team Knowledge

The tacit knowledge of veteran designers often leads to siloing. Because Agent Plugins outputs the rationale for recommendations, costs, and IaC, knowledge sharing and reviews become much easier. This results in:

Transparency in the design decision process.
Formalization of best practices.
Reduced learning costs for new members.

3) Integration with CI/CD and Quality Evaluation

Generated IaC and configurations can be integrated directly into CI/CD pipelines.

Automatically validating IaC in pull requests.
Attaching cost comparison reports to the review stage.
Linking to automated deployment approval workflows.

Considerations and Risks

Model Errors and Recency: As stated in the official README, outputs may contain errors, and all results require human review.
Security and Permissions: Careful design of AWS CLI and IAM settings is essential. Risks increase with excessive permissions; establishing proper approval flows for automated deployments is vital.

Future Outlook

Agent Plugins for AWS is a foundation for evolving AI agents from "explanatory assistants" into "orchestration engines for execution." The underlying MCP servers and ecosystem (Claude, Cursor, etc.) are continuously developing, potentially leading to further automation of cloud operations.

Furthermore, AWS has announced the preview of the AWS MCP Server, a remote/fully-managed Model Context Protocol server. This suggests a direction where governance—such as authentication/authorization via IAM and log collection via CloudTrail—will be natively supported.

https://aws.amazon.com/about-aws/whats-new/2025/11/aws-mcp-server/

Conclusion

Agent Plugins for AWS is a significant evolution that moves the role of AI from "assistance" to "execution." By providing a foundation based on real-time data, consistent workflows, and reasoned support, it enables both productivity and quality in cloud design, migration, and operations.

Solving Parenting Pain Points with Generative AI — A Potty-Training Support Device Built with Kiro and M5Stack

Kento IKEDA — Sat, 30 Aug 2025 08:47:02 +0000

Introduction

Potty training can be quite a trial for parents. Our family was no exception—we were stuck. At daycare our child could go frequently, but at home the toilet was boycotted. Sometimes the motivation appeared, but it just wouldn’t stick as a habit. That’s when I thought, “If this is a motivation problem, maybe a little ‘mechanism’ could change the situation,” which became the spark for this project.

There are plenty of commercial potty-training products, but kids get used to them quickly and the motivation fades. So I decided to make an original device that shows a favorite animal character and praises the child. If the novelty wears off, I can add more animal variations—or switch to vehicles or something else—so the device can keep their interest almost indefinitely.

It also happened to be a period when I was evaluating Kiro (an AI IDE), and I wanted to test its capabilities. I usually develop web applications and rarely do embedded development with hardware, but I hoped that with AI’s help I could pull it off.

In the end, I could rely on Kiro far more than expected and let AI handle almost every step. From tech selection and implementation to debugging—and even generating images and audio—what would normally take a few days was finished in about an hour on a Sunday morning. That was astonishing.

In this post, I’ll review the development process, share what it was like to build with Kiro, highlight where human judgment matters, and reflect on what AI-driven development could look like going forward.

What I Built

It’s a “Potty-Training Cheer Device” using an M5Stack Grey. It’s a palm-sized gadget packed with features kids love.

The usage is simple: when the child succeeds at the toilet, they press a button. Then an animal character (rabbit, penguin, or cat) appears at random on the screen and plays a sound.

The UI is in Japanese, with easy-to-read hiragana like “といれできた？” (“Did you use the toilet?”) and “すごい！” (“Great!”). Even if they haven’t learned hiragana yet, hearts and star symbols are displayed so they can still react to those in addition to the animals.

Functionally, the device automatically counts successes and saves the count to EEPROM, so the record persists even if power is turned off. On special counts—first time, fifth time, tenth time, etc.—a celebratory screen with a rainbow background is shown to help maintain motivation.

For practicality, I added a volume control, a battery level check, and a long-press reset. The battery check was especially helpful while I was still getting used to the M5Stack; when behavior seemed odd during development, I could quickly assess the state by pressing the left and middle buttons simultaneously.

I kept the operation as simple as possible, used clear Japanese for the display, and chose catchy electronic sounds—prioritizing usability from a child’s point of view.

Building with Kiro

A Kiro-Led Development Flow

This time I took the reverse approach from typical development. Normally, a human would prepare some requirements and development policy—say, in a file like AGENTS.md . But to test Kiro’s abilities, I intentionally prepared nothing. I started only by telling it that I wanted to make a potty-training device for my child, that I had an M5Stack Grey, and roughly how old the child is.

Expected Approach vs. Actual Approach

Kiro is known as an “AI IDE that’s good at planning,” so I expected a structured spec-writing process like:

Requirements (requirements.md) — user stories and EARS-style acceptance criteria
Design (design.md) — architecture, data flow, and rationale for tech choices
Implementation plan (tasks.md) — stepwise implementation and task breakdown

In practice, Kiro chose a more hands-on approach rather than waterfall-style documentation.

Why the Gap?

I can only speculate about the reasons:

No explicit request to write specs — My vague “I want to try Kiro’s capabilities” may not have triggered a formal spec process.
Prototyping context detected — It likely recognized this as a personal, experimental project and avoided heavyweight enterprise-style process.
Reading my expectations — It might have sensed “I want something working quickly,” prioritizing implementation over documents.
Learned behavior as an AI IDE — From similar projects, it may have learned that implementation-first often yields higher satisfaction for solo developers.

Was That the Right Call?

Looking back, it felt appropriate. If we had started with a detailed requirements doc, we might have wasted time on unknowable questions like “a UI kids will love” or “electronic tones that resemble animal calls.” Those are better answered by building and testing.

That said, we didn’t fully exploit Kiro’s strength in planning. Ideally we could have done structured spec → staged implementation to leverage Kiro’s advantages.

A Pragmatic Documentation Strategy

Instead of large requirements and design docs, Kiro created lean, practical docs incrementally:

setup.md — concrete steps for environment setup
testing.md — how to verify behavior and debug
voice_creation.md — automating voice file generation
image_preparation.md — image generation and format conversion
troubleshooting.md — common issues and fixes

This was spot-on for solo prototyping. In a big company, requirements and design docs matter; for a small one-person prototype, “how to set up,” “how to test,” and “what to do when things break” are far more valuable.

Agile by Nature

Rather than perfect docs first, we organized information as working code emerged. When we needed sounds, we created voice_creation.md . When “Mac doesn’t recognize the device,” we expanded troubleshooting.md .

It felt like Kiro understood my development style and context and chose an optimal approach. Whether that was intentional or simply a bias toward implementation is hard to say.

Transparency of AI Decisions

One thing I noticed: the reasons behind Kiro’s choices aren’t always clear. I can’t tell whether it chose implementation-first due to nuanced situational judgment or simply because code generation is a strong suit. This opacity can be a challenge in AI-driven development, where a human developer would normally explain their rationale.

From First Chat to a Working Device

We started with the vague idea of “a gadget where pressing a button does something.” As Kiro proposed features and I saw them working, more concrete requirements surfaced.

Kiro built something minimal, I tried it, then I gave feedback. For example, the “smart random” logic—preventing the same character from appearing twice in a row—came from me noticing during testing that kids might get bored without change.

The long-press reset was also my suggestion, considering I’d be developing on the same device I’d be using in real life.

“Think While Building”

Traditional development fixes requirements (to some degree) before implementation. Kiro suggested, “Let’s build the basics first and improve as we go.” That proved highly effective. Seeing something real made it easy to spot improvements like “use larger text here” or “a volume control would help.”

Especially in solo projects, the motivation to “just make it run” can outweigh careful up-front planning. This approach created a positive loop: get it running fast, then keep improving.

Code Generation Quality

Kiro generated everything consistently—from PlatformIO settings and C++ for M5Stack to Python scripts for image generation. I didn’t write a single character of code.

Library Selection and Setup

It selected and configured appropriate libraries: M5Stack libraries, ESP8266Audio for audio playback, and SPIFFS for the filesystem. The platformio.ini—from board selection to library dependencies—was completed without my intervention.

Error Handling

We added fallbacks when asset files were missing and handled audio playback failures—folding in feedback from real-device testing.

Memory Management

The implementation accounted for the ESP32’s limited memory: proper freeing of dynamically allocated memory, chunked reads for large files, and other embedded-specific considerations.

Automated Image & Audio Generation

Automatic generation for images and audio was especially helpful.

Finding or crafting assets takes time, so automation was practically a must-have. It also matters for future extensibility when adding more assets.

Japanese Text as Images

At first I planned to render Japanese text like “といれできた？” and “すごい！” directly on the M5Stack. But out of the box, M5Stack can’t display Japanese fonts without extra libraries.

While Kiro initially claimed “this will display Japanese,” it was garbled. I directed it to follow a blog post to support Japanese, but after multiple failed attempts, I pivoted: “Let’s render the text to images instead,” which avoids Japanese font support on-device. Kiro then generated a Python script to produce BMPs that display well on M5Stack. For a personal, kid-facing project, pixel-perfect beauty wasn’t required—the results were readable and “good enough.”

Auto-Generated Sounds

I asked for automatically generated animal-like calls—but this fell short of expectations.

Kiro synthesized plausible effects in Python, but they didn’t quite sound like animal calls. It tried techniques like ADSR envelopes for natural attack/decay and frequency changes tailored to each animal’s “character,” but in the end, the sounds didn’t truly feel like calls. Still, I appreciated the attempt to generate audio mathematically without external libraries.

It also generated spoken lines like “すごいといれできたね” as audio files.

A Natural Development Flow

The loop of “run what Kiro built → test → give feedback” progressed as follows:

Initialize the PlatformIO project — M5Stack-specific configuration
Basic display — simple “Hello World”-level verification
Audio playback — play a simple audio file
Image display — show Japanese text as images
Randomization — character selection logic
Persistence — saving success counts
Special features — celebration screen and battery check

The beauty of this flow: you have something working at each stage. If text appears on the M5Stack screen, wiring is likely correct. If sound plays, the speaker and library settings are likely correct.

Kiro consistently took the “get it working first, then improve it” approach. Rather than chasing perfection, it raised quality step by step—very much like an experienced developer.

That human-like flow kept collaboration smooth. Kiro didn’t try to create a perfect solution in one shot; it suggested iterative steps. When problems occurred, its behavior resembled an experienced engineer: investigate based on the symptoms I shared, fix the issue, and document it—exactly how you’d want someone to handle a bug ticket.

Working incrementally with continuous verification felt like pairing with a human developer, giving me a glimpse of a new mode of AI pair programming.

Where Humans Should Intervene

Not everything should be left to AI. There were several moments where human judgment mattered.

Prioritizing Requirements

Kiro proposed many features, but humans must set priorities. We adjusted the implementation order based on child usability and actual need, even though features like Wi-Fi connectivity and cloud sync sounded attractive.

For potty training, Wi-Fi isn’t essential; reliability and simplicity matter more. Practical choices—handling dead batteries or preventing accidental resets—trumped data sync.

Humans are best at imagining real usage contexts and balancing technical possibilities with real-world constraints. That role still belongs to us.

Hardware-Specific Issues

The M5Stack has unit variance; SPIFFS has size limits; power management has quirks. These are things you only notice on real hardware. A human needs to observe them and report back to Kiro.

For example, when a heart image didn’t display, I first suspected code issues. But it turned out SPIFFS might not have uploaded the file correctly. After I shared serial logs and error messages, Kiro proposed stepwise debugging, and we solved it by adding file-existence checks and fallbacks.

This kind of cooperation—human observation on a real device plus AI-suggested fixes—worked very well.

Final Design Decisions

Color choices, layout, and sound length—these sensory judgments were made by me. Kiro can implement technically correct solutions, but the question “Will a child enjoy this?” is where human sensibility shines.

The initial effect sounded like a warning beep—attention-grabbing but not pleasant. Noticing and adjusting that is still a human strength.

Productivity Gains

Compared to conventional development, several areas improved dramatically—especially the time from initial research to a working prototype.

Faster Research

How to use the M5Stack, set up PlatformIO, and work with ESP8266Audio—Kiro provided the right information instantly, not just API references but project-ready sample code. I could move straight to verification.

It also covered tricky integration details that usually bite beginners—like memory management when using ESP8266Audio with SPIFFS and settings for the M5Stack’s built-in speaker.

Rapid Prototyping

The “get something working” phase was vastly faster. Kiro generated basic functionality in minutes, and I could test on the M5Stack right away.

Previously, I’d have to hunt for sample code, adapt it, resolve build errors, and so on. This time, I started with runnable code and focused immediately on validating features.

Automated Tooling Around the Edges

Image generation, audio generation, format conversion—tasks that normally require separate tools or manual work—were delivered as automation scripts.

For example, the script that converts Japanese text into M5Stack-friendly BMPs handled font choice, sizing, background color, and format conversion. Otherwise I’d have been manually crafting each image in an editor.

Closing Thoughts

This project made me feel how impactful tools like Kiro can be on development. Especially for solo work and small prototypes, AI can dramatically improve efficiency.

Hardware projects used to feel daunting—particularly for those without embedded experience. With AI helping from tech selection to implementation, even web engineers can take on hardware.

At the same time, the importance of human decisions became clearer: defining requirements, prioritizing, judging quality—especially designing from the actual user’s perspective (in this case, a child). With the right division of labor between AI and humans, we can make better things.

What intrigued me most was not just the technical implementation, but how AI also raised the product’s overall completeness—image generation, voice synthesis, and documentation included. It suggests that even solo developers can ship something reasonably polished, quickly.

This started as a small, everyday parenting challenge—potty training—but turned into a great hands-on exploration of the possibilities and challenges of AI-driven development. Most importantly, from the very day we finished, our child started going to the toilet. They’re eager to press the button—“Can I press it?”—and it’s clearly boosting motivation. I truly felt how technology can improve parenting.

As technology advances, the distance from idea to implementation keeps shrinking. Little sparks like this can now take shape in just a few hours over a weekend. That means many more opportunities to unleash individual creativity—and many more diverse, delightful products appearing in the world.

Fastest MVP with Amplify Kiro Amazon Q Developer —Practical playbook for startups and enterprise innovation teams—

Kento IKEDA — Sat, 09 Aug 2025 05:39:33 +0000

Introduction

If your goal is to compress the design → release → improvement loop and ship an MVP as fast as possible, a pragmatic approach in 2025 is to lean on three AWS tools that work well together.

Amplify Gen 2
- A code-first platform where defining requirements in TypeScript (data models / auth / functions) automatically provisions the AWS resources you need.
Kiro
- AWS’s agentic AI IDE (preview) that runs the pipeline end-to-end: spec → design → coding → tests → documentation.
Amazon Q Developer
- Your IDE “pair-programmer” for code understanding, doc/test generation, and design guidance.

This article explains how to combine these three to shorten the path to MVP for both startups and enterprise innovation teams.

Why center the stack on Amplify Gen2

Amplify Gen2 puts the “write requirements in TypeScript = the infra stands up” experience front and center. From your laptop, npx ampx sandbox launches a personal cloud sandbox; changes under amplify/ are applied in real time (via CDK hot-swap). You also get full-stack PR previews per pull request.

Key takeaways:

In the first few days, you can keep the requirements ↔ implementation loop unbroken
- Use Sandbox + PR previews
Future headroom is handled with CDK
- Gen2 is continuous with CDK, so extensions live in the same repository

Docs:

Startup lens — go fast, frugal, and scalable at once

For a small team, being up and running in ~30 minutes is a big deal. A Japanese health-tech startup, KAKEHASHI Inc., publicly documented how they hosted multiple apps on Amplify Console, offloading CI/CD and hosting to AWS and shortening their development cycle. It predates Gen2, but the hosting × CI/CD value carries over today.

On costs, there’s a tailwind: new accounts get up to $200 in Free Tier credits, which materially lowers the cost of learning and early experiments.

Refs:

Enterprise innovation lens — balance speed × governance

If you need multiple PoCs running in parallel across departments, create owner-scoped sandboxes and institutionalize “build and discard.” With ampx sandbox, every save is applied immediately, so requirement validation moves fast.

Governance hinges on the fact that Gen2 backends are CDK-based. That aligns cleanly with Control Tower / GuardDuty and lets you add VPC / PrivateLink / legacy integrations in the same stack. The flip side: Amplify’s autogenerated resource names can be noisy in audit logs—define naming conventions and tag standards up front and wire them into CI.

Costs follow the startup case: the Free Tier credits are handy for early exploration at the department level.

Refs:

Bringing generative AI in: Amplify AI Kit × Bedrock

Amplify AI Kit adds AI routes—Conversation and Generation—as TypeScript definitions. In minutes, you get a front-end scaffold and a Bedrock connection. The design is intentionally TypeScript-first, not a one-off magic CLI incantation.

Fastest setup:

Scaffold with npm create amplify@latest
Reflect changes instantly with npx ampx sandbox (hot-swap)
Add AI routes in TypeScript and wire up Bedrock
Iterate using the preview

Refs:

“If we have Kiro, do we still need Amplify?” — the three-in-one division of roles

Let’s recap what each tool owns:

Kiro — draws the blueprint and work plan
- Spec-driven pipeline that runs requirements → design → coding → tests → docs end-to-end -MCP support to connect external knowledge bases and tools (preview)
Amplify — the land and utilities to run it
- Hosting, CI/CD, auth, data, plus Sandbox and PR previews to keep the loop turning
Amazon Q Developer — your teammate on the ground
- Code understanding, doc/test generation, design guidance

In short: Kiro drives design→code aggressively forward, but where you run it safely is a separate question. Amplify gives you a production-grade cloud execution base quickly, and Q Developer keeps daily improvements flowing. Together, they close the design → release → improvement loop.

A minimal example flow:

Scaffold: npm create amplify@latest and write backend requirements in TypeScript
Instant verify: npx ampx sandbox and hot-swap on every save
Review: use full-stack PR previews
Add AI: define AI routes with Amplify AI Kit in TypeScript and connect Bedrock
Thicken in the IDE: firm up specs/tests with Kiro; generate docs/tests with Amazon Q Developer

Conclusion

Startups can accelerate hypothesis testing with Amplify + AI Kit, and Free Tier credits make new bets cheaper.

Enterprises can balance speed × governance with owner-scoped sandboxes plus naming/tag standards.

Kiro (design) × Amplify (execution) × Q Developer (improvement) shortens the distance to MVP. They’re complementary, not substitutes.

AWS Amplify Evolution Timeline — 2017 to 2025

Kento IKEDA — Sat, 07 Jun 2025 07:19:24 +0000

When AWS Amplify debuted in 2017, even “just adding a login page” meant wrestling with CloudFormation templates and IAM policies. Eight years of steady releases have chipped away at those manual chores and debugging marathons. Today you can spin up a prototype on par with well-known SaaS products—often in a single weekend—using Amplify alone.

This post walks through that journey, highlighting what each release simplified and which kinds of products suddenly became feasible. Use it as a retrospective or as a compass for planning your own adoption.

Year-Month	Feature Update	What It Unlocks	Product You Can Now Ship Fast	How We Did It Before
2017-11	Amplify CLI	One-command creation of Cognito, API Gateway, S3, etc.	Todoist-style auth-enabled task tracker MVP	Hand-written SDK calls plus IAM policies
2018-11	Amplify Console (CI/CD & Hosting)	`git push` → build → CloudFront deploy	Developer docs updated daily	Wire S3, CloudFront, CodePipeline by hand
2019-12	DataStore (offline sync)	Managed diff sync between device & cloud	WhatsApp-like chat / field-inspection app	IndexedDB / SQLite and custom sync layer
2021-11	GraphQL Transformer v2	`@primaryKey`, `@index`, etc. for complex models	Shopify-style multi-tenant SaaS	Edit resolvers & hand-manage DynamoDB GSIs
2021-12	Amplify Studio × Figma	Auto-gen React components & mock data	Squarespace-like LP builder	Recreate in Storybook, compare screenshots
2022-11	Next.js SSR / ISR Support	Prod-ready SSR/ISR after `next build`	Airbnb-like search UI	Craft Lambda@Edge & CloudFront configs
2023-06	Monorepo (Nx / Turborepo) Support	Diff builds + caching across workspace	Uber-style micro-frontends	Many Amplify apps & too many webhooks
2024-05	Amplify Gen 2 (GA, code-first)	Type-safe DB/Auth/Func in a TS file	Notion-like permissioned docs	CloudFormation nesting nightmares
2024-11	Amplify AI Kit (Bedrock)	Chat/summary in a few lines of code	Duolingo-style AI explainer mode	DIY API Gateway → Lambda → Bedrock
2025-03	WAF Integration	One-click OWASP rule sets	Payment-ready e-commerce	External CloudFront-WAF stack
2025-03	Deploy Skew Protection	Auto pin clients to one JS version	TikTok-scale SPA updates	S3 versioning + header gymnastics
2025-05	Upgraded Build Instances	Large/XLarge build nodes	Astro / huge monorepos build fast	Fixed 8 GiB / 4 vCPU time-outs

Looking Back, One Release at a Time

Amplify CLI (2017-11)

The CLI collapsed the Cognito, API Gateway, and S3 boilerplate into an interactive wizard. Even Amplify first-timers could build a login-ready web app in half a day. Most important: the CLI autogenerates least-privilege IAM roles, eliminating the old routine of chasing 403s and puzzling over stray 500s in CloudWatch logs.

Announcing AWS Amplify and the AWS Mobile CLI | Front-End Web & Mobile

aws.amazon.com

Amplify Console (2018-11)

Connect your GitHub repo and Amplify wires up build, test, deploy, and global CDN caching. A merge to main shows up worldwide within minutes—perfect for daily-updated docs sites. Before this, every release meant manual S3 uploads, CloudFront invalidations, and the occasional “white screen” due to stale JS.

Amplify Console – Hosting for Fullstack Serverless Web Apps | AWS News Blog

aws.amazon.com

DataStore (2019-12)

DataStore automates diff sync and conflict resolution between local storage and AppSync. You can now build offline-first field apps or chat systems that replay queued changes once a signal returns. Custom implementations once burned days on timestamp collisions and edge-case testing.

Amplify DataStore – Simplify Development of Offline Apps with GraphQL | AWS News Blog

aws.amazon.com

GraphQL Transformer v2 (2021-11)

With directives like @primaryKey and @index, intricate schemas live entirely in GraphQL. Design docs are code, so no more “What does the real table look like?” meetings. In v1 we edited resolvers by hand, added GSIs, and watched CloudFormation templates balloon until code reviews stalled.

AWS Amplify announces the new GraphQL Transformer v2. More feature-rich, flexible, and extensible. | Front-End Web & Mobile

aws.amazon.com

Amplify Studio × Figma (2021-12)

Import a Figma design into Amplify Studio and out comes React components plus mock data. Designers tweak colors or spacing; a pull request appears moments later. The Storybook-screenshot-manual-fix loop that used to choke sprints is gone.

AWS Amplify Studio – Figma to Fullstack React App With Minimal Programming | Front-End Web & Mobile

aws.amazon.com

Next.js SSR / ISR (2022-11)

Amplify Hosting officially supports Next.js 12/13, so SSR, ISR, middleware, and image optimization all “just work” after next build. Dynamic-inventory e-commerce and travel sites can now manage cache lifetimes in seconds without caring about Edge Lambda order or us-east-1 quirks.

Deploy a Next.js 13 app to AWS with Amplify Hosting | Front-End Web & Mobile

aws.amazon.com

Monorepo Support (2023-06)

Amplify detects npm/Yarn/pnpm workspaces, Nx, or Turborepo setups, then runs diff builds with caching. One repo holds shared code, types, and libraries; multiple SPAs flow through the same pipeline. We used to duplicate Amplify projects and hit GitHub’s webhook limit—maintenance hell.

docs.aws.amazon.com

Amplify Gen 2 (2024-05)

Define data models, auth rules, and functions in TypeScript and deploy—type safety end-to-end. Business logic and schema live together, making reviews and refactors painless. Field-level permissions à la Notion are doable without touching CloudFormation.

Fullstack TypeScript: Reintroducing AWS Amplify | Front-End Web & Mobile

aws.amazon.com

Amplify AI Kit (2024-11)

Bundled chat UI components and typed clients let you call Amazon Bedrock LLMs in minutes. Combine history, summaries, and feedback to prototype a Duolingo-style learning app in a few days. The old stack—API Gateway→Lambda→Bedrock plus CORS and token juggling—is history.

Build fullstack AI apps in minutes with the new Amplify AI Kit | Front-End Web & Mobile

aws.amazon.com

WAF Integration (2025-03)

Choose a Web ACL in the console and SQLi/XSS protections snap on in minutes. PCI-DSS-grade e-commerce now needs no extra CloudFront layer. Previously we juggled external WAF stacks and risked downtime at every CNAME switch.

Firewall support for AWS Amplify hosted sites | AWS News Blog

aws.amazon.com

Deploy Skew Protection (2025-03)

During rolling deploys, browsers are pinned to one JS bundle, preventing “white screens” even under high-frequency releases. We used to cobble together S3 versioning and custom headers, and 404s still leaked through.

Amplify Hosting Announces Skew Protection Support | Front-End Web & Mobile

aws.amazon.com

Upgraded Build Instances (2025-05)

Large (16 GiB/8 vCPU) and XLarge (32 GiB/16 vCPU) build nodes slash wait times for Astro sites and giant monorepos. Pricing scales by build-minutes × instance factor, so you can upsize only for monthly big releases and stay cost-efficient. Goodbye to endless timeout retries.

Introducing Upgraded Build Instances on Amplify Hosting | Front-End Web & Mobile

aws.amazon.com

Closing Thoughts

Year after year, Amplify has folded one more “time-sucking chore” into a managed feature. By 2025 it’s a weekend-friendly platform for spinning up prototypes that feel like production-grade SaaS. Hope this timeline helps you map the landscape and pick the right entry point for your own projects.

A Roadmap to AWS Amplify Gen2: Shifting Your Mindset from Gen1 to a Modern, AWS-Integrated Frontend CI/CD Platform

Kento IKEDA — Wed, 08 Jan 2025 06:01:57 +0000

This article is an English translation of "AWS Amplify Gen2への道標：Gen1からの“考え方シフト”でAWS連携型の現代的フロントエンドCI/CD基盤へ" which was originally published in Japanese.

Introduction

Hello everyone, this is Ikeda (@ikenyal)—an AWS Community Builder, AWS User Group Leader, and core member of the Amplify Japan User Group. I serve as an EM at GENDA Inc. and as Director/CTO at ZINE Inc.

Here, I’ve compiled information that may help those who’ve been using Amplify Gen1 for a long time and are now considering migrating to Gen2.

There may be plenty of people who have been using Amplify by just “clicking a few buttons to deploy” and called it good. But now that we’re moving into a new generation—“Gen2”—let’s explore Gen2-specific concepts and setup methods that differ from the past.

Amplify, which was once mainly a simple hosting and deployment tool, has evolved into a more modern CI/CD platform in Gen2. Its integration with AWS resources such as CloudFront, S3, and IAM is clearer, enhancing extensibility, speed, and security.

In this article, I’ll introduce key points you should know when moving from Gen1 to Gen2, along with links to official documentation and other reference materials. Lastly, I’ll touch on a much-anticipated new update: WAF support, which was just implemented a few days ago.

If you’ve been mastering Gen1 but haven’t yet tried Gen2, I hope this provides that “Aha, so that’s how it works!” moment.

“What’s different about Gen2?” Getting the Big Picture

Gen1:

For many users in the Gen1 era, the main approach was: “Deploy following a set pattern prepared by Amplify.” It was easy and straightforward, but as operations grew more complex, you may have felt the limitations of scalability and flexibility. Some users managed advanced setups by extending Amplify as far as possible, but that took a certain level of expertise.

Gen2:

With Gen2, Amplify has evolved into “a highly tunable service that’s part of the broader AWS infrastructure.” Integration with AWS resources like CloudFront, S3, and IAM is more transparent, making it easier to handle monorepo structures and more complex projects. The build and deployment processes have been updated, and improved caching strategies and environment variable management have greatly boosted performance, scalability, and operability.

In short, think of Amplify not just as a “simple hosting tool,” but rather as a “CI/CD platform that integrates seamlessly with AWS, enabling fast and flexible frontend operations.”

References:

1. Evolution in Performance and Scalability

Gen1:

You might have worried about performance lags as traffic increased, and experienced longer build times. The caching strategy was limited, so in projects with complex dependencies, inefficiencies could be quite noticeable.

Gen2:

Gen2 introduces a completely redesigned build and deployment system, supporting fast, stable operations even for monorepo structures and multi-framework projects. An intelligent caching strategy is introduced to avoid rebuilding unchanged parts, reducing build times. Plus, with CDN optimizations via CloudFront and compatibility with modern toolchains, it can handle significant simultaneous user traffic without noticeable response delays.

Key Mindset Shift:

In Gen1, you might have been “building and deploying everything,” whereas in Gen2, you’ll think “build and deploy only what’s changed for a faster release cycle.”

2. Reconsidering Configuration Files and Build Processes

Gen1:

Many operations relied on amplify.yml for build commands and environment variables, but flexibility was limited. Managing environment variables or controlling caching could be challenging in more sophisticated scenarios.

Gen2:

The build process is more refined, and handling environment variables and cache strategies has improved. Settings from Gen1 might not transfer directly to Gen2, so it’s essential to review and align with the official docs. By doing so, environment variables can be managed more safely and clearly, and appropriate cache policies can help shorten build times and optimize deployment flows.

Key Mindset Shift:

Instead of “we’ll just fit into the environment provided,” think “let’s tune Amplify’s settings to match our own build and deployment requirements.”

3. Refined Permissions and Security

Gen1:

IAM roles and policies were often handled behind the scenes. Aligning with more complex requirements meant you might have to dive into CloudFormation directly, which could be daunting.

Gen2:

It’s now easier to understand and apply IAM and permission settings via the Amplify CLI or console. In addition, when integrating Amplify with backend services like GraphQL API, Lambda, and DynamoDB, Gen2 has further optimizations, making it easier for developers to focus on business logic and UX improvements.

Key Mindset Shift:

Move from “managing permissions within Amplify’s unique environment” to “seamlessly using AWS standard permission management through Amplify.”

4. Think of Amplify as “Part of AWS” — A Mindset Shift

Gen1:

Amplify was somewhat of a “black box.” You could use it without really understanding AWS resources, but you’d quickly run into limitations if you needed significant customizations.

Gen2:

Amplify is most effective when viewed as part of your wider AWS infrastructure. You can now fine-tune cache and security settings, set up redirects or headers for specific paths, and more at the AWS resource level (CloudFront, S3), leading to a better user experience and improved SEO.

Key Mindset Shift:

Reframe your thinking from “Amplify = a simplified gateway to AWS services” to “Amplify + AWS as a comprehensive CI/CD platform.”

5. Migration as an Opportunity: Reassess for Streamlined Operations

Moving from Gen1 to Gen2 might sound like a hassle, but think of it as the perfect chance to review your entire project. This could include:

Improving your CI/CD pipeline
Reconsidering your testing strategy
Optimizing the staging and production deployment flow

By stepping away from the “it works for now” mentality, you can build a more efficient, modern development and operational environment.

You can minimize risk by taking a step-by-step approach based on official documentation. Embrace the excitement of adopting the “next generation of AWS hosting” as you go along.

WAF Support Arrived on December 18, 2024!

Finally, let’s introduce a hotly anticipated topic in the Amplify community. On December 18, 2024, Amplify Hosting gained WAF (Web Application Firewall) support. Previously, you had to manage WAF separately—like applying it through CloudFront on your own or leveraging solutions outside of Amplify. But now, you can directly apply WAF policies to Amplify Hosting itself.

This simplifies defense against common web attacks (SQL injection, XSS, etc.) using Amplify alone.

References:

Conclusion

Gen2 transforms Amplify into a faster, more flexible AWS frontend CI/CD platform.
Multiple improvements have been made in performance, scalability, build processes, environment variable management, and permission handling.
Viewing Amplify as an integral part of AWS—and optimizing resources at the AWS level when needed—will be crucial going forward.
With WAF support added on December 18, 2024, Amplify offers even more robust security out of the box.

Building on your existing Gen1 knowledge, leverage the official guides and documentation to make the leap to Gen2. It’s more than a simple version upgrade—it’s a new frontier in frontend development and operations. This is the perfect opportunity to explore Gen2—give it a try and experience the potential for next-level, modern workflows!

Aiming for sustainable community management - Improvement of Amplify Japan User Group's management system in 2023

Kento IKEDA — Sun, 24 Dec 2023 14:23:31 +0000

This article is an English translation of "持続可能なコミュニティ運営を目指して〜Amplify Japan User Group 2023年の運営体制改善" which was originally published in Japanese.

Hi, I'm @ikenyal, AWS Community Builder, AWS User Group Leaders and Amplify Japan User Group Steering Member. I work as an EM at GENDA Inc. and as a Director, CTO at ZINE INC.

My focus as an AWS Community Builder, in addition to my day job and side business, is the management of the Amplify Japan User Group. In this article, I would like to introduce the efforts I have made to improve the management system of Amplify Japan User Group in 2023. We will introduce some of the challenges that arise and how we are trying to maintain sustainability of the community organization, which is not a business, but a group of volunteers.

If you are wondering "What is AWS Community Builder? please also refer to this article 挑戦権は誰にでもある、日本からもAWS Community Buildersを目指してみよう (in Japanese)!

Community Management Features and Issues

Community management differs from that of a typical company in that it is a volunteer-based organization. The members of the management team have their own day jobs, side jobs, and private lives, but they also participate in the management activities as volunteers.

Therefore, unlike a job, it is not mandatory, and there will be a need to have each individual work in his or her own spare time. This is a feature and a potential problem for the continuity of community management.

In fact, the Amplify Japan User Group also experienced issues due to this problem, and there was a period of stagnation in its activities. This stagnation resulted in a lack of meetups and community improvement activities, and the growth of the community itself stagnated and eventually declined.

To stem the tide, the Amplify Japan User Group has redesigned its management structure this year.

We focus on maintaining the continuity of the management system, as well as the community itself. We believe that if this is maintained, community activities will naturally continue and be activated.

Organizational Problems in the Amplify Japan User Group

In the Amplify Japan User Group, the change in community management from an AWS-driven state to a community-driven state was one of the reasons for the stagnation of the management structure.

Until the community was able to stand on its own, AWS employees took the lead in organizing the meetups. However, as the community gradually becomes established, there is a need to switch to community-led management at some point. Of course, while switching the initiative, AWS employees will continue to provide support.

The timing was right for the Amplify Japan User Group around 2021, and while we continued to host the event, we changed the name of the event to the current "Amplify Boost Up" and tried to steer the event toward community-driven activities.

However, at that time, we did not make any major changes to the operation structure or mechanism, but switched only close to the mindset of "let's keep the AWS employees' lead low key and let the rest of the operation members do their best".

COVID-19 was still having a large impact, and our activities were limited to holding ongoing online events and interacting with each other on Slack. Even in these limited activities, events were being held less frequently, and Slack was becoming more like a place to see bot posts, with almost no postings. The aforementioned mind-switching alone made it difficult to maintain that mind-switching, and the management tasks tended to remain static.

After such a period of time, in 2023, we decided to restart the operation of the event, and since the onslaught of COVID-19 had passed and the number of hybrid online/offline events was increasing, we decided to restart the community event as a hybrid event, and at the same time, we improved the operation of the event to avoid a similar situation. We restarted the event with an improved management system to avoid a similar situation.

Improvement of the management system for sustainable community management

In order to achieve sustainable community management, we have implemented several operational structure improvements, some of which are shown below as examples.

Clarification of management team members

Although it is a natural thing to do, we have again clarified the members of the management team.

Community management is solely a volunteer-based activity. Therefore, it should be allowed that there may be periods of inactivity due to work or personal circumstances. With this in mind, until then, people who were interested in the management would join the Slack channel where the management members gathered, and the management member management was close to just that. As a result, the situation became "There are many members in the Slack channel, but who are the members who are actually involved in the management and can share the tasks? The management members themselves were becoming unclear.

The adverse effects of this condition were the following events

When someone asks for an opinion or reaction on Slack, we don't know if the unrespondent forgot to respond or is inactive and unwilling to respond, and we have to keep reminding them.
This results in an atmosphere of "it seems okay not to respond".
Since only a few more members of this obscure management team will be involved, there will be no sense of unity as a team.

In order to solve these problems, we have started to take inventory and clarification of the management members by periodically confirming their willingness to participate as management members once a year.

Leaving the operation is not a negative thing. Since work and personal situations change, by making it easier to express your intention to continue or stop as a management member each year, you can avoid the awkward situation where you are too busy to show up and participate in the management more easily.

Then, by constantly updating the management member ledger maintained within the operation, we were able to maintain that all members of the operation are Active.

Regularization of management meetings

The previous management organization did not have regular management meetings, but rather set them up as needed, such as prior to an event.

As a result, decision-making was not progressing during periods when meetings were not being held, although of course this could be done to some extent on Slack. This led to a vicious cycle in which events were not planned and the need for meetings declined.

If we are in a state of "no agenda," it could be an indication that there is nothing moving as a community. For this reason, we decided to make the management meeting a regular meeting, to be held monthly. In doing so, we have also established a mailing list of management members by clarifying the above-mentioned management members, and we send calendar invitations to the mailing list.

Management meetings, which tended to be held irregularly in the past, are now scheduled on a regular schedule, creating an opportunity to move forward as a community and a rhythm for such meetings. In addition, the act of checking everyone's schedule and setting up a meeting each time was expensive to coordinate. Therefore, being able to block each person's schedule in advance as a regular schedule has been very effective.

Maintenance of tools

The Amplify Japan User Group does not have a specific budget, so we need to make sure that the tools we use in our operations can be covered by free ones. In parallel with the organization of the regular meetings mentioned above, we also organized the tools. In addition, a ledger has been created by clarifying the members of the management team, and we are now organizing the tools, including their authorization.

Here are two examples of the tools we use.

Document management: GitHub

We need a management tool for documents used within the management team, such as management meeting memos and management member ledgers. Since we do not have a budget, We decided to use GitHub because it is free of charge regardless of the number of users to be invited, and because it can be set to be private.

The Amplify Japan User Group already had a GitHub Organization for use with source code for the community site and CFP for meetups. Therefore, while using GitHub's Free plan, we created a private repository for the management team, and we use GitHub Discussions in that repository to manage documents.

Until then, the base was based on the granting and revoking of privileges to related tools and mailing lists. Therefore, in addition to management of operating members, a ledger was maintained for tools, and the status of authorization grants was visualized.

In the above GitHub Discussions, "アナウンス" (it means "Announcements") places ledgers and other documents that should always be referenced.

Official community communication tool: Discord

One of the most significant tool maintenance changes was the switch from Slack to Discord as the communication tool in the community.

Due to a lack of budget, we could not afford a paid plan for Slack, and as a result, the past logs were hidden because it was a free version. To improve this situation, we switched from Slack to Discord.

Logs of questions and discussions within the community can now be utilized, including those from the past, as they are important community assets in themselves.

In addition to the above, we also use Google Group, YouTube, connpass, Slido, etc.

Improving the sense of unity among management members

The management team is engaged as volunteers, so they need some motivation to keep it going.

Of course, the community management activities themselves can be one of the achievements to be certified as AWS Community Builder or AWS User Group Leaders, and in fact, many AWS Community Builders and AWS User Group Leaders are on the management team.

However, this is the only situation in which each individual must feel motivation on their own. Therefore, it was not possible to create motivation derived from a sense of unity as a team of management members.

Therefore, we produced management T-shirts, although each member had to pay for his or her own T-shirt. Please note that the management members are wearing these T-shirts at the meetups.

Future community management

In community management, there are times when the workload is temporarily high, such as during events, but it is important to create a system and structure that will allow for continuous operation during normal times. This is because everyone involved has other jobs and private lives, and the community is made up of volunteer activities in addition to these.

How to create and maintain a lean and efficient management organization that is built on such good intentions is an important issue for the survival of the community.

The Amplify Japan User Group is now well organized, and I would like to contribute to the development of the community itself by further enhancing the sense of unity and motivation of the management as the finishing touch to the group.