AI has changed the speed at which software can be built.
A developer can describe an API endpoint and receive an implementation in seconds. An AI coding agent can inspect a repository, modify multiple files, generate tests, run commands, debug failures, and prepare a pull request.
This is a significant improvement in development productivity.
It also creates a new problem.
Generating software faster does not mean generating scalable software.
In fact, AI can help you create a poorly designed system faster than ever.
Ask an AI agent to "build a complete SaaS backend" and it may happily generate controllers, models, migrations, services, background jobs, authentication, caching, queues, and dozens of abstractions.
The application may even work.
Then six months later:
- changing one feature breaks three others
- database queries become increasingly expensive
- business logic exists in five different places
- background jobs execute twice
- APIs become inconsistent
- every developer implements things differently
- tests become difficult to maintain
- AI agents make increasingly dangerous changes because the architecture is unclear
The fundamental lesson is simple:
AI increases implementation capacity. Architecture determines whether that capacity produces a scalable system.
Building scalable software with AI therefore requires a different mindset from simply using AI to write code.
You need to design the environment in which AI writes code.
You need architectural boundaries.
You need rules.
You need automated verification.
You need documentation that machines can understand.
And increasingly, you need to treat your repository as an operating environment for both human developers and software engineering agents.
This article explains how to do that.
What Does "Scalable Software" Actually Mean?
Developers often use scalability to mean:
"The system can handle more traffic."
That is only one form of scalability.
A production system has to scale across several dimensions.
Traffic scalability
Can the system handle:
100 users
1,000 users
100,000 users
1,000,000 users
without collapsing?
This involves:
- application servers
- database performance
- caching
- queues
- load balancing
- storage
- connection management
- horizontal scaling
Data scalability
What happens when:
orders = 10,000
becomes:
orders = 500,000,000
Queries that looked harmless during development may become serious bottlenecks.
Team scalability
Can twenty developers work on the codebase without constantly breaking each other's changes?
A system can handle millions of requests and still have terrible engineering scalability.
Feature scalability
Can you add new functionality without modifying half the application?
Good architecture minimizes the number of unrelated components affected by a change.
Operational scalability
Can your team understand what is happening when production fails?
Can you answer:
Why is checkout slow?
Which service is failing?
Which request triggered this exception?
How many jobs are stuck?
Which deployment introduced the regression?
AI scalability
There is now another dimension worth considering.
Can multiple AI coding agents safely contribute to the repository?
This is becoming increasingly relevant.
Coding systems such as Codex and GitHub Copilot support repository-level instructions that give agents persistent information about project structure, conventions, testing, and validation. OpenAI recommends AGENTS.md for persistent repository context, while GitHub Copilot supports repository instructions and AGENTS.md files for similar purposes.
That means modern software architecture must increasingly optimize for two types of developers:
Human developers
+
AI engineering agents
Fortunately, the things that make software easier for AI to understand are usually the same things that make it easier for humans to maintain.
AI Should Be an Implementation Engine, Not Your Architect
One of the most dangerous prompts in AI-assisted development is something like:
Build the complete backend architecture for my SaaS.
The problem is not that AI cannot produce architecture.
It can.
The problem is that architecture is a collection of contextual decisions.
Consider a payment system.
Should you use:
monolith?
modular monolith?
microservices?
event-driven architecture?
serverless?
There is no universally correct answer.
The decision depends on:
- expected traffic
- engineering team size
- deployment model
- transaction requirements
- data consistency requirements
- operational capabilities
- budget
- expected product evolution
- failure tolerance
- integration requirements
AI does not automatically know these constraints.
If you ask it to decide without supplying them, it will fill the missing context with assumptions.
The code may be technically reasonable while being completely wrong for your product.
A better relationship looks like this:
Human:
defines architecture
AI:
explores options
implements components
writes tests
performs refactoring
finds inconsistencies
reviews code
generates documentation
investigates failures
Think of AI as an extremely fast engineering team working inside boundaries that you define.
Start With Architecture Before Writing Prompts
Suppose we are building a project management SaaS.
A naive architecture might look like:
Frontend
|
Backend API
|
Database
This is technically architecture, but it tells an AI agent almost nothing.
Instead, define the important boundaries.
┌──────────────────┐
│ Web App │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ API Layer │
└────────┬─────────┘
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Projects │ │ Billing │ │ Accounts │
└─────┬────┘ └────┬─────┘ └────┬─────┘
│ │ │
└─────────────┼──────────────┘
│
▼
┌──────────────────┐
│ Database │
└──────────────────┘
Background Work
│
▼
┌──────────────────┐
│ Queue │
└──────────────────┘
Now define responsibilities.
API layer
Responsible for:
- authentication
- request parsing
- validation
- authorization
- response formatting
Not responsible for business rules.
Project module
Responsible for:
- projects
- tasks
- assignments
- project permissions
- project workflows
Billing module
Responsible for:
- subscriptions
- invoices
- payments
- billing providers
- usage calculations
Account module
Responsible for:
- organizations
- users
- memberships
- roles
This creates boundaries an AI agent can follow.
Start With a Modular Monolith More Often Than You Think
AI makes microservices unusually tempting.
You can ask:
Create an authentication microservice.
Then:
Create a billing microservice.
Then:
Create an order service.
Soon you have twelve services.
Unfortunately, you also have:
- twelve deployments
- twelve logs
- twelve CI pipelines
- distributed tracing
- network failures
- duplicated authentication logic
- service discovery
- asynchronous consistency
- versioned contracts
- complicated local development
You solved a scaling problem you probably did not have.
For many SaaS applications, a modular monolith is a better starting point.
For example:
src/
Modules/
Accounts/
Projects/
Billing/
Notifications/
Reporting/
Each module contains its own:
Controllers
Services
Models
Repositories
Events
Jobs
Policies
Tests
You receive many benefits of service separation without distributed-system complexity.
Later, if billing becomes computationally expensive or organizationally independent, you can extract it.
Modular Monolith
↓
Identify scaling boundary
↓
Extract Billing Module
↓
Billing Service
AI is extremely useful during this extraction because clear boundaries allow it to identify dependencies and migrate them systematically.
The important decision is establishing those boundaries early.
Create an Architecture Contract for AI
One of the biggest improvements you can make to AI-assisted development is creating repository-level engineering instructions.
OpenAI specifically recommends using AGENTS.md to supply persistent context to Codex. OpenAI's guidance notes that these files can document naming conventions, business logic, dependencies, and repository-specific information that agents cannot reliably infer from source code alone.
GitHub Copilot similarly supports repository-wide instructions, path-specific instructions, and AGENTS.md files.
This changes how we should think about documentation.
Documentation is no longer only for developers.
It can function as executable context for engineering agents.
A useful file could look like this:
# Architecture
This application uses a modular monolith.
Modules:
- Accounts
- Projects
- Billing
- Notifications
- Reporting
Modules must not directly access another module's database models.
Cross-module communication must use application services or domain events.
# Controllers
Controllers are responsible only for:
- request validation
- authorization
- calling application services
- returning responses
Never place business logic inside controllers.
# Database
All list endpoints must support pagination.
Avoid queries inside loops.
Use eager loading when relationships are required.
All new queries on high-volume tables must consider indexes.
# Jobs
External API calls should execute asynchronously unless an immediate response is required.
Jobs must be safe to retry where possible.
# Testing
All business logic requires unit or integration tests.
Every bug fix requires a regression test.
Run:
npm test
before completing any task.
# Security
Never expose database IDs when public UUIDs are available.
Never log passwords, tokens, secrets, or payment credentials.
Now imagine five AI agents working on different parts of the application.
Without this document, each agent may invent its own conventions.
With it, you have dramatically increased architectural consistency.
Give AI Constraints, Not Just Requirements
Compare these two prompts.
Weak prompt
Create an endpoint for creating orders.
AI has to decide:
- validation
- authorization
- architecture
- transaction handling
- event handling
- error handling
- testing
- response structure
Different runs may produce completely different solutions.
Now consider:
Implement POST /api/orders.
Requirements:
- authenticated users only
- use CreateOrderRequest for validation
- controller must contain no business logic
- business logic belongs in OrderService
- wrap inventory reservation and order creation in a database transaction
- dispatch OrderCreated only after successful persistence
- duplicate requests with the same idempotency key must not create multiple orders
- return OrderResource
- add integration tests
- follow the existing CreateInvoice implementation
The second prompt does something important.
It reduces the AI's decision surface.
This is one of the biggest secrets of scalable AI-assisted development.
The more architectural decisions you encode into the system, the fewer architectural decisions AI has to invent.
Think in Invariants
Strong systems are built around invariants.
An invariant is something that must always remain true.
For an e-commerce platform:
stock >= 0
For financial software:
total debits = total credits
For a subscription system:
a customer cannot have two active subscriptions
for the same plan unless explicitly allowed
For multi-tenant SaaS:
a user from Tenant A must never access Tenant B's data
AI should know these rules.
A repository might contain:
## Critical Domain Invariants
1. Inventory must never become negative.
2. Payment callbacks may be delivered multiple times.
3. Tenant data must always be scoped by tenant_id.
4. Subscription state changes must go through SubscriptionService.
5. Paid invoices cannot be deleted.
Now an agent modifying the system has significantly more useful context than one merely reading database schemas.
Design the Database Before Asking AI to Generate Models
AI coding tools make schema generation extremely convenient.
That convenience can hide bad database design.
Imagine asking:
Create tables for an e-commerce system.
You might receive:
users
products
orders
order_items
payments
Everything looks fine.
But scalable database design requires asking deeper questions.
For orders, for example:
How many rows might this table contain?
How will orders be queried?
Can orders change after payment?
Should customer information be snapshotted?
How are refunds represented?
How are currencies handled?
What indexes are necessary?
Are IDs sequential or public?
Do we need tenant partitioning?
How long must records be retained?
The important insight is:
Database schemas encode product assumptions.
AI can generate migrations quickly, but you still need to determine those assumptions.
Teach AI to Think About Query Complexity
A classic AI-generated implementation might look like:
$orders = Order::all();
foreach ($orders as $order) {
echo $order->customer->name;
}
It works perfectly with twenty orders.
With 100,000 orders, things become different.
Good architecture should explicitly instruct agents to consider:
- pagination
- indexes
- N+1 queries
- selective columns
- aggregation cost
- query plans
- memory usage
- sorting
- filtering
Instead of:
Order::all();
you might use:
Order::query()
->with('customer:id,name')
->latest()
->paginate(50);
But even that may eventually require optimization.
A scalable engineering workflow tells AI to inspect the actual query pattern rather than blindly copying ORM conventions.
Put Indexing Into the Feature Design Process
Suppose your application frequently executes:
SELECT *
FROM orders
WHERE tenant_id = ?
AND status = ?
ORDER BY created_at DESC
LIMIT 50;
At sufficient scale, indexing becomes part of the feature.
You might need:
CREATE INDEX idx_orders_tenant_status_created
ON orders (tenant_id, status, created_at);
The important point is not the exact index.
The important point is that AI should be instructed to ask:
How will this data be accessed?
instead of only:
What columns should this table contain?
Avoid Premature Distributed Architecture
Developers frequently associate scalability with microservices.
That is a mistake.
Consider:
Application
|
├── User Service
├── Product Service
├── Order Service
├── Payment Service
├── Notification Service
└── Analytics Service
This architecture may scale beautifully.
It can also become a nightmare.
Every network boundary introduces potential:
timeout
retry
partial failure
version mismatch
latency
authentication failure
message duplication
deployment coordination
Distributed systems require explicit failure design.
AWS's current reliability guidance, for example, recommends making mutating operations idempotent so retries do not create unintended duplicate effects. It also recommends controlling retries with techniques such as exponential backoff, jitter, and retry limits.
AI can generate REST clients in seconds.
It cannot eliminate network uncertainty.
So extract services when there is a reason.
Good reasons include:
- independent scaling requirements
- strong team ownership boundaries
- security isolation
- fundamentally different workloads
- independent deployment requirements
- technological constraints
"Microservices are scalable" is not enough.
Use Queues to Separate User Latency From Heavy Work
Suppose a user uploads a video.
A naive request might do this:
Upload
↓
Generate thumbnail
↓
Transcode video
↓
Analyze metadata
↓
Send notification
↓
Return HTTP response
The user waits for everything.
A scalable design looks different.
Upload
↓
Save metadata
↓
Queue processing
↓
Return response
Background Workers
|
┌──────┼───────┐
▼ ▼ ▼
thumbnail transcode analysis
AI is particularly effective at implementing workers and background jobs once the architecture is defined.
The developer decides:
what must happen synchronously?
what can happen asynchronously?
what can fail independently?
what must be retried?
AI implements those decisions.
Make Background Jobs Idempotent
This deserves special attention.
Consider:
class ChargeCustomer
{
public function handle()
{
PaymentGateway::charge(
$this->customer,
$this->amount
);
}
}
What happens if:
payment succeeds
↓
worker crashes
↓
job remains unacknowledged
↓
queue retries
↓
customer charged again
The code looked correct.
The system was not correct.
A scalable system assumes work can execute more than once.
The job might instead use an idempotency key:
PaymentGateway::charge(
customer: $this->customer,
amount: $this->amount,
idempotencyKey: $this->paymentAttemptId
);
AWS describes this principle explicitly: retryable operations should avoid additional side effects when the same request is processed again.
Your AI instructions should therefore contain rules such as:
All payment jobs must be idempotent.
All webhook handlers must tolerate duplicate delivery.
All external API retries must consider whether the operation is safe to repeat.
These small instructions can prevent extremely expensive bugs.
Use Caching Intentionally
Another common AI response to performance problems is:
Add Redis caching.
That is not a caching strategy.
Before caching something, answer:
What are we caching?
Why?
How long?
What invalidates it?
Can stale data be returned?
What happens if Redis is unavailable?
Suppose this query runs constantly:
SELECT COUNT(*)
FROM orders
WHERE tenant_id = ?
AND status = 'pending';
You might cache:
tenant:42:pending_orders_count
But then:
When should it be invalidated?
After an order is created?
After status changes?
After cancellation?
After import?
After payment?
Cache invalidation is part of the architecture.
AI can implement the invalidation hooks quickly, but humans need to define the consistency expectation.
Build Stable Interfaces Between Components
Good scalability depends heavily on contracts.
Suppose the billing module exposes:
interface BillingService
{
public function subscribe(
User $user,
Plan $plan
): Subscription;
}
Other modules depend on this interface.
They should not depend on:
StripeSubscriptionRepository
StripeWebhookParser
StripePaymentIntentService
StripeCustomerModel
Why?
Because implementation details spread coupling.
If every module directly accesses another module's internals, AI-generated changes become dangerous.
A small implementation change can trigger modifications throughout the codebase.
Stable contracts reduce the blast radius.
AI Works Best With Strong Local Patterns
One of OpenAI's recommendations for Codex is to structure tasks similarly to GitHub issues and point agents toward existing implementations when relevant.
For example:
Implement password reset rate limiting.
Follow the same architecture used by
LoginRateLimiter.
Files to inspect:
src/Auth/LoginRateLimiter.php
src/Auth/LoginController.php
tests/Auth/LoginRateLimiterTest.php
This is dramatically safer than:
Add rate limiting.
The first prompt gives the agent a local architectural reference.
The second asks it to invent architecture.
That suggests a powerful rule:
Create one excellent implementation of every important pattern, then tell AI to follow it.
Examples include:
CreateOrder
UpdateSubscription
UploadFile
HandleWebhook
ProcessPayment
ExportReport
SendNotification
Once those patterns exist, development becomes pattern replication rather than architecture invention.
Tests Become More Important in the AI Era, Not Less
There is a common temptation:
AI wrote the code.
AI reviewed the code.
Therefore the code is probably correct.
This is dangerous.
AI can produce an implementation and then confidently approve its own incorrect assumptions.
The scalable approach is automated verification.
Think of this relationship:
AI speed increases
↓
number of changes increases
↓
verification requirements increase
If an engineering team moves from ten changes per day to fifty, manual verification becomes increasingly insufficient.
Use Multiple Layers of Testing
A scalable system should have several verification layers.
Unit tests
Validate isolated business rules.
public function test_discount_cannot_exceed_order_total()
{
$order = new Order(total: 100);
$this->expectException(
InvalidDiscountException::class
);
$order->applyDiscount(120);
}
Integration tests
Validate component interactions.
Database
Queue
Cache
External service adapters
API tests
Validate actual contracts.
POST /api/orders
201
{
"id": "...",
"status": "pending"
}
End-to-end tests
Validate critical user journeys.
Register
↓
Create organization
↓
Subscribe
↓
Create project
↓
Invite teammate
AI can generate all of these.
The important part is making tests a required deliverable.
Instead of:
Implement this feature.
write:
Implement this feature.
Add unit tests for business rules.
Add integration tests for persistence.
Add API tests for authorization and validation.
Run the relevant suite and fix failures before completing the task.
Every Bug Should Produce a Regression Test
This is one of the most effective rules you can introduce.
Suppose production reveals:
Users can redeem the same coupon twice
when two requests arrive simultaneously.
Do not ask AI:
Fix duplicate coupon redemption.
Ask:
Reproduce the duplicate coupon redemption bug
with a failing test.
Then fix the implementation.
The test must pass after the fix.
Now your repository gains permanent knowledge.
The bug becomes:
production incident
↓
regression test
↓
architectural knowledge
Over time, the test suite becomes a memory system.
This is especially valuable for AI agents that did not participate in earlier incidents.
Use Static Analysis as an Architectural Guardrail
AI agents are excellent at generating code.
Static analyzers are excellent at refusing certain classes of bad code.
Combine them.
Examples include:
TypeScript compiler
PHPStan
Psalm
ESLint
Ruff
mypy
SpotBugs
SonarQube
ArchUnit
Suppose your architecture says:
Domain code cannot depend on HTTP controllers.
Do not leave this entirely in documentation.
Where possible, enforce it.
For example:
Domain
X
Controllers
If an agent violates the rule, CI should fail.
The ultimate scalable AI workflow is not:
Tell AI what to do correctly.
It is:
Tell AI what to do
+
automatically reject invalid solutions.
Treat CI as the Gatekeeper
The development process should look something like:
AI Agent
|
▼
Code Change
|
▼
┌─────────────────────┐
│ CI │
├─────────────────────┤
│ Formatting │
│ Linting │
│ Static analysis │
│ Unit tests │
│ Integration tests │
│ Architecture tests │
│ Security checks │
│ Build │
└──────────┬──────────┘
│
pass
│
▼
Review
The stronger the pipeline becomes, the more safely AI can operate autonomously.
Observability Is Part of Scalability
Suppose AI helps you build an API handling 20,000 requests per minute.
Then customers begin reporting:
"Sometimes checkout takes 12 seconds."
What do you do?
If the application only logs:
Something went wrong
you have a problem.
Scalable systems require observability.
At minimum, consider:
Structured logs
Instead of:
Payment failed
log something structured:
{
"event": "payment_failed",
"payment_id": "pay_123",
"order_id": "ord_456",
"provider": "stripe",
"error_code": "timeout",
"duration_ms": 5021
}
Metrics
Track:
requests/sec
error rate
latency
database connections
queue depth
job failure rate
cache hit ratio
CPU
memory
external API latency
Tracing
A request might travel through:
API
↓
Order Service
↓
Payment Service
↓
Database
↓
Message Queue
Distributed tracing helps identify which stage consumed time.
The critical point is that observability should not be added after the system becomes large.
Build it while the architecture is still understandable.
Give AI Access to Operational Context Carefully
AI can help investigate production issues when given:
logs
stack traces
metrics
query plans
deployment diffs
For example:
p95 API latency increased from 180 ms to 1.8 seconds
after deployment 2026-08-07. Analyze these traces and
identify likely regressions.
This is far more effective than:
My application is slow. Fix it.
AI becomes powerful when provided evidence.
The general principle applies everywhere:
Context quality determines AI engineering quality.
Design for Horizontal Scaling
Suppose one application instance handles:
500 requests/second
Eventually you need:
Server A
Server B
Server C
Server D
behind a load balancer.
The application should therefore avoid storing critical state in local process memory.
Bad:
Server A memory:
user_123_session
If the next request reaches Server B:
session missing
Better:
Load Balancer
|
┌────────────┼────────────┐
▼ ▼ ▼
Server A Server B Server C
│ │ │
└────────────┼────────────┘
▼
Shared Session
Storage
Common choices include:
Redis
database-backed sessions
signed stateless tokens
This architectural rule should be documented for AI:
Application instances must remain stateless.
Do not introduce process-local state required across requests.
Now an agent writing features is less likely to accidentally undermine horizontal scalability.
Separate Storage Concerns
At small scale, developers often put everything into the relational database.
For example:
users
orders
sessions
logs
notifications
files
analytics
This can work initially.
As workloads grow, storage systems should match access patterns.
For example:
Transactional data
→ PostgreSQL / MySQL
Cache
→ Redis
Object files
→ S3-compatible storage
Search
→ OpenSearch / Elasticsearch
Analytics
→ analytical database or warehouse
Events
→ Kafka / queue system
Do not introduce all of these on day one.
But design interfaces that allow evolution.
For file uploads, for example, do not scatter:
file_put_contents(...)
throughout the application.
Create:
interface FileStorage
{
public function put(
string $path,
mixed $content
): string;
}
Today:
LocalFileStorage
Tomorrow:
S3FileStorage
The system evolves without rewriting every feature.
AI Makes Refactoring Economically Different
Historically, large refactors were expensive because someone had to:
find references
change interfaces
update implementations
modify tests
update documentation
fix compilation errors
Coding agents can now automate a large portion of this mechanical work.
OpenAI describes internal teams using Codex for large-codebase refactoring, performance optimization, test coverage improvements, and other engineering work.
This means architecture can be more evolutionary than before.
You do not need to perfectly predict the next five years.
Instead:
choose simple architecture
↓
create strong boundaries
↓
measure the system
↓
identify pressure points
↓
refactor deliberately
AI reduces the cost of the last step.
It does not remove the need for the first four.
Keep Functions and Modules Small Enough to Understand
AI can produce huge implementations because generating code is cheap.
This sometimes results in:
public function processOrder()
{
// validation
// inventory check
// customer lookup
// discount calculation
// tax calculation
// payment
// database updates
// email
// analytics
// webhook
// notification
// logging
}
A 500-line method may technically work.
It is still architectural debt.
Split responsibilities:
OrderProcessor
├── InventoryReservation
├── PricingCalculator
├── TaxCalculator
├── PaymentProcessor
├── OrderRepository
└── OrderEventPublisher
But avoid the opposite extreme too.
AI can also generate abstraction explosions:
OrderCreatorInterface
DefaultOrderCreator
OrderCreationManager
OrderCreationCoordinator
OrderCreationFactory
OrderCreationContext
OrderCreationProvider
for something that required twenty lines.
Scalable architecture is not maximum abstraction.
It is the right abstraction at the right boundary.
Make AI Explain Before It Changes Critical Systems
For dangerous or high-impact changes, use an analysis-first workflow.
Instead of:
Optimize our checkout database queries.
use:
Analyze checkout database behavior.
Do not modify code yet.
Identify:
1. expensive queries
2. N+1 patterns
3. missing indexes
4. unnecessary writes
5. transaction boundaries
6. potential race conditions
Propose changes with expected benefits and risks.
Then review the plan.
Then:
Implement items 1, 2, and 4.
Do not change the transaction model yet.
This creates an engineering loop:
Explore
↓
Understand
↓
Plan
↓
Review
↓
Implement
↓
Verify
OpenAI similarly recommends beginning with an ask or exploration mode for many Codex workflows rather than immediately requesting modifications.
Break Large Features Into Vertical Slices
Avoid prompts like:
Build the entire marketplace system.
Instead:
Phase 1
Vendor registration
Phase 2
Product creation
Phase 3
Product publishing
Phase 4
Checkout
Phase 5
Commission calculation
Phase 6
Vendor payouts
Even better, make each phase independently testable.
For example:
Vendor registration
Requirements:
- user can apply
- application status starts pending
- admin can approve
- approval creates vendor profile
- operation must be transactional
- duplicate approval must not duplicate profile
- audit event must be recorded
Now the AI has a well-defined engineering problem.
Use ADRs for Important Architecture Decisions
Architecture Decision Records are extremely useful in AI-assisted projects.
Suppose you choose PostgreSQL rather than MongoDB for your transactional system.
Create:
docs/adr/001-use-postgresql.md
Example:
# ADR 001: Use PostgreSQL as Primary Database
## Context
The application requires:
- relational transactions
- complex reporting
- financial consistency
- strong foreign-key relationships
## Decision
Use PostgreSQL as the primary transactional database.
## Consequences
Benefits:
- strong transactional guarantees
- mature relational capabilities
- rich indexing
- reliable reporting
Trade-offs:
- schema migrations must be managed
- horizontal write scaling may require future work
## Revisit When
- write throughput exceeds current architecture
- geographically distributed writes become necessary
Now six months later, an AI agent does not recommend MongoDB simply because a particular feature contains flexible JSON.
It can understand why PostgreSQL exists.
Architecture documentation becomes institutional memory.
Document the "Why", Not Just the "What"
Bad documentation:
We use Redis.
Better:
Redis stores short-lived cache entries and distributed locks.
Redis must not be the authoritative source for billing data.
If Redis is unavailable, critical transactional operations should remain correct, although performance may degrade.
That second version tells an agent how the technology fits into the architecture.
Security Needs Hard Boundaries
AI generates code quickly.
Security mistakes also scale quickly.
Repository rules should explicitly address:
authentication
authorization
tenant isolation
input validation
secret handling
logging
SQL injection
file uploads
webhooks
rate limiting
encryption
For example:
Never trust tenant_id supplied by the client.
Determine tenant context from authenticated membership.
Every tenant-owned query must explicitly scope by tenant.
That rule can prevent an entire class of vulnerabilities.
For external callbacks:
All webhook endpoints must verify provider signatures before processing payloads.
For secrets:
Never hardcode credentials.
Never log authorization headers.
Never commit .env files.
Do not expect AI to infer your threat model.
Encode it.
Review AI Code Differently
Traditional code review asks:
Is this code correct?
AI-assisted code review needs additional questions.
Does the code follow existing architecture?
AI frequently produces locally correct but globally inconsistent code.
Did it introduce a new pattern unnecessarily?
For example, perhaps the repository already has:
OrderRepository
and AI creates:
OrderDataAccessor
for the same responsibility.
Did it duplicate existing functionality?
Search the repository.
Are failure scenarios handled?
Especially:
timeouts
retries
duplicate requests
partial writes
concurrency
network failures
Are queries scalable?
Inspect loops and relationship loading.
Are transactions correct?
AI sometimes places too much inside a transaction or not enough.
Is observability sufficient?
Can the feature be debugged in production?
Did tests verify behavior or merely implementation?
This distinction matters.
Never Blindly Accept Large AI Diffs
If an AI agent changes:
47 files
3,200 lines
for a relatively small feature, investigate.
Large diffs hide mistakes.
Ask the agent:
Explain why each changed file is necessary.
Identify changes that can be removed without affecting
the requested functionality.
Then reduce the change.
AI code is cheap.
Code ownership is not.
Every generated line becomes something your team potentially maintains for years.
Use AI to Delete Code Too
One of the healthiest uses of AI is identifying unnecessary complexity.
Prompt:
Analyze this module for:
- duplicate abstractions
- dead code
- redundant wrappers
- unnecessary interfaces
- duplicated validation
- unused dependencies
Propose simplifications without changing behavior.
A scalable codebase is often one that contains less code, not more.
Design an AI-Friendly Repository
A repository that works well with AI often has a predictable structure.
For example:
project/
├── AGENTS.md
├── README.md
├── docs/
│ ├── architecture.md
│ ├── domains.md
│ ├── database.md
│ ├── security.md
│ └── adr/
├── src/
│ ├── Accounts/
│ ├── Billing/
│ ├── Projects/
│ └── Notifications/
├── tests/
├── scripts/
└── .github/
└── workflows/
The repository itself communicates its architecture.
Then AGENTS.md might say:
Before implementing changes:
1. Read docs/architecture.md.
2. Inspect the closest existing implementation.
3. Do not create new architectural patterns without justification.
4. Add tests for behavioral changes.
5. Run the relevant test suite.
6. Run static analysis.
7. Summarize architectural impact.
GitHub's documentation similarly recommends repository instructions that explain project structure, coding conventions, test frameworks, and build/run procedures.
Create Standard Commands
Agents perform much better when they do not need to discover basic development commands.
For example:
make setup
make dev
make test
make lint
make analyze
make build
or:
{
"scripts": {
"dev": "...",
"test": "...",
"lint": "...",
"typecheck": "...",
"build": "..."
}
}
Your agent instructions can simply state:
After changes run:
npm run lint
npm run typecheck
npm test
npm run build
OpenAI notes that coding agents perform better when repositories have configured development environments, reliable tests, and clear documentation.
The less time an AI agent spends guessing how your project works, the more time it can spend solving the actual problem.
A Practical AI-Assisted Development Workflow
Here is a workflow I would recommend for building serious software with AI.
Step 1: Describe the product
Write:
What are we building?
Who uses it?
What problem does it solve?
Example:
Multi-tenant project management SaaS for agencies.
Organizations contain users and projects.
Customers subscribe to plans with usage limits.
Step 2: Define expected scale
Do not say:
It should scale.
Write assumptions.
Initial:
1,000 organizations
10,000 users
Expected:
50,000 organizations
500,000 users
50 million tasks
Peak API traffic:
2,000 requests/second
These estimates do not need to be perfect.
They force architectural reasoning.
Step 3: Identify domains
Accounts
Organizations
Projects
Billing
Notifications
Reporting
Define ownership.
Step 4: Establish architectural rules
For example:
Modular monolith.
Modules communicate through public services/events.
Controllers contain no business logic.
Database access remains inside owning modules.
Background work runs through queues.
Application nodes remain stateless.
Step 5: Define critical invariants
Tenant data never crosses tenants.
Payment webhook processing is idempotent.
Usage counters cannot become negative.
Paid invoices cannot be modified.
Step 6: Create AI instructions
Add:
AGENTS.md
or the repository instruction format supported by your coding environment.
Document:
architecture
commands
patterns
testing
security
database conventions
Step 7: Build one module properly
Do not generate the whole product at once.
Implement:
Accounts
carefully.
Use it to establish patterns.
Step 8: Let AI replicate proven patterns
Now tell AI:
Implement Project membership following
the same layering, validation, testing,
and authorization approach used by
Organization membership.
This is where AI becomes extraordinarily productive.
Step 9: Automate verification
CI should enforce:
format
lint
types
architecture
tests
build
security
Step 10: Measure before optimizing
Deploy.
Measure:
latency
throughput
query times
queue depth
cache performance
error rates
resource usage
Then optimize actual bottlenecks.
Do not ask AI to solve hypothetical scale problems before evidence exists.
Example: Building a Scalable Order System With AI
Consider an e-commerce order service.
A simplistic flow:
POST /orders
|
▼
Create order
|
▼
Charge card
|
▼
Send email
We can improve it.
Client
|
▼
POST /orders
|
▼
API
|
▼
OrderService
|
├── Validate inventory
├── Calculate price
├── Create order
└── Commit transaction
|
▼
OrderCreated
|
┌─────┼─────────┐
▼ ▼ ▼
Payment Email Analytics
Worker Worker Worker
Now specify invariants:
Order creation is idempotent.
Inventory cannot become negative.
Order prices are snapshotted.
Payment failure does not delete the order.
Email failure does not affect payment.
Webhook processing supports duplicate events.
Then instruct AI:
Implement CreateOrder.
Architecture:
- controller handles HTTP only
- business logic belongs in CreateOrderService
- use a transaction for inventory reservation and order persistence
- use idempotency keys
- dispatch OrderCreated after persistence
- payment execution must happen asynchronously
- add concurrency tests around inventory
- add duplicate-request tests
- add structured logging
Now AI is not designing the system.
It is implementing a system you designed.
That distinction is enormous.
What AI Should Do in a Scalable Engineering Team
AI is particularly good at:
Repository exploration
Explain how payments currently work.
Pattern discovery
Find all implementations of idempotency handling.
Code generation
Implement this service following the existing pattern.
Testing
Generate edge-case tests for this state machine.
Refactoring
Move this domain logic from controllers into services
without changing API behavior.
Performance investigation
Analyze these database queries for N+1 patterns.
Documentation
Document the billing workflow based on the implementation.
Migration work
Update all call sites to use the new interface.
Review
Review this diff specifically for concurrency,
authorization, and database performance issues.
These tasks benefit enormously from AI.
What Humans Should Continue Owning
Humans should remain responsible for:
product goals
architecture
trade-offs
domain boundaries
risk tolerance
security model
data ownership
consistency requirements
operational strategy
cost constraints
AI can advise on all of these.
But someone needs to own the decision.
The Biggest Mistake: Optimizing for AI Output Instead of System Quality
AI produces visible progress.
You prompt:
Build notification system
and suddenly:
14 files created
2 migrations
6 services
3 jobs
4 tests
It feels productive.
But lines of code are not progress.
Architecture is not measured by how much software exists.
The relevant questions are:
Does this solve the problem?
Is the boundary correct?
Can this be changed later?
Can it fail safely?
Can it be observed?
Can it scale?
Can another engineer understand it?
Can an AI agent understand it six months later?
Sometimes the best AI-assisted implementation is fifty lines.
A Better Mental Model for AI-Assisted Software Engineering
Traditional development looked roughly like:
Developer
↓
Design
↓
Code
↓
Test
↓
Deploy
AI-assisted engineering increasingly looks like:
Human Engineer
|
Architecture + Constraints
|
▼
AI Agents
|
┌───────────┼────────────┐
▼ ▼ ▼
Coding Testing Refactoring
│ │ │
└───────────┼────────────┘
▼
Automated Guardrails
|
┌─────────────────┼──────────────────┐
▼ ▼ ▼
Tests Static Analysis Security
│ │ │
└─────────────────┼──────────────────┘
▼
Human Review
|
▼
Production
|
▼
Observability
|
▼
Feedback
That feedback goes back into:
tests
architecture
documentation
agent instructions
The system becomes easier to develop over time.
The Real Scalability Advantage of AI
AI's biggest impact on scalable software may not be code generation.
It may be reducing the cost of maintaining architectural consistency.
Imagine an engineer saying:
Find every endpoint that returns an unpaginated collection.
An agent can search the repository.
Then:
Identify all jobs that perform non-idempotent external API calls.
Then:
Find modules that directly access Billing models.
Then:
Find database queries inside loops.
Then:
Generate regression tests for these cases.
This turns architectural maintenance from an occasional manual audit into something much closer to continuous inspection.
That is extremely powerful.
Five Principles I Would Use on Every AI-Assisted Project
If I had to reduce this entire approach to five rules, they would be these.
1. Architecture before generation
Do not allow implementation speed to replace design.
Understand:
boundaries
data
failure modes
scale
ownership
before generating large amounts of code.
2. Constraints before prompts
Give AI a system of rules.
Do not repeatedly explain your architecture in every conversation.
Encode it in the repository.
3. Patterns before autonomy
Build one good example.
Then let agents repeat the pattern.
Consistency beats creativity in most production codebases.
4. Verification before trust
AI output is untrusted until verified.
Use:
tests
types
static analysis
CI
architecture rules
security checks
5. Measurement before optimization
Do not build Kafka because somebody said you might eventually have millions of users.
Measure the system.
Optimize the bottleneck that actually exists.
Final Thoughts
AI has made writing software dramatically cheaper.
That does not make software engineering less important.
It makes engineering discipline more important.
When generating code required significant human effort, development speed naturally limited how quickly complexity could accumulate.
AI removes part of that limitation.
A team can now produce enormous amounts of code very quickly.
Without strong architecture, that becomes enormous amounts of technical debt very quickly.
The teams that benefit most from AI will therefore not necessarily be those that generate the most code.
They will be the teams that create environments where AI can safely generate code.
They will build clear module boundaries.
They will define invariants.
They will document architectural decisions.
They will design retry-safe operations.
They will automate tests and static analysis.
They will build observable systems.
They will measure real bottlenecks.
They will maintain repositories that both humans and AI agents can understand.
And they will use AI for what it is exceptionally good at: accelerating implementation, exploration, testing, refactoring, investigation, and repetitive engineering work.
The goal is not:
AI writes my software.
A much better goal is:
I design a system where AI can safely help build,
test, evolve, and operate the software.
That is the difference between using AI to generate code and using AI to build scalable software.
Top comments (0)