DEV Community

Cover image for How to Build Scalable Software Using AI Without Creating an Unmaintainable Mess
Moniruzzaman Saikat
Moniruzzaman Saikat

Posted on

How to Build Scalable Software Using AI Without Creating an Unmaintainable Mess

AI has changed the speed at which software can be built.

A developer can describe an API endpoint and receive an implementation in seconds. An AI coding agent can inspect a repository, modify multiple files, generate tests, run commands, debug failures, and prepare a pull request.

This is a significant improvement in development productivity.

It also creates a new problem.

Generating software faster does not mean generating scalable software.

In fact, AI can help you create a poorly designed system faster than ever.

Ask an AI agent to "build a complete SaaS backend" and it may happily generate controllers, models, migrations, services, background jobs, authentication, caching, queues, and dozens of abstractions.

The application may even work.

Then six months later:

  • changing one feature breaks three others
  • database queries become increasingly expensive
  • business logic exists in five different places
  • background jobs execute twice
  • APIs become inconsistent
  • every developer implements things differently
  • tests become difficult to maintain
  • AI agents make increasingly dangerous changes because the architecture is unclear

The fundamental lesson is simple:

AI increases implementation capacity. Architecture determines whether that capacity produces a scalable system.

Building scalable software with AI therefore requires a different mindset from simply using AI to write code.

You need to design the environment in which AI writes code.

You need architectural boundaries.

You need rules.

You need automated verification.

You need documentation that machines can understand.

And increasingly, you need to treat your repository as an operating environment for both human developers and software engineering agents.

This article explains how to do that.


What Does "Scalable Software" Actually Mean?

Developers often use scalability to mean:

"The system can handle more traffic."

That is only one form of scalability.

A production system has to scale across several dimensions.

Traffic scalability

Can the system handle:

100 users
1,000 users
100,000 users
1,000,000 users
Enter fullscreen mode Exit fullscreen mode

without collapsing?

This involves:

  • application servers
  • database performance
  • caching
  • queues
  • load balancing
  • storage
  • connection management
  • horizontal scaling

Data scalability

What happens when:

orders = 10,000
Enter fullscreen mode Exit fullscreen mode

becomes:

orders = 500,000,000
Enter fullscreen mode Exit fullscreen mode

Queries that looked harmless during development may become serious bottlenecks.

Team scalability

Can twenty developers work on the codebase without constantly breaking each other's changes?

A system can handle millions of requests and still have terrible engineering scalability.

Feature scalability

Can you add new functionality without modifying half the application?

Good architecture minimizes the number of unrelated components affected by a change.

Operational scalability

Can your team understand what is happening when production fails?

Can you answer:

Why is checkout slow?

Which service is failing?

Which request triggered this exception?

How many jobs are stuck?

Which deployment introduced the regression?
Enter fullscreen mode Exit fullscreen mode

AI scalability

There is now another dimension worth considering.

Can multiple AI coding agents safely contribute to the repository?

This is becoming increasingly relevant.

Coding systems such as Codex and GitHub Copilot support repository-level instructions that give agents persistent information about project structure, conventions, testing, and validation. OpenAI recommends AGENTS.md for persistent repository context, while GitHub Copilot supports repository instructions and AGENTS.md files for similar purposes.

That means modern software architecture must increasingly optimize for two types of developers:

Human developers
+
AI engineering agents
Enter fullscreen mode Exit fullscreen mode

Fortunately, the things that make software easier for AI to understand are usually the same things that make it easier for humans to maintain.


AI Should Be an Implementation Engine, Not Your Architect

One of the most dangerous prompts in AI-assisted development is something like:

Build the complete backend architecture for my SaaS.
Enter fullscreen mode Exit fullscreen mode

The problem is not that AI cannot produce architecture.

It can.

The problem is that architecture is a collection of contextual decisions.

Consider a payment system.

Should you use:

monolith?
modular monolith?
microservices?
event-driven architecture?
serverless?
Enter fullscreen mode Exit fullscreen mode

There is no universally correct answer.

The decision depends on:

  • expected traffic
  • engineering team size
  • deployment model
  • transaction requirements
  • data consistency requirements
  • operational capabilities
  • budget
  • expected product evolution
  • failure tolerance
  • integration requirements

AI does not automatically know these constraints.

If you ask it to decide without supplying them, it will fill the missing context with assumptions.

The code may be technically reasonable while being completely wrong for your product.

A better relationship looks like this:

Human:
defines architecture

AI:
explores options
implements components
writes tests
performs refactoring
finds inconsistencies
reviews code
generates documentation
investigates failures
Enter fullscreen mode Exit fullscreen mode

Think of AI as an extremely fast engineering team working inside boundaries that you define.


Start With Architecture Before Writing Prompts

Suppose we are building a project management SaaS.

A naive architecture might look like:

Frontend
   |
Backend API
   |
Database
Enter fullscreen mode Exit fullscreen mode

This is technically architecture, but it tells an AI agent almost nothing.

Instead, define the important boundaries.

                    ┌──────────────────┐
                    │     Web App      │
                    └────────┬─────────┘
                             │
                             ▼
                    ┌──────────────────┐
                    │    API Layer     │
                    └────────┬─────────┘
                             │
              ┌──────────────┼──────────────┐
              │              │              │
              ▼              ▼              ▼
        ┌──────────┐   ┌──────────┐   ┌──────────┐
        │ Projects │   │ Billing  │   │ Accounts │
        └─────┬────┘   └────┬─────┘   └────┬─────┘
              │             │              │
              └─────────────┼──────────────┘
                            │
                            ▼
                    ┌──────────────────┐
                    │     Database     │
                    └──────────────────┘

                     Background Work
                            │
                            ▼
                    ┌──────────────────┐
                    │      Queue       │
                    └──────────────────┘
Enter fullscreen mode Exit fullscreen mode

Now define responsibilities.

API layer

Responsible for:

  • authentication
  • request parsing
  • validation
  • authorization
  • response formatting

Not responsible for business rules.

Project module

Responsible for:

  • projects
  • tasks
  • assignments
  • project permissions
  • project workflows

Billing module

Responsible for:

  • subscriptions
  • invoices
  • payments
  • billing providers
  • usage calculations

Account module

Responsible for:

  • organizations
  • users
  • memberships
  • roles

This creates boundaries an AI agent can follow.


Start With a Modular Monolith More Often Than You Think

AI makes microservices unusually tempting.

You can ask:

Create an authentication microservice.
Enter fullscreen mode Exit fullscreen mode

Then:

Create a billing microservice.
Enter fullscreen mode Exit fullscreen mode

Then:

Create an order service.
Enter fullscreen mode Exit fullscreen mode

Soon you have twelve services.

Unfortunately, you also have:

  • twelve deployments
  • twelve logs
  • twelve CI pipelines
  • distributed tracing
  • network failures
  • duplicated authentication logic
  • service discovery
  • asynchronous consistency
  • versioned contracts
  • complicated local development

You solved a scaling problem you probably did not have.

For many SaaS applications, a modular monolith is a better starting point.

For example:

src/
    Modules/
        Accounts/
        Projects/
        Billing/
        Notifications/
        Reporting/
Enter fullscreen mode Exit fullscreen mode

Each module contains its own:

Controllers
Services
Models
Repositories
Events
Jobs
Policies
Tests
Enter fullscreen mode Exit fullscreen mode

You receive many benefits of service separation without distributed-system complexity.

Later, if billing becomes computationally expensive or organizationally independent, you can extract it.

Modular Monolith

      ↓

Identify scaling boundary

      ↓

Extract Billing Module

      ↓

Billing Service
Enter fullscreen mode Exit fullscreen mode

AI is extremely useful during this extraction because clear boundaries allow it to identify dependencies and migrate them systematically.

The important decision is establishing those boundaries early.


Create an Architecture Contract for AI

One of the biggest improvements you can make to AI-assisted development is creating repository-level engineering instructions.

OpenAI specifically recommends using AGENTS.md to supply persistent context to Codex. OpenAI's guidance notes that these files can document naming conventions, business logic, dependencies, and repository-specific information that agents cannot reliably infer from source code alone.

GitHub Copilot similarly supports repository-wide instructions, path-specific instructions, and AGENTS.md files.

This changes how we should think about documentation.

Documentation is no longer only for developers.

It can function as executable context for engineering agents.

A useful file could look like this:

# Architecture

This application uses a modular monolith.

Modules:

- Accounts
- Projects
- Billing
- Notifications
- Reporting

Modules must not directly access another module's database models.

Cross-module communication must use application services or domain events.

# Controllers

Controllers are responsible only for:

- request validation
- authorization
- calling application services
- returning responses

Never place business logic inside controllers.

# Database

All list endpoints must support pagination.

Avoid queries inside loops.

Use eager loading when relationships are required.

All new queries on high-volume tables must consider indexes.

# Jobs

External API calls should execute asynchronously unless an immediate response is required.

Jobs must be safe to retry where possible.

# Testing

All business logic requires unit or integration tests.

Every bug fix requires a regression test.

Run:

npm test

before completing any task.

# Security

Never expose database IDs when public UUIDs are available.

Never log passwords, tokens, secrets, or payment credentials.
Enter fullscreen mode Exit fullscreen mode

Now imagine five AI agents working on different parts of the application.

Without this document, each agent may invent its own conventions.

With it, you have dramatically increased architectural consistency.


Give AI Constraints, Not Just Requirements

Compare these two prompts.

Weak prompt

Create an endpoint for creating orders.
Enter fullscreen mode Exit fullscreen mode

AI has to decide:

  • validation
  • authorization
  • architecture
  • transaction handling
  • event handling
  • error handling
  • testing
  • response structure

Different runs may produce completely different solutions.

Now consider:

Implement POST /api/orders.

Requirements:

- authenticated users only
- use CreateOrderRequest for validation
- controller must contain no business logic
- business logic belongs in OrderService
- wrap inventory reservation and order creation in a database transaction
- dispatch OrderCreated only after successful persistence
- duplicate requests with the same idempotency key must not create multiple orders
- return OrderResource
- add integration tests
- follow the existing CreateInvoice implementation
Enter fullscreen mode Exit fullscreen mode

The second prompt does something important.

It reduces the AI's decision surface.

This is one of the biggest secrets of scalable AI-assisted development.

The more architectural decisions you encode into the system, the fewer architectural decisions AI has to invent.


Think in Invariants

Strong systems are built around invariants.

An invariant is something that must always remain true.

For an e-commerce platform:

stock >= 0
Enter fullscreen mode Exit fullscreen mode

For financial software:

total debits = total credits
Enter fullscreen mode Exit fullscreen mode

For a subscription system:

a customer cannot have two active subscriptions
for the same plan unless explicitly allowed
Enter fullscreen mode Exit fullscreen mode

For multi-tenant SaaS:

a user from Tenant A must never access Tenant B's data
Enter fullscreen mode Exit fullscreen mode

AI should know these rules.

A repository might contain:

## Critical Domain Invariants

1. Inventory must never become negative.
2. Payment callbacks may be delivered multiple times.
3. Tenant data must always be scoped by tenant_id.
4. Subscription state changes must go through SubscriptionService.
5. Paid invoices cannot be deleted.
Enter fullscreen mode Exit fullscreen mode

Now an agent modifying the system has significantly more useful context than one merely reading database schemas.


Design the Database Before Asking AI to Generate Models

AI coding tools make schema generation extremely convenient.

That convenience can hide bad database design.

Imagine asking:

Create tables for an e-commerce system.
Enter fullscreen mode Exit fullscreen mode

You might receive:

users
products
orders
order_items
payments
Enter fullscreen mode Exit fullscreen mode

Everything looks fine.

But scalable database design requires asking deeper questions.

For orders, for example:

How many rows might this table contain?

How will orders be queried?

Can orders change after payment?

Should customer information be snapshotted?

How are refunds represented?

How are currencies handled?

What indexes are necessary?

Are IDs sequential or public?

Do we need tenant partitioning?

How long must records be retained?
Enter fullscreen mode Exit fullscreen mode

The important insight is:

Database schemas encode product assumptions.

AI can generate migrations quickly, but you still need to determine those assumptions.


Teach AI to Think About Query Complexity

A classic AI-generated implementation might look like:

$orders = Order::all();

foreach ($orders as $order) {
    echo $order->customer->name;
}
Enter fullscreen mode Exit fullscreen mode

It works perfectly with twenty orders.

With 100,000 orders, things become different.

Good architecture should explicitly instruct agents to consider:

  • pagination
  • indexes
  • N+1 queries
  • selective columns
  • aggregation cost
  • query plans
  • memory usage
  • sorting
  • filtering

Instead of:

Order::all();
Enter fullscreen mode Exit fullscreen mode

you might use:

Order::query()
    ->with('customer:id,name')
    ->latest()
    ->paginate(50);
Enter fullscreen mode Exit fullscreen mode

But even that may eventually require optimization.

A scalable engineering workflow tells AI to inspect the actual query pattern rather than blindly copying ORM conventions.


Put Indexing Into the Feature Design Process

Suppose your application frequently executes:

SELECT *
FROM orders
WHERE tenant_id = ?
AND status = ?
ORDER BY created_at DESC
LIMIT 50;
Enter fullscreen mode Exit fullscreen mode

At sufficient scale, indexing becomes part of the feature.

You might need:

CREATE INDEX idx_orders_tenant_status_created
ON orders (tenant_id, status, created_at);
Enter fullscreen mode Exit fullscreen mode

The important point is not the exact index.

The important point is that AI should be instructed to ask:

How will this data be accessed?
Enter fullscreen mode Exit fullscreen mode

instead of only:

What columns should this table contain?
Enter fullscreen mode Exit fullscreen mode

Avoid Premature Distributed Architecture

Developers frequently associate scalability with microservices.

That is a mistake.

Consider:

Application
    |
    ├── User Service
    ├── Product Service
    ├── Order Service
    ├── Payment Service
    ├── Notification Service
    └── Analytics Service
Enter fullscreen mode Exit fullscreen mode

This architecture may scale beautifully.

It can also become a nightmare.

Every network boundary introduces potential:

timeout
retry
partial failure
version mismatch
latency
authentication failure
message duplication
deployment coordination
Enter fullscreen mode Exit fullscreen mode

Distributed systems require explicit failure design.

AWS's current reliability guidance, for example, recommends making mutating operations idempotent so retries do not create unintended duplicate effects. It also recommends controlling retries with techniques such as exponential backoff, jitter, and retry limits.

AI can generate REST clients in seconds.

It cannot eliminate network uncertainty.

So extract services when there is a reason.

Good reasons include:

  • independent scaling requirements
  • strong team ownership boundaries
  • security isolation
  • fundamentally different workloads
  • independent deployment requirements
  • technological constraints

"Microservices are scalable" is not enough.


Use Queues to Separate User Latency From Heavy Work

Suppose a user uploads a video.

A naive request might do this:

Upload
  ↓
Generate thumbnail
  ↓
Transcode video
  ↓
Analyze metadata
  ↓
Send notification
  ↓
Return HTTP response
Enter fullscreen mode Exit fullscreen mode

The user waits for everything.

A scalable design looks different.

Upload
  ↓
Save metadata
  ↓
Queue processing
  ↓
Return response

      Background Workers
             |
      ┌──────┼───────┐
      ▼      ▼       ▼
 thumbnail transcode analysis
Enter fullscreen mode Exit fullscreen mode

AI is particularly effective at implementing workers and background jobs once the architecture is defined.

The developer decides:

what must happen synchronously?
what can happen asynchronously?
what can fail independently?
what must be retried?
Enter fullscreen mode Exit fullscreen mode

AI implements those decisions.


Make Background Jobs Idempotent

This deserves special attention.

Consider:

class ChargeCustomer
{
    public function handle()
    {
        PaymentGateway::charge(
            $this->customer,
            $this->amount
        );
    }
}
Enter fullscreen mode Exit fullscreen mode

What happens if:

payment succeeds
↓
worker crashes
↓
job remains unacknowledged
↓
queue retries
↓
customer charged again
Enter fullscreen mode Exit fullscreen mode

The code looked correct.

The system was not correct.

A scalable system assumes work can execute more than once.

The job might instead use an idempotency key:

PaymentGateway::charge(
    customer: $this->customer,
    amount: $this->amount,
    idempotencyKey: $this->paymentAttemptId
);
Enter fullscreen mode Exit fullscreen mode

AWS describes this principle explicitly: retryable operations should avoid additional side effects when the same request is processed again.

Your AI instructions should therefore contain rules such as:

All payment jobs must be idempotent.

All webhook handlers must tolerate duplicate delivery.

All external API retries must consider whether the operation is safe to repeat.
Enter fullscreen mode Exit fullscreen mode

These small instructions can prevent extremely expensive bugs.


Use Caching Intentionally

Another common AI response to performance problems is:

Add Redis caching.
Enter fullscreen mode Exit fullscreen mode

That is not a caching strategy.

Before caching something, answer:

What are we caching?

Why?

How long?

What invalidates it?

Can stale data be returned?

What happens if Redis is unavailable?
Enter fullscreen mode Exit fullscreen mode

Suppose this query runs constantly:

SELECT COUNT(*)
FROM orders
WHERE tenant_id = ?
AND status = 'pending';
Enter fullscreen mode Exit fullscreen mode

You might cache:

tenant:42:pending_orders_count
Enter fullscreen mode Exit fullscreen mode

But then:

When should it be invalidated?

After an order is created?

After status changes?

After cancellation?

After import?

After payment?
Enter fullscreen mode Exit fullscreen mode

Cache invalidation is part of the architecture.

AI can implement the invalidation hooks quickly, but humans need to define the consistency expectation.


Build Stable Interfaces Between Components

Good scalability depends heavily on contracts.

Suppose the billing module exposes:

interface BillingService
{
    public function subscribe(
        User $user,
        Plan $plan
    ): Subscription;
}
Enter fullscreen mode Exit fullscreen mode

Other modules depend on this interface.

They should not depend on:

StripeSubscriptionRepository
StripeWebhookParser
StripePaymentIntentService
StripeCustomerModel
Enter fullscreen mode Exit fullscreen mode

Why?

Because implementation details spread coupling.

If every module directly accesses another module's internals, AI-generated changes become dangerous.

A small implementation change can trigger modifications throughout the codebase.

Stable contracts reduce the blast radius.


AI Works Best With Strong Local Patterns

One of OpenAI's recommendations for Codex is to structure tasks similarly to GitHub issues and point agents toward existing implementations when relevant.

For example:

Implement password reset rate limiting.

Follow the same architecture used by
LoginRateLimiter.

Files to inspect:

src/Auth/LoginRateLimiter.php
src/Auth/LoginController.php
tests/Auth/LoginRateLimiterTest.php
Enter fullscreen mode Exit fullscreen mode

This is dramatically safer than:

Add rate limiting.
Enter fullscreen mode Exit fullscreen mode

The first prompt gives the agent a local architectural reference.

The second asks it to invent architecture.

That suggests a powerful rule:

Create one excellent implementation of every important pattern, then tell AI to follow it.

Examples include:

CreateOrder
UpdateSubscription
UploadFile
HandleWebhook
ProcessPayment
ExportReport
SendNotification
Enter fullscreen mode Exit fullscreen mode

Once those patterns exist, development becomes pattern replication rather than architecture invention.


Tests Become More Important in the AI Era, Not Less

There is a common temptation:

AI wrote the code.
AI reviewed the code.
Therefore the code is probably correct.
Enter fullscreen mode Exit fullscreen mode

This is dangerous.

AI can produce an implementation and then confidently approve its own incorrect assumptions.

The scalable approach is automated verification.

Think of this relationship:

AI speed increases
       ↓
number of changes increases
       ↓
verification requirements increase
Enter fullscreen mode Exit fullscreen mode

If an engineering team moves from ten changes per day to fifty, manual verification becomes increasingly insufficient.


Use Multiple Layers of Testing

A scalable system should have several verification layers.

Unit tests

Validate isolated business rules.

public function test_discount_cannot_exceed_order_total()
{
    $order = new Order(total: 100);

    $this->expectException(
        InvalidDiscountException::class
    );

    $order->applyDiscount(120);
}
Enter fullscreen mode Exit fullscreen mode

Integration tests

Validate component interactions.

Database
Queue
Cache
External service adapters
Enter fullscreen mode Exit fullscreen mode

API tests

Validate actual contracts.

POST /api/orders

201
{
    "id": "...",
    "status": "pending"
}
Enter fullscreen mode Exit fullscreen mode

End-to-end tests

Validate critical user journeys.

Register
↓
Create organization
↓
Subscribe
↓
Create project
↓
Invite teammate
Enter fullscreen mode Exit fullscreen mode

AI can generate all of these.

The important part is making tests a required deliverable.

Instead of:

Implement this feature.
Enter fullscreen mode Exit fullscreen mode

write:

Implement this feature.

Add unit tests for business rules.

Add integration tests for persistence.

Add API tests for authorization and validation.

Run the relevant suite and fix failures before completing the task.
Enter fullscreen mode Exit fullscreen mode

Every Bug Should Produce a Regression Test

This is one of the most effective rules you can introduce.

Suppose production reveals:

Users can redeem the same coupon twice
when two requests arrive simultaneously.
Enter fullscreen mode Exit fullscreen mode

Do not ask AI:

Fix duplicate coupon redemption.
Enter fullscreen mode Exit fullscreen mode

Ask:

Reproduce the duplicate coupon redemption bug
with a failing test.

Then fix the implementation.

The test must pass after the fix.
Enter fullscreen mode Exit fullscreen mode

Now your repository gains permanent knowledge.

The bug becomes:

production incident
      ↓
regression test
      ↓
architectural knowledge
Enter fullscreen mode Exit fullscreen mode

Over time, the test suite becomes a memory system.

This is especially valuable for AI agents that did not participate in earlier incidents.


Use Static Analysis as an Architectural Guardrail

AI agents are excellent at generating code.

Static analyzers are excellent at refusing certain classes of bad code.

Combine them.

Examples include:

TypeScript compiler
PHPStan
Psalm
ESLint
Ruff
mypy
SpotBugs
SonarQube
ArchUnit
Enter fullscreen mode Exit fullscreen mode

Suppose your architecture says:

Domain code cannot depend on HTTP controllers.
Enter fullscreen mode Exit fullscreen mode

Do not leave this entirely in documentation.

Where possible, enforce it.

For example:

Domain
   X
Controllers
Enter fullscreen mode Exit fullscreen mode

If an agent violates the rule, CI should fail.

The ultimate scalable AI workflow is not:

Tell AI what to do correctly.
Enter fullscreen mode Exit fullscreen mode

It is:

Tell AI what to do
+
automatically reject invalid solutions.
Enter fullscreen mode Exit fullscreen mode

Treat CI as the Gatekeeper

The development process should look something like:

                AI Agent
                   |
                   ▼
               Code Change
                   |
                   ▼
        ┌─────────────────────┐
        │         CI          │
        ├─────────────────────┤
        │ Formatting          │
        │ Linting             │
        │ Static analysis     │
        │ Unit tests          │
        │ Integration tests   │
        │ Architecture tests  │
        │ Security checks     │
        │ Build               │
        └──────────┬──────────┘
                   │
                  pass
                   │
                   ▼
                Review
Enter fullscreen mode Exit fullscreen mode

The stronger the pipeline becomes, the more safely AI can operate autonomously.


Observability Is Part of Scalability

Suppose AI helps you build an API handling 20,000 requests per minute.

Then customers begin reporting:

"Sometimes checkout takes 12 seconds."

What do you do?

If the application only logs:

Something went wrong
Enter fullscreen mode Exit fullscreen mode

you have a problem.

Scalable systems require observability.

At minimum, consider:

Structured logs

Instead of:

Payment failed
Enter fullscreen mode Exit fullscreen mode

log something structured:

{
  "event": "payment_failed",
  "payment_id": "pay_123",
  "order_id": "ord_456",
  "provider": "stripe",
  "error_code": "timeout",
  "duration_ms": 5021
}
Enter fullscreen mode Exit fullscreen mode

Metrics

Track:

requests/sec
error rate
latency
database connections
queue depth
job failure rate
cache hit ratio
CPU
memory
external API latency
Enter fullscreen mode Exit fullscreen mode

Tracing

A request might travel through:

API
 ↓
Order Service
 ↓
Payment Service
 ↓
Database
 ↓
Message Queue
Enter fullscreen mode Exit fullscreen mode

Distributed tracing helps identify which stage consumed time.

The critical point is that observability should not be added after the system becomes large.

Build it while the architecture is still understandable.


Give AI Access to Operational Context Carefully

AI can help investigate production issues when given:

logs
stack traces
metrics
query plans
deployment diffs
Enter fullscreen mode Exit fullscreen mode

For example:

p95 API latency increased from 180 ms to 1.8 seconds
after deployment 2026-08-07. Analyze these traces and
identify likely regressions.
Enter fullscreen mode Exit fullscreen mode

This is far more effective than:

My application is slow. Fix it.
Enter fullscreen mode Exit fullscreen mode

AI becomes powerful when provided evidence.

The general principle applies everywhere:

Context quality determines AI engineering quality.


Design for Horizontal Scaling

Suppose one application instance handles:

500 requests/second
Enter fullscreen mode Exit fullscreen mode

Eventually you need:

Server A
Server B
Server C
Server D
Enter fullscreen mode Exit fullscreen mode

behind a load balancer.

The application should therefore avoid storing critical state in local process memory.

Bad:

Server A memory:
user_123_session
Enter fullscreen mode Exit fullscreen mode

If the next request reaches Server B:

session missing
Enter fullscreen mode Exit fullscreen mode

Better:

                Load Balancer
                     |
        ┌────────────┼────────────┐
        ▼            ▼            ▼
     Server A     Server B     Server C
        │            │            │
        └────────────┼────────────┘
                     ▼
               Shared Session
                  Storage
Enter fullscreen mode Exit fullscreen mode

Common choices include:

Redis
database-backed sessions
signed stateless tokens
Enter fullscreen mode Exit fullscreen mode

This architectural rule should be documented for AI:

Application instances must remain stateless.

Do not introduce process-local state required across requests.
Enter fullscreen mode Exit fullscreen mode

Now an agent writing features is less likely to accidentally undermine horizontal scalability.


Separate Storage Concerns

At small scale, developers often put everything into the relational database.

For example:

users
orders
sessions
logs
notifications
files
analytics
Enter fullscreen mode Exit fullscreen mode

This can work initially.

As workloads grow, storage systems should match access patterns.

For example:

Transactional data
→ PostgreSQL / MySQL

Cache
→ Redis

Object files
→ S3-compatible storage

Search
→ OpenSearch / Elasticsearch

Analytics
→ analytical database or warehouse

Events
→ Kafka / queue system
Enter fullscreen mode Exit fullscreen mode

Do not introduce all of these on day one.

But design interfaces that allow evolution.

For file uploads, for example, do not scatter:

file_put_contents(...)
Enter fullscreen mode Exit fullscreen mode

throughout the application.

Create:

interface FileStorage
{
    public function put(
        string $path,
        mixed $content
    ): string;
}
Enter fullscreen mode Exit fullscreen mode

Today:

LocalFileStorage
Enter fullscreen mode Exit fullscreen mode

Tomorrow:

S3FileStorage
Enter fullscreen mode Exit fullscreen mode

The system evolves without rewriting every feature.


AI Makes Refactoring Economically Different

Historically, large refactors were expensive because someone had to:

find references
change interfaces
update implementations
modify tests
update documentation
fix compilation errors
Enter fullscreen mode Exit fullscreen mode

Coding agents can now automate a large portion of this mechanical work.

OpenAI describes internal teams using Codex for large-codebase refactoring, performance optimization, test coverage improvements, and other engineering work.

This means architecture can be more evolutionary than before.

You do not need to perfectly predict the next five years.

Instead:

choose simple architecture
       ↓
create strong boundaries
       ↓
measure the system
       ↓
identify pressure points
       ↓
refactor deliberately
Enter fullscreen mode Exit fullscreen mode

AI reduces the cost of the last step.

It does not remove the need for the first four.


Keep Functions and Modules Small Enough to Understand

AI can produce huge implementations because generating code is cheap.

This sometimes results in:

public function processOrder()
{
    // validation
    // inventory check
    // customer lookup
    // discount calculation
    // tax calculation
    // payment
    // database updates
    // email
    // analytics
    // webhook
    // notification
    // logging
}
Enter fullscreen mode Exit fullscreen mode

A 500-line method may technically work.

It is still architectural debt.

Split responsibilities:

OrderProcessor
├── InventoryReservation
├── PricingCalculator
├── TaxCalculator
├── PaymentProcessor
├── OrderRepository
└── OrderEventPublisher
Enter fullscreen mode Exit fullscreen mode

But avoid the opposite extreme too.

AI can also generate abstraction explosions:

OrderCreatorInterface
DefaultOrderCreator
OrderCreationManager
OrderCreationCoordinator
OrderCreationFactory
OrderCreationContext
OrderCreationProvider
Enter fullscreen mode Exit fullscreen mode

for something that required twenty lines.

Scalable architecture is not maximum abstraction.

It is the right abstraction at the right boundary.


Make AI Explain Before It Changes Critical Systems

For dangerous or high-impact changes, use an analysis-first workflow.

Instead of:

Optimize our checkout database queries.
Enter fullscreen mode Exit fullscreen mode

use:

Analyze checkout database behavior.

Do not modify code yet.

Identify:

1. expensive queries
2. N+1 patterns
3. missing indexes
4. unnecessary writes
5. transaction boundaries
6. potential race conditions

Propose changes with expected benefits and risks.
Enter fullscreen mode Exit fullscreen mode

Then review the plan.

Then:

Implement items 1, 2, and 4.

Do not change the transaction model yet.
Enter fullscreen mode Exit fullscreen mode

This creates an engineering loop:

Explore
   ↓
Understand
   ↓
Plan
   ↓
Review
   ↓
Implement
   ↓
Verify
Enter fullscreen mode Exit fullscreen mode

OpenAI similarly recommends beginning with an ask or exploration mode for many Codex workflows rather than immediately requesting modifications.


Break Large Features Into Vertical Slices

Avoid prompts like:

Build the entire marketplace system.
Enter fullscreen mode Exit fullscreen mode

Instead:

Phase 1
Vendor registration

Phase 2
Product creation

Phase 3
Product publishing

Phase 4
Checkout

Phase 5
Commission calculation

Phase 6
Vendor payouts
Enter fullscreen mode Exit fullscreen mode

Even better, make each phase independently testable.

For example:

Vendor registration

Requirements:
- user can apply
- application status starts pending
- admin can approve
- approval creates vendor profile
- operation must be transactional
- duplicate approval must not duplicate profile
- audit event must be recorded
Enter fullscreen mode Exit fullscreen mode

Now the AI has a well-defined engineering problem.


Use ADRs for Important Architecture Decisions

Architecture Decision Records are extremely useful in AI-assisted projects.

Suppose you choose PostgreSQL rather than MongoDB for your transactional system.

Create:

docs/adr/001-use-postgresql.md
Enter fullscreen mode Exit fullscreen mode

Example:

# ADR 001: Use PostgreSQL as Primary Database

## Context

The application requires:

- relational transactions
- complex reporting
- financial consistency
- strong foreign-key relationships

## Decision

Use PostgreSQL as the primary transactional database.

## Consequences

Benefits:

- strong transactional guarantees
- mature relational capabilities
- rich indexing
- reliable reporting

Trade-offs:

- schema migrations must be managed
- horizontal write scaling may require future work

## Revisit When

- write throughput exceeds current architecture
- geographically distributed writes become necessary
Enter fullscreen mode Exit fullscreen mode

Now six months later, an AI agent does not recommend MongoDB simply because a particular feature contains flexible JSON.

It can understand why PostgreSQL exists.

Architecture documentation becomes institutional memory.


Document the "Why", Not Just the "What"

Bad documentation:

We use Redis.
Enter fullscreen mode Exit fullscreen mode

Better:

Redis stores short-lived cache entries and distributed locks.

Redis must not be the authoritative source for billing data.

If Redis is unavailable, critical transactional operations should remain correct, although performance may degrade.
Enter fullscreen mode Exit fullscreen mode

That second version tells an agent how the technology fits into the architecture.


Security Needs Hard Boundaries

AI generates code quickly.

Security mistakes also scale quickly.

Repository rules should explicitly address:

authentication
authorization
tenant isolation
input validation
secret handling
logging
SQL injection
file uploads
webhooks
rate limiting
encryption
Enter fullscreen mode Exit fullscreen mode

For example:

Never trust tenant_id supplied by the client.

Determine tenant context from authenticated membership.

Every tenant-owned query must explicitly scope by tenant.
Enter fullscreen mode Exit fullscreen mode

That rule can prevent an entire class of vulnerabilities.

For external callbacks:

All webhook endpoints must verify provider signatures before processing payloads.
Enter fullscreen mode Exit fullscreen mode

For secrets:

Never hardcode credentials.

Never log authorization headers.

Never commit .env files.
Enter fullscreen mode Exit fullscreen mode

Do not expect AI to infer your threat model.

Encode it.


Review AI Code Differently

Traditional code review asks:

Is this code correct?
Enter fullscreen mode Exit fullscreen mode

AI-assisted code review needs additional questions.

Does the code follow existing architecture?

AI frequently produces locally correct but globally inconsistent code.

Did it introduce a new pattern unnecessarily?

For example, perhaps the repository already has:

OrderRepository
Enter fullscreen mode Exit fullscreen mode

and AI creates:

OrderDataAccessor
Enter fullscreen mode Exit fullscreen mode

for the same responsibility.

Did it duplicate existing functionality?

Search the repository.

Are failure scenarios handled?

Especially:

timeouts
retries
duplicate requests
partial writes
concurrency
network failures
Enter fullscreen mode Exit fullscreen mode

Are queries scalable?

Inspect loops and relationship loading.

Are transactions correct?

AI sometimes places too much inside a transaction or not enough.

Is observability sufficient?

Can the feature be debugged in production?

Did tests verify behavior or merely implementation?

This distinction matters.


Never Blindly Accept Large AI Diffs

If an AI agent changes:

47 files
3,200 lines
Enter fullscreen mode Exit fullscreen mode

for a relatively small feature, investigate.

Large diffs hide mistakes.

Ask the agent:

Explain why each changed file is necessary.

Identify changes that can be removed without affecting
the requested functionality.
Enter fullscreen mode Exit fullscreen mode

Then reduce the change.

AI code is cheap.

Code ownership is not.

Every generated line becomes something your team potentially maintains for years.


Use AI to Delete Code Too

One of the healthiest uses of AI is identifying unnecessary complexity.

Prompt:

Analyze this module for:

- duplicate abstractions
- dead code
- redundant wrappers
- unnecessary interfaces
- duplicated validation
- unused dependencies

Propose simplifications without changing behavior.
Enter fullscreen mode Exit fullscreen mode

A scalable codebase is often one that contains less code, not more.


Design an AI-Friendly Repository

A repository that works well with AI often has a predictable structure.

For example:

project/
├── AGENTS.md
├── README.md
├── docs/
│   ├── architecture.md
│   ├── domains.md
│   ├── database.md
│   ├── security.md
│   └── adr/
├── src/
│   ├── Accounts/
│   ├── Billing/
│   ├── Projects/
│   └── Notifications/
├── tests/
├── scripts/
└── .github/
    └── workflows/
Enter fullscreen mode Exit fullscreen mode

The repository itself communicates its architecture.

Then AGENTS.md might say:

Before implementing changes:

1. Read docs/architecture.md.
2. Inspect the closest existing implementation.
3. Do not create new architectural patterns without justification.
4. Add tests for behavioral changes.
5. Run the relevant test suite.
6. Run static analysis.
7. Summarize architectural impact.
Enter fullscreen mode Exit fullscreen mode

GitHub's documentation similarly recommends repository instructions that explain project structure, coding conventions, test frameworks, and build/run procedures.


Create Standard Commands

Agents perform much better when they do not need to discover basic development commands.

For example:

make setup
make dev
make test
make lint
make analyze
make build
Enter fullscreen mode Exit fullscreen mode

or:

{
  "scripts": {
    "dev": "...",
    "test": "...",
    "lint": "...",
    "typecheck": "...",
    "build": "..."
  }
}
Enter fullscreen mode Exit fullscreen mode

Your agent instructions can simply state:

After changes run:

npm run lint
npm run typecheck
npm test
npm run build
Enter fullscreen mode Exit fullscreen mode

OpenAI notes that coding agents perform better when repositories have configured development environments, reliable tests, and clear documentation.

The less time an AI agent spends guessing how your project works, the more time it can spend solving the actual problem.


A Practical AI-Assisted Development Workflow

Here is a workflow I would recommend for building serious software with AI.

Step 1: Describe the product

Write:

What are we building?
Who uses it?
What problem does it solve?
Enter fullscreen mode Exit fullscreen mode

Example:

Multi-tenant project management SaaS for agencies.

Organizations contain users and projects.

Customers subscribe to plans with usage limits.
Enter fullscreen mode Exit fullscreen mode

Step 2: Define expected scale

Do not say:

It should scale.
Enter fullscreen mode Exit fullscreen mode

Write assumptions.

Initial:
1,000 organizations
10,000 users

Expected:
50,000 organizations
500,000 users
50 million tasks

Peak API traffic:
2,000 requests/second
Enter fullscreen mode Exit fullscreen mode

These estimates do not need to be perfect.

They force architectural reasoning.


Step 3: Identify domains

Accounts
Organizations
Projects
Billing
Notifications
Reporting
Enter fullscreen mode Exit fullscreen mode

Define ownership.


Step 4: Establish architectural rules

For example:

Modular monolith.

Modules communicate through public services/events.

Controllers contain no business logic.

Database access remains inside owning modules.

Background work runs through queues.

Application nodes remain stateless.
Enter fullscreen mode Exit fullscreen mode

Step 5: Define critical invariants

Tenant data never crosses tenants.

Payment webhook processing is idempotent.

Usage counters cannot become negative.

Paid invoices cannot be modified.
Enter fullscreen mode Exit fullscreen mode

Step 6: Create AI instructions

Add:

AGENTS.md
Enter fullscreen mode Exit fullscreen mode

or the repository instruction format supported by your coding environment.

Document:

architecture
commands
patterns
testing
security
database conventions
Enter fullscreen mode Exit fullscreen mode

Step 7: Build one module properly

Do not generate the whole product at once.

Implement:

Accounts
Enter fullscreen mode Exit fullscreen mode

carefully.

Use it to establish patterns.


Step 8: Let AI replicate proven patterns

Now tell AI:

Implement Project membership following
the same layering, validation, testing,
and authorization approach used by
Organization membership.
Enter fullscreen mode Exit fullscreen mode

This is where AI becomes extraordinarily productive.


Step 9: Automate verification

CI should enforce:

format
lint
types
architecture
tests
build
security
Enter fullscreen mode Exit fullscreen mode

Step 10: Measure before optimizing

Deploy.

Measure:

latency
throughput
query times
queue depth
cache performance
error rates
resource usage
Enter fullscreen mode Exit fullscreen mode

Then optimize actual bottlenecks.

Do not ask AI to solve hypothetical scale problems before evidence exists.


Example: Building a Scalable Order System With AI

Consider an e-commerce order service.

A simplistic flow:

POST /orders
     |
     ▼
Create order
     |
     ▼
Charge card
     |
     ▼
Send email
Enter fullscreen mode Exit fullscreen mode

We can improve it.

Client
  |
  ▼
POST /orders
  |
  ▼
API
  |
  ▼
OrderService
  |
  ├── Validate inventory
  ├── Calculate price
  ├── Create order
  └── Commit transaction
            |
            ▼
       OrderCreated
            |
      ┌─────┼─────────┐
      ▼     ▼         ▼
  Payment  Email   Analytics
   Worker  Worker    Worker
Enter fullscreen mode Exit fullscreen mode

Now specify invariants:

Order creation is idempotent.

Inventory cannot become negative.

Order prices are snapshotted.

Payment failure does not delete the order.

Email failure does not affect payment.

Webhook processing supports duplicate events.
Enter fullscreen mode Exit fullscreen mode

Then instruct AI:

Implement CreateOrder.

Architecture:

- controller handles HTTP only
- business logic belongs in CreateOrderService
- use a transaction for inventory reservation and order persistence
- use idempotency keys
- dispatch OrderCreated after persistence
- payment execution must happen asynchronously
- add concurrency tests around inventory
- add duplicate-request tests
- add structured logging
Enter fullscreen mode Exit fullscreen mode

Now AI is not designing the system.

It is implementing a system you designed.

That distinction is enormous.


What AI Should Do in a Scalable Engineering Team

AI is particularly good at:

Repository exploration

Explain how payments currently work.
Enter fullscreen mode Exit fullscreen mode

Pattern discovery

Find all implementations of idempotency handling.
Enter fullscreen mode Exit fullscreen mode

Code generation

Implement this service following the existing pattern.
Enter fullscreen mode Exit fullscreen mode

Testing

Generate edge-case tests for this state machine.
Enter fullscreen mode Exit fullscreen mode

Refactoring

Move this domain logic from controllers into services
without changing API behavior.
Enter fullscreen mode Exit fullscreen mode

Performance investigation

Analyze these database queries for N+1 patterns.
Enter fullscreen mode Exit fullscreen mode

Documentation

Document the billing workflow based on the implementation.
Enter fullscreen mode Exit fullscreen mode

Migration work

Update all call sites to use the new interface.
Enter fullscreen mode Exit fullscreen mode

Review

Review this diff specifically for concurrency,
authorization, and database performance issues.
Enter fullscreen mode Exit fullscreen mode

These tasks benefit enormously from AI.


What Humans Should Continue Owning

Humans should remain responsible for:

product goals
architecture
trade-offs
domain boundaries
risk tolerance
security model
data ownership
consistency requirements
operational strategy
cost constraints
Enter fullscreen mode Exit fullscreen mode

AI can advise on all of these.

But someone needs to own the decision.


The Biggest Mistake: Optimizing for AI Output Instead of System Quality

AI produces visible progress.

You prompt:

Build notification system
Enter fullscreen mode Exit fullscreen mode

and suddenly:

14 files created
2 migrations
6 services
3 jobs
4 tests
Enter fullscreen mode Exit fullscreen mode

It feels productive.

But lines of code are not progress.

Architecture is not measured by how much software exists.

The relevant questions are:

Does this solve the problem?

Is the boundary correct?

Can this be changed later?

Can it fail safely?

Can it be observed?

Can it scale?

Can another engineer understand it?

Can an AI agent understand it six months later?
Enter fullscreen mode Exit fullscreen mode

Sometimes the best AI-assisted implementation is fifty lines.


A Better Mental Model for AI-Assisted Software Engineering

Traditional development looked roughly like:

Developer
   ↓
Design
   ↓
Code
   ↓
Test
   ↓
Deploy
Enter fullscreen mode Exit fullscreen mode

AI-assisted engineering increasingly looks like:

                    Human Engineer
                         |
             Architecture + Constraints
                         |
                         ▼
                    AI Agents
                         |
             ┌───────────┼────────────┐
             ▼           ▼            ▼
          Coding       Testing      Refactoring
             │           │            │
             └───────────┼────────────┘
                         ▼
                Automated Guardrails
                         |
       ┌─────────────────┼──────────────────┐
       ▼                 ▼                  ▼
    Tests          Static Analysis      Security
       │                 │                  │
       └─────────────────┼──────────────────┘
                         ▼
                    Human Review
                         |
                         ▼
                    Production
                         |
                         ▼
                   Observability
                         |
                         ▼
                    Feedback
Enter fullscreen mode Exit fullscreen mode

That feedback goes back into:

tests
architecture
documentation
agent instructions
Enter fullscreen mode Exit fullscreen mode

The system becomes easier to develop over time.


The Real Scalability Advantage of AI

AI's biggest impact on scalable software may not be code generation.

It may be reducing the cost of maintaining architectural consistency.

Imagine an engineer saying:

Find every endpoint that returns an unpaginated collection.
Enter fullscreen mode Exit fullscreen mode

An agent can search the repository.

Then:

Identify all jobs that perform non-idempotent external API calls.
Enter fullscreen mode Exit fullscreen mode

Then:

Find modules that directly access Billing models.
Enter fullscreen mode Exit fullscreen mode

Then:

Find database queries inside loops.
Enter fullscreen mode Exit fullscreen mode

Then:

Generate regression tests for these cases.
Enter fullscreen mode Exit fullscreen mode

This turns architectural maintenance from an occasional manual audit into something much closer to continuous inspection.

That is extremely powerful.


Five Principles I Would Use on Every AI-Assisted Project

If I had to reduce this entire approach to five rules, they would be these.

1. Architecture before generation

Do not allow implementation speed to replace design.

Understand:

boundaries
data
failure modes
scale
ownership
Enter fullscreen mode Exit fullscreen mode

before generating large amounts of code.

2. Constraints before prompts

Give AI a system of rules.

Do not repeatedly explain your architecture in every conversation.

Encode it in the repository.

3. Patterns before autonomy

Build one good example.

Then let agents repeat the pattern.

Consistency beats creativity in most production codebases.

4. Verification before trust

AI output is untrusted until verified.

Use:

tests
types
static analysis
CI
architecture rules
security checks
Enter fullscreen mode Exit fullscreen mode

5. Measurement before optimization

Do not build Kafka because somebody said you might eventually have millions of users.

Measure the system.

Optimize the bottleneck that actually exists.


Final Thoughts

AI has made writing software dramatically cheaper.

That does not make software engineering less important.

It makes engineering discipline more important.

When generating code required significant human effort, development speed naturally limited how quickly complexity could accumulate.

AI removes part of that limitation.

A team can now produce enormous amounts of code very quickly.

Without strong architecture, that becomes enormous amounts of technical debt very quickly.

The teams that benefit most from AI will therefore not necessarily be those that generate the most code.

They will be the teams that create environments where AI can safely generate code.

They will build clear module boundaries.

They will define invariants.

They will document architectural decisions.

They will design retry-safe operations.

They will automate tests and static analysis.

They will build observable systems.

They will measure real bottlenecks.

They will maintain repositories that both humans and AI agents can understand.

And they will use AI for what it is exceptionally good at: accelerating implementation, exploration, testing, refactoring, investigation, and repetitive engineering work.

The goal is not:

AI writes my software.
Enter fullscreen mode Exit fullscreen mode

A much better goal is:

I design a system where AI can safely help build,
test, evolve, and operate the software.
Enter fullscreen mode Exit fullscreen mode

That is the difference between using AI to generate code and using AI to build scalable software.

Top comments (0)