I have been wondering whether the economics of software architecture are changing as agents write more of our code. If an agent does most of the implementation, does the structure we put in place still save enough work later to justify its cost?
As a principal engineer, I started with a fairly strong expectation. Clear boundaries, architectural guidance and tests should help an agent make changes without having to understand everything at once. I expected that investment to become more useful as a project grew.
So I built an experiment around two versions of the same service. I started with feature additions, tried harder challenges, grew both repositories, and finally tested a change specifically chosen to benefit hexagonal architecture.
This follows my first post, but I will go through the setup and the results here so you do not need to read that first.
The experiment and the setup
The application is ChargeLedger, a Java service for an EV charging network. It turns charging sessions into invoices, with energy charges, idle charges, tax and stored totals. Over the experiment it gained effective-dated tariffs, refunds, tenant isolation and other billing behavior.
The rules matter. Old invoices must keep their original prices. Retrying an operation must not refund twice. Financial changes and their audit records must stay consistent.
I prepared two versions with the same baseline business requirements:
| Version | Starting approach |
|---|---|
| Hexagonal | Separate domain and application code, ports and adapters, architectural guidance, and architecture tests |
| Flat | The same functional requirements, with no prescribed architecture |
“Flat” is the experiment's label. I did not ask the model to write bad code or put everything in one file. It made its own choices, including shared pricing and persistence code. Both versions had tests.
The stack started with Java 21, Javalin, SQLite and Gradle. A local Qwen model ran on my DGX Spark through vLLM. OpenCode ran on my Mac. ChatGPT/Codex helped prepare requirements, build the external evaluator and orchestrate the runs. The local model implemented the measured application changes. The original hexagonal baseline received both local-model and OpenAI assistance during preparation; that preparation was outside the measured time.
Each task had a written contract and external validation. The agent worked in a fresh session with the current repository. After implementation, the controller ran the project's tests and the external checks. A failure went back to the agent for repair within a fixed budget. Only one participant ran at a time, and each accepted feature got a commit.
The main measure is time to acceptance: agent work, project checks, external checks and repairs. It excludes initial project preparation and controller preparation before dispatch. Both sides of a task had the same functional contract and budget, but their architecture guidance and internal test suites differed. This compares two complete setups, not architecture as an isolated variable.
I also recorded input and output tokens, model steps, tool calls and controller-requested repairs. Input includes context sent repeatedly across model steps; it is not unique source code, GPU work or a bill. Output and reasoning are separate recorded fields. A controller repair is a feedback round after validation, not every correction the agent makes while working.
First, nine cumulative features: F1–F9
The first sequence began with CSV export and an offline invoice CLI. It then changed idle billing while preserving historical invoices, added issuance auditing and rollback behavior, and introduced effective-dated tariffs, partial refunds and tenant routing.
Each feature built on the accepted version of the previous one. I expected the architectural separation to help as more rules and entry points were added.
| F1–F9 total | Flat | Hexagonal | Hexagonal compared with flat |
|---|---|---|---|
| Time to acceptance | 165.93 min | 228.57 min | 37.8% more |
| Input tokens | 31.25 million | 53.40 million | 70.9% more |
| Output tokens | 164,177 | 217,296 | 32.4% more |
| Reasoning tokens | 154,599 | 236,416 | 52.9% more |
| Model steps | 426 | 612 | 43.7% more |
| Tool calls | 520 | 749 | 44.0% more |
| Controller-requested repairs | 2 | 2 | Same |
Both implementations passed all nine tasks. Hexagonal took longer overall and logged more tokens, steps and tool calls. The original architectural preparation was not included, so these figures describe subsequent work rather than the full investment.
Then, six harder challenges: C1–C6
My next question was whether the tasks had been demanding enough. I added six challenges:
| Challenge | Requirement |
|---|---|
| C1 | Calculate a quote without persisting an invoice |
| C2 | Void a refund while preserving its history |
| C3 | Import charging sessions from NDJSON through the CLI |
| C4 | Create and deliver transactional outbox events |
| C5 | Reconcile daily tenant balances |
| C6 | Paginate invoices with a tenant-bound keyset cursor |
These were independent branches. Each challenge started from its condition's accepted F9 version, not from the previous challenge. They cannot be treated as six more cumulative features.
| C1–C6 total | Flat | Hexagonal | Hexagonal compared with flat |
|---|---|---|---|
| Time to acceptance | 161.55 min | 174.24 min | 7.9% more |
| Input tokens | 33.69 million | 51.23 million | 52.1% more |
| Output tokens | 120,784 | 179,600 | 48.7% more |
| Reasoning tokens | 180,707 | 153,479 | 15.1% less |
| Model steps | 357 | 546 | 52.9% more |
| Tool calls | 442 | 611 | 38.2% more |
| Controller-requested repairs | 0 | 1 | One additional repair |
Both passed all six challenges. Hexagonal was faster on batch import and transactional outbox delivery; flat was faster on the other four. The overall time difference was smaller, and reasoning tokens went in the other direction from input and output.
That was a more mixed result. It did not establish that hexagonal was catching up as the code grew, because these were different tasks on independent branches.
Growing the repositories: S01–S15
A fair criticism of the first results was that the repositories were small. The matched C2 versions had 4,072 production Java lines in flat and 4,939 in hexagonal. A focused service is a relevant setting, but that does not settle an argument about larger systems.
I continued from those accepted C2 checkpoints and added fifteen changes in sequence. The first five added ordinary resource management across HTTP, CLI, validation and persistence. The later tasks crossed those resources and changed earlier requirements.
| Stages | What changed |
|---|---|
| S01–S05 | Customers and contacts, charging sites, chargers, fleet vehicles and versioned billing profiles |
| S06–S07 | Assign customers to draft invoices and freeze billing details when an invoice is issued |
| S08–S10 | Allow draft reassignment with history, transfer chargers between sites, and manage maintenance windows |
| S11–S12 | Import fleet vehicles with per-row results, then add an all-or-nothing import mode |
| S13–S15 | Redact customer data while preserving financial evidence, generate monthly statements, then change how statement addresses are selected |
The changes of direction were intentional. A design has to handle the business changing its mind as well as adding functionality that fits the original plan.
This sequence inherited F9 plus C2, followed by each accepted S task. It did not contain all six earlier challenges.
| S01–S15 total | Flat | Hexagonal | Hexagonal compared with flat |
|---|---|---|---|
| Time to acceptance | 298.86 min | 389.45 min | 30.3% more |
| Input tokens | 83.04 million | 126.86 million | 52.8% more |
| Output tokens | 278,069 | 417,686 | 50.2% more |
| Reasoning tokens | 203,985 | 292,081 | 43.2% more |
| Model steps | 998 | 1,219 | 22.1% more |
| Tool calls | 1,027 | 1,505 | 46.5% more |
| Controller-requested repairs | 1 | 1 | Same |
Both implementations passed all fifteen stages. Hexagonal had small time wins on S08 and S15, but took longer overall.
These points connect tasks of different difficulty with a growing validation suite. They are not a controlled growth curve or evidence of a specific size threshold.
What the repositories looked like by then
By S15, these were substantially larger applications than the C2 starting points.
| Repository measure | Flat at C2 | Hexagonal at C2 | Flat at S15 | Hexagonal at S15 |
|---|---|---|---|---|
| Production Java lines | 4,072 | 4,939 | 11,709 | 15,205 |
| Production Java files | 37 | 85 | 84 | 239 |
| Test Java lines | 4,480 | 5,836 | 10,305 | 13,724 |
| Test Java files | 15 | 34 | 29 | 74 |
These are physical source lines, including comments and blanks. They exclude the external evaluator, dependencies and generated build output.
They were still single services, not large enterprise systems. But they had grown beyond the original few thousand lines and accumulated more behavior. That addressed part of the size criticism without turning line count into a measure of quality or difficulty.
At this point I was surprised. I had expected hexagonal to start winning more consistently as the application grew and requirements changed. It had individual wins, but the larger cumulative sequence still took 30.3% longer. More structure had not produced the general implementation advantage I expected.
So I chose a one-off task designed to give the existing boundaries a strong opportunity to help: replace the persistence backend.
S16: a change chosen to favor hexagonal
Both implementations had to support PostgreSQL across the service while keeping SQLite working. This directly exercises the replaceable persistence boundary that ports and adapters are supposed to provide.
It was more than changing a connection string. PostgreSQL had to preserve the accumulated behavior, survive restarts and pass checks involving separate JVMs, concurrent invoice operations, refunds, tenant isolation and invalid configuration. The old SQLite regression suite still had to pass.
This was not a historical data migration or a zero-downtime rollout. PostgreSQL and the same JDBC dependency were prepared for both projects before timing. Each run had a three-hour budget. Hexagonal ran first, then flat.
| S16 only | Flat | Hexagonal | Hexagonal compared with flat |
|---|---|---|---|
| Time to acceptance | 71.25 min | 38.92 min | 45.4% less |
| Agent work | 63.64 min | 31.10 min | 51.1% less |
| External evaluation | 7.56 min | 7.77 min | 2.8% more |
| Input tokens | 24.53 million | 14.75 million | 39.9% less |
| Output tokens | 55,523 | 32,504 | 41.5% less |
| Reasoning tokens | 61,492 | 23,011 | 62.6% less |
| Model steps | 174 | 120 | 31.0% less |
| Tool calls | 173 | 119 | 31.2% less |
| Controller-requested repairs | 0 | 0 | Same |
Both passed their first controller submission. That does not mean the agents made no mistakes or ran no internal test-and-fix loops.
Here, hexagonal had a clear advantage in time and recorded agent activity. Most of the time difference came from implementation work; external evaluation took roughly the same time on both sides.
What changed in the code
The hexagonal implementation added PostgreSQL repositories and supporting connection, schema and transaction code. Existing domain code, application service, ports, HTTP and CLI adapters, and SQLite adapters stayed unchanged. Among existing production files, only configuration and startup wiring changed.
Flat reused its central repository, introduced database-specific branches and locking behavior, and translated its SQLite schema into PostgreSQL form. Its HTTP and business value classes stayed unchanged too. Both designs accommodated the change, but they did so differently.
| Production code in S16 | Flat | Hexagonal |
|---|---|---|
| New files | 3 | 11 |
| Existing files modified | 4 | 2 |
| Lines added | 390 | 4,527 |
| Lines removed | 83 | 11 |
| Net new lines | 307 | 4,516 |
| Final production lines | 12,016 | 19,721 |
| Final production files | 87 | 250 |
Much of the hexagonal PostgreSQL repository code closely follows the SQLite implementations. The separation kept the existing core stable, but left two sets of persistence logic to maintain.
Flat avoided most of that duplication. It also put more database-specific decisions into an already large repository. Its schema translation uses text substitutions, which makes future schema changes something to review carefully.
The faster clock and the smaller diff each describe part of the result. Neither settles which implementation will be easier to maintain.
The results across the phases
The broad pattern was higher accumulated time and input/output tokens for hexagonal in the feature and challenge phases, followed by a reversal on the adapter task.
Within the cumulative S extension, the adapter win did not recover all the earlier time difference.
| Time accounting | Minutes |
|---|---|
| Extra hexagonal time across S01–S15 | 90.59 |
| Time saved by hexagonal on S16 | 32.33 |
| Extra hexagonal time still remaining | 58.26 |
S16 recovered 35.7% of that gap. Across S01–S16, flat totaled 370.12 minutes and hexagonal 428.37 minutes, or 15.7% more.
This is measured run time, not a complete return-on-investment calculation. Original preparation, future maintenance and human debugging are outside it. One replacement also cannot establish the cost of the next one.
What the evidence can support
This is one service and one run per condition per task. The starting implementations, guidance and internal tests differ. I alternated which version ran first during S01–S15; S16 had a fixed order. Server-wide prefix-cache counters during S16 showed about 97% hits for both intervals, but those counters can include other clients and cannot establish each run's exact benefit.
The first S01 attempt also exposed error-response expectations that the written contract had not specified clearly. I clarified the contract and restarted both sides from their original C2 checkpoints. All S01–S15 figures here use that fresh paired rerun. The invalidated attempts are kept separately.
The results show how these implementations handled these tasks. They do not establish a scale threshold, long-term maintainability or production readiness. Passing the external suite means passing the behavior it covers. This study also did not compare agents with human developers, so it cannot by itself prove that agents changed the economics.
The version 2 technical report and supporting data cover the full experiment described here, including S01–S16, the methods, source checkpoints and limitations.
AI systems assisted with planning, orchestration, analysis, visuals and drafting. The measured application changes were made by the local coding agent.
My opinion: the architectural investment still has value
The following is my interpretation, informed by the results and my experience as a principal engineer.
The adapter task is a concrete example of architectural investment helping. The existing boundary gave the agent a place to implement PostgreSQL without changing the application core. It reached acceptance with less time, fewer tool calls and fewer tokens. That is useful value, even though it had not recovered the earlier time difference.
I no longer expect that same investment to make every kind of change cheaper. For the feature work we tested, the structured version usually took longer. The benefit appeared most clearly when the task matched a boundary the architecture had already established.
But how often do you change your database?
If database replacement is the reason we give for a design, we should ask how often we expect to use that flexibility. Most of the work on a service may be new fields, validation rules, reports and changes to existing behavior. Those changes deserve a place in the calculation too.
A replacement that becomes cheaper does not automatically justify making frequent work more expensive. In this experiment, the adapter task saved about 32 minutes after the preceding fifteen changes had accumulated about 91 extra minutes. That is a tradeoff I want to discuss explicitly when choosing a design.
Database replacement is also only one reason to separate persistence. Testing, ownership, dependency isolation and keeping business decisions understandable can matter even if the database never changes. If those are the benefits we need, we should name them and assess whether the design actually delivers them.
What I am questioning is the habit of accepting the ongoing cost of an architecture without being clear about which of its benefits the project needs.
Project timelines change
There is an equally important problem with planning only for the work in front of us: we are often wrong about how long a project will live.
The proof of concept becomes production. A temporary integration becomes a permanent dependency. A service meant for one customer ends up serving several. By the time a boundary becomes necessary, introducing it may be much harder.
That is why I would not turn these results into advice to remove structure and let the agent do whatever it wants. Someone still has to understand the system, review a financial change or investigate a failed refund during an outage.
My default would be to keep important responsibilities clear and behavior well tested, then add separation where the cost of introducing it later would be high. Pricing rules should have an identifiable home. Transaction ownership should be clear. A second real integration is a stronger reason for an abstraction than a diagram that says every service needs one.
A larger experiment could test that approach directly: a third implementation with a few deliberate boundaries, followed by both routine changes and disruptive ones. We have not established whether that would offer the best balance.
Conclusion: choose the tradeoff, not the diagram
Architectural investment can make a major change easier while adding cost to smaller, more frequent changes. That is the tradeoff this experiment made concrete for me. The existing persistence boundary helped with PostgreSQL; the same overall setup took longer across the preceding feature work.
This does not mean that every big change favors hexagonal or that every small change favors flat. Both implementations completed the adapter task, and some earlier tasks also favored hexagonal. The useful question is whether the design fits the changes and risks of this particular service.
For me, evaluating that case by case means being specific: what are we likely to change, what behavior must remain safe, what does the structure cost during ordinary work, and how difficult would it be to introduce later? We also need to allow for a longer project lifetime than the current plan suggests.
Agents are making writing and shipping code faster in my own work. I believe that makes it more important to revisit the practices we apply by default. Strong behavioral validation and understandable responsibilities remain worth investing in. Additional layers and abstractions should have a reason tied to the system we are building.
I would start with the simplest design that protects those needs, and let further architectural investment earn its place. I would not follow an architecture diagram blindly.
I am hyped to keep exploring this. Which boundary in your own service has earned its cost, and what change made that clear?
Top comments (2)
The useful measure is not total tokens for a single change, but how much irrelevant system context each architecture forces into the agent’s working set. I’d report token and time distributions by change type, plus the number of files and invariants touched. That can show when boundaries actually reduce search and regression risk rather than merely adding ceremony.
Thanks for the feedback, I think also that general token use and time are a meaningful measure as this is what directly impacts cost, but I can do some analysis trying to isolate files and invariants touched, but I think generally this drill down will require semantic analysis more than something hard and reproducible- how would you go about it if you had to design an experiment?