DEV Community: Maciej Pieniak

I Delayed Multi-Model Orchestration. Then the Economics Changed.

Maciej Pieniak — Mon, 13 Jul 2026 17:14:12 +0000

I Delayed Multi-Model Orchestration. Then the Economics Changed.

In my previous article, I described how AI coding agents can generate software quickly while quality is maintained by the process: tests, CI, quality gates, review and human decisions.

At that time, I was still thinking mostly in terms of one provider and one primary coding agent.

I deliberately postponed advanced orchestration. Not because I considered it useless, but because the extra complexity did not yet seem justified. Every additional model means more configuration, more failure modes, more logs and more routing decisions.

For a while, one strong model was easier.

Then the economics changed.

One premium model for every task is wasteful

A coding workflow includes very different kinds of work: reading a repository, planning, implementing changes, generating tests, reviewing diffs, debugging failures and updating documentation.

These tasks do not require the same level of reasoning.

Using the most capable and expensive model for every step is like assigning a senior architect to every minor maintenance task. It works, but it does not scale economically.

The key question stopped being:

Which model is the best?

It became:

Which model is good enough for this step?

My current setup

My workflow is becoming a small multi-model production system:

Codex / OpenAI coordinates work in the repository.
DeepSeek V4 Flash is the default worker for fast and repetitive tasks.
DeepSeek V4 Pro is the escalation path for harder implementation or reasoning.
Gemini 3.1 Pro Preview can provide a second review or another perspective.
OpenRouter provides routing, provider selection, logs and cost telemetry.

This is no longer a single AI assistant. It is closer to a small team of specialized models.

A recent test produced 57 requests, 344K tokens and a total cost of about $0.22. The cache hit rate was only 4.3%, so there is still room for optimization.

Routing matters more than the model list

A multi-model setup can easily become a model zoo. The goal is not to connect as many providers as possible, but to define roles and escalation rules.

Task enters the workflow
        |
        v
Flash model handles the default path
        |
        +--> result passes validation --> continue
        |
        +--> failure, uncertainty or high-risk change
                    |
                    v
             Pro model escalation
                    |
                    +--> optional second review
                    |
                    v
             quality gates and human decision

The cheaper model is not trusted blindly. Tests, static analysis, architecture guards and review determine whether its output can move forward.

This follows the same principle I used earlier:

The agent may generate the change, but the pipeline decides whether the change is acceptable.

Orchestration adds one more rule:

The router decides which agent should attempt the task first.

Why I waited too long

My earlier concern was technical complexity. The new pressure is operational cost.

Agentic workflows are chains of calls. One request may trigger planning, implementation, testing, review, correction and documentation. Even when every call is affordable, the complete workflow can become expensive.

At that scale, routing is no longer a luxury. It becomes part of system design.

The architecture must balance quality, latency, cost, provider availability, context size, retries and auditability.

The best model should still be used—but only where its additional capability creates real value.

From choosing models to managing production

I no longer want to select one model for the entire workflow.

I want to manage a portfolio of models, each with a defined role, budget and escalation path.

The human role changes as well. The work becomes less about prompting one assistant and more about designing the production system: classifying tasks, setting routing rules, controlling budgets, observing failures and deciding when escalation is justified.

That is why orchestration now feels like the natural next step after quality gates and CI.

First, I learned how to control the output of coding agents.

Now I am learning how to control which agent receives the task in the first place.

Final thought

For small experiments, one powerful model is often enough.

For continuous agentic software development, it may be the most expensive possible default.

The future is probably not one best model.

It is a well-managed system in which several models cooperate under clear technical, quality and economic constraints.

How are you approaching this today: one trusted model, manual switching, or automated routing?

Hello DEV Community — I’m Maciej, writing about AI, software processes and digital transformation

Maciej Pieniak — Sun, 05 Jul 2026 11:52:05 +0000

Hi DEV Community,

I’m Maciej Pieniak, a business and systems analyst from Poland, working at the intersection of business processes, software development and digital transformation.

My professional background comes from IT projects in areas such as public sector, logistics, banking, telecom and web systems. Over time, my focus has naturally moved toward AI, automation and the question that I find especially interesting:

How can AI support real work without turning the whole process into a very expensive magic trick?

I write about AI from a practical and analytical perspective — not only as a technology trend, but as a change in how we define requirements, design processes, build software and make decisions.

Recently, I have also been exploring AI agents in software development, including how they affect throughput, code quality, architecture, testing and the role of the human operator in the process.

On DEV, I plan to share notes and articles about:

AI agents and software development workflows
business and systems analysis in the age of AI
process automation and digital transformation
practical lessons from building and reviewing AI-supported solutions
the gap between AI hype and real implementation work

I’m here to learn, exchange ideas and share observations from the perspective of someone who likes asking: “OK, but how does this process actually work?”

Happy to join the community.

introduction

Coding with AI Agents: Between High Throughput and Code Quality

Maciej Pieniak — Sat, 27 Jun 2026 17:28:58 +0000

Recently, I had to present the results of my work in a formal way, so I decided to share the material with a broader group of readers.

Coding with AI agents can move very fast. The challenge is not code generation itself, but maintaining code quality, consistency and control over the process.

Based on one of my projects — Project Venom — I prepared a practical summary showing that high throughput with AI agents can be combined with code quality control.

Sources and supplementary data

Project Venom repository: https://github.com/mpieniak01/Venom
Research summary: https://research.strefa.org/

The project is being developed with the use of AI coding agents, and its final state looked as follows:

technology stack: Python 73.1%, TypeScript 24.9%, Shell 1.0%, Makefile 0.5%, JavaScript 0.3%, CSS 0.2%,
coding supported by AI agents: 138,011 LOC,
repository activity: 1,587 commits,
quality: reduction from 1,650 issues in the first SonarQube measurement to 0 issues,
test coverage: 92.2%.

Conclusion:

AI agents increase throughput, but code quality is maintained by the process: pre-commit, CI, quality gate and review performed by AI agents and a human.

The data shows three phases of work

The quality and code flow chart shows three phases.

Phase I — intensive coding

In the first phase, AI agents generated a large part of the code. Quality validation mainly covered unit tests. This made it possible to quickly expand the scope of the project.

Phase II — raising the quality bar

In the second phase, quality control was tightened. Connecting SonarQube showed 1,650 quality issues. This was the moment of transition from fast generation to controlled code cleanup.

In this phase, local hard pre-commit gates were created. They included:

static code validation,
checking code coverage with tests,
architecture drift control.

The agent could not complete the task without passing the quality gates. Code generated by AI did not go directly into the repository. It first had to pass a local filter.

Phase III — stabilization and quality maintenance

In the third phase, the process was already stabilized by pre-commit, CI, SonarQube / SonarCloud and review performed by AI agents and a human. Quality was not checked only at the end; it was maintained during the work.

The final result: 0 issues and 92.2% test coverage, expressed in more than 5,000 unit tests. At the same time, the testing procedure was not supposed to exceed 10 minutes — as an assumption for a correct test pass.

How the workflow worked

AI agents supported execution: planning the change, implementation, refactoring, generating tests, analyzing errors and updating documentation.

However, they did not take over boundary decisions. The human still defined the goal and scope, accepted the architecture, and decided on merge, deployment and risk.

The key workflow was:

Human defines the goal
   ↓
AI agent helps prepare the plan and implementation
   ↓
Pre-commit checks the change locally
   ↓
GitHub Actions runs CI
   ↓
SonarQube / SonarCloud evaluates quality
   ↓
Pull request goes to review by AI agents and a human
   ↓
Merge

Key points

The first quality gate was local. Pre-commit scripts, prepared with the help of AI agents, ran static code validation, basic quality rules, local tests and an initial coverage check. A commit was possible only after a green result.
After pushing to GitHub, the second control layer started: GitHub Actions and pull request review. The pipeline worked like a set of automated reviewers, supported in pull requests by AI code review tools such as CodeRabbit, Copilot code review and Gemini Code Assist. It included, among others, backend tests, frontend linting, a quick validator, OpenAPI contract checks, architecture guards, forbidden path checks and documentation validation.
The next layer was SonarQube / SonarCloud. Its role was not to create a report after the fact. It was a quality gate: it checked whether new issues appeared, whether technical debt was growing, whether the code was becoming too complex, whether test coverage was falling and whether the change degraded the state of the project.
Why this model worked

The model worked because it combined three elements.

First: the throughput of AI agents.
Agents accelerated the transition from idea to working change. They helped with analysis, implementation, refactoring, tests and documentation.
Second: automated quality control.
Pre-commit, CI, guards, tests and SonarQube created a system of gates. The agent could generate code, but it could not decide on its own that the code was ready.
Third: human decision loops.
The human retained control over the goal, scope, architecture, change acceptance and risk.

In short:

AI generates and proposes.
Pre-commit catches basic errors.
CI checks the change in the repository.
SonarQube evaluates quality.
The human makes the decision.
This is the core of the approach.

What other teams can take from this

For teams starting work with coding agents, the most important question is not: “Which AI agent is the best?”. The word “best” changes from day to day.

A better question is:

How do we prepare a process that can safely accept code generated by AI agents?

A minimal starting set:

GitHub repository,
pull requests,
pre-commit,
unit tests,
GitHub Actions,
SonarQube / SonarCloud,
instructions for agents,
tasks for the agent,
review: AI agents and a human.

AI coding without a pipeline is a fast experiment. AI coding with a pipeline becomes a controlled software development process.

The most important experience from Project Venom:

A properly designed process makes it possible to combine high throughput from AI agents with a real improvement in code quality.

The agent can write code, but the pipeline must decide whether that code moves forward.

Question

How do you organize work with AI coding agents?

Do you treat them mainly as assistants in the IDE, or already as participants in the process with their own instructions, guards and quality gates?

I would be happy to compare approaches.

Disclosure

AI support in the article; presentation graphics. Data cited from the knowledge base.