DEV Community: Raicho Minev

Is an AI Product Still the Same After Its AI Layer Changes?

Raicho Minev — Mon, 31 Aug 2026 20:00:57 +0000

An AI-augmented product, released — same version as three months ago. But is it really same?

Since then, the frontier model behind it got migrated, the RAG was refreshed 50 times, the system prompt "improved", a tool permission widened to close a support issue. And so on. All of it silently changes behavior — sometimes noticeably, sometimes not (until too late?). Was any of it re-checked? You bet - most of the time it wasn't. Does it still work as intended?

The AI scene moves in months, not the yearly cycles the industry was used to. The leading model companies are hungry for success — and still count as startups.

So you put cutting-edge AI capabilities into your product? You may have silently introduced risk.

Now, management standards/regulations do exist — but they predate the RAG-and-agents era, target only the risky use cases, etc. Nothing stable invented yet for day-to-day AI change management. I'm used to dynamic environments, but this one makes me a bit uncomfortable. Thoughts?

Beyond the AI Second Opinion

Raicho Minev — Mon, 24 Aug 2026 21:08:18 +0000

Problem-solving by combining frontier models from different providers.

Modern agent workflows increasingly move work between models and providers - a session may begin with one model, hand a task to another and fan out across several agents. Graph engineering rocks by cleverly applying divide anq conquer tactics and for most task this is perfectly enough. However, decision-making is capped by the model's own thinking capability.

For some difficult, vague or simply open-research problems with no existing solution this cap begins to matter more than the multi-agent brute force approach. Combining the power of two and more frontier models becomes a natural way to gain added value, and it appears when differently trained models form their own readings of the same problem. Their disagreement may come from unique evidence interpretation, might be optimized for different risks or quietly answer different versions of the same question.

What happens when you do that manually?

When a reviewer sees the existing chat, it also acquires the way the first model concluded the problem. It inherits its terminology, its selection of evidence and much of its sense of what deserves attention. The reviewer may criticize the reasoning carefully while remaining inside boundaries chosen by the first model. Moreover, cutting the summary does not help, data is already silently imbued within the text.

This does not make review ineffective. It is the right tool when a concrete claim must survive pressure. A model can be strongly critical without reconsidering the frame it was given.

The concept

Let the readings form naturally

Beyond pure automation, Concilium skill intervenes one step earlier. The orchestrator does an initial setup and review but presents the problem without revealing its own conclusion. Other models must first decide what they think the problem is, which evidence matters and where the uncertainty lies.

“Independent” is deliberately a limited word here. AI models share parts of the public record and many common reasoning habits. They are not independent in the statistical sense. The practical independence comes from forming a view before seeing another model's answer. Using differently trained model families makes it less likely that all of those first readings will simplify the problem in exactly the same way.

Disagreement is a core value

While voting between models is somewhat useful, disagreements is the silent outline of different thinking, instead of burring it Concilium imposes an investigation on what produced the difference. Is it based on an original actionable idea or the model assumed it without verifying?

The orchestrator ratifies, retries, resolves, but the strongest rejected assumption still belongs in the result, together with the observation that rejected it.

Conclusion

Concilium deliberately moves beyond the second opinion paradigm. The different lineage models are not there merely to approve or reject, but to give the possibility of another route into a hard-to-solve problem.

Concilium is available through the Claude Code plugin marketplace: add raichominev/concilium, then install concilium@raicho-skills. Its source and documentation are public in the Concilium GitHub repository.

Previous: How Concilium Works

Introducing Concilium: A Multi-Model Method for Hard Problems

Raicho Minev — Thu, 20 Aug 2026 16:21:22 +0000

One question, several models

Concilium is a method for bringing several AI model families to one difficult question. Instead of asking one assistant for a confident answer, it gives models different roles and lets them explore, challenge and extend one another's work. The human remains responsible for the question, the evidence and the final judgment.

The model families

The council can include Claude, which usually orchestrates the work and keeps the whole problem in view; Codex, which provides an independent OpenAI perspective and can review claims or run its own investigation; Kimi, an optional Moonshot seat that adds another independently trained lineage; and Grok, an optional xAI seat that brings a fourth route through the problem. Their roles can change—the point is to combine models whose strengths and weaknesses do not completely overlap.

Where it helps

Concilium is useful for scientific and scholarly research, hard technical problems, consequential decisions and open questions where the route to an answer is not yet clear. It can search for new directions, test an existing claim, or keep several interpretations alive until evidence separates them. It is most valuable when a normal search or a single model keeps returning the obvious answer, but the problem deserves a deeper one.

Concilium is available through the Claude Code plugin marketplace: add raichominev/concilium, then install concilium@raicho-skills. Its source, documentation and manual installation instructions are public in the Concilium GitHub repository.