DEV Community: Atanas Stoyanov

7 Ensemble AI Patterns for Reliable LLM Systems

Atanas Stoyanov — Sat, 06 Dec 2025 17:20:47 +0000

Beyond Prompt Engineering: 7 Ensemble AI Patterns for Reliable LLM Systems

_Forget "one prompt to rule them all." The future of robust AI applications lies in multi-model orchestration and ensemble learning.
_

As developers, we've spent the last two years mastering the art of the perfect prompt. We optimize context windows, chain-of-thought instructions, and few-shot examples. But there's a hard ceiling to what a single model inference can achieve. Even GPT-4o or Claude 3.5 Sonnet have blind spots, hallucinate, or get stuck in local optima.

In traditional Machine Learning, we solved this decades ago with Ensemble Learning (Random Forests, Gradient Boosting). We didn't try to make one perfect decision tree; we trained many and averaged their wisdom.

Why aren't we doing this for LLMs?

At AI Crucible, we've formalized 7 architectural patterns for Ensemble AI that treat models not as oracles, but as components in a larger intelligent system.

Here is a technical breakdown of how these strategies work and how you can think about them from a systems engineering perspective.

The Core Concept: Orchestration over Generation

The shift from "Prompt Engineering" to "AI Orchestration" means moving logic out of the prompt and into the control flow. Instead of Input -> Model -> Output, we design systems like Input -> Strategy Layer -> [Model A, Model B, Model C] -> Synthesis Layer -> Output.

This improves reliability (AEO optimization), reduces hallucination rates, and allows for specialized sub-tasks.

Pattern 1: Competitive Refinement (Iterative Improvement)

The Pattern: Parallel Execution -> Cross-Review -> Self-Correction.

In this architecture, multiple models (e.g., GPT-4 and Claude 3) generate independent candidate solutions. Then, distinct "Reviewer" steps allow models to critique peer outputs. Finally, models regenerate their answers incorporating the valid feedback.

Developer Analogy: Like a Git Pull Request process where code is reviewed and refined before merging.
Best For: Code generation, refactoring, and creative optimization.

Pattern 2: Collaborative Synthesis (Map-Reduce)

The Pattern: Parallel Generation (Map) -> Aggregation (Reduce).

We fan-out the prompt to $N$ diverse models to get a wide distribution of perspectives (improving recall). Then, a dedicated "Synthesizer" model acts as a reducer, taking the array of outputs and merging them into a single, comprehensive document (improving precision).

Developer Analogy: A Map-Reduce job or scattering microservices to gather data and an API gateway determining the final response.
Best For: Technical research, requirements gathering, and system design docs.

Pattern 3: The Expert Panel (Role-Based Routing)

The Pattern: Static Analysis -> Role Assignment -> Multi-Agent Discussion.

Instead of a generic system prompt, we instantiate specific "Agents" with narrow domain contexts (e.g., "Security Engineer," "DBA," "Frontend Architect"). A moderator loop passes state between these agents to identify gaps in the proposed solution.

Developer Analogy: Microservices architecture where each service owns a specific domain context.
Best For: Architecture reviews and complex system design.

Pattern 4: The Debate Tournament (Adversarial Validation)

The Pattern: Proposition -> Adversarial Loops -> Judicial Evaluation.

This pattern specifically targets confirmation bias. We force models into "Pro" and "Con" states. The "Pro" model generates a solution, while the "Con" model specifically parses the output for logical fallacies or factual errors. A "Judge" model evaluates purely based on the strength of the arguments, not the priors.

Developer Analogy: Blue/Green deployment testing or Chaos Engineering (intentionally trying to break the system).
Best For: Critical decision making and validating "too good to be true" solutions.

Pattern 5: Hierarchical Planning (Recursive Decomposition)

The Pattern: Planner -> Worker -> Verifier (Tree of Thought).

A top-level "Strategist" model decomposes a complex task into a dependency graph (DAG). "Implementer" models execute the leaf nodes. "Reviewer" models validate per-node success before bubbling up.

Developer Analogy: A CI/CD pipeline with distinct build, test, and deploy stages.
Best For: Full-stack feature implementation and complex project planning.

Pattern 6: Chain-of-Thought Verification (Test-Driven Inference)

The Pattern: Step-by-Step Inference -> Logic Unit Testing.

Standard Chain-of-Thought (CoT) is just a prompt trick. In the ensemble version, we break the CoT into discrete steps and have secondary models "unit test" each logical leap. If step $N$ fails verification, we branch and retry before proceeding to $N+1$.

Developer Analogy: Unit Testing individual functions rather than just end-to-end testing the whole app.
Best For: Algorithmic logic, debugging data pipelines, and mathematical proofs.

Pattern 7: Red Team / Blue Team (Security Hardening)

The Pattern: Generator -> Attacker -> Hardener.

Similar to GANs (Generative Adversarial Networks), the Blue Team proposes a functional implementation. The Red Team explicitly attempts to find injections, race conditions, or logic gaps. The Blue Team must then patch the specific vulnerabilities found.

Developer Analogy: Automated Penetration Testing in your build pipeline.
Best For: Writing security rules, auth flows, and critical infrastructure code.

Implementing This Today

Building these harnesses from scratch requires managing substantial async complexity, state management, and retry logic.

AI Crucible abstracts these patterns into a clean API and UI. You select the strategy ("The Ring"), choose your model composition (mixing specific versions of GPT, Claude, etc.), and the platform handles the orchestration, state passing, and convergence detection.

By moving from single-prompting to Ensemble AI, we stop treating LLMs as magic boxes and start treating them as deterministic components in a reliable software architecture.

Read the full deep dive on these strategies: Seven Rings of Power: Ensemble AI Strategies Explained

Custom ESM pages in component-controls

Atanas Stoyanov — Wed, 30 Dec 2020 06:08:43 +0000

Originally published on https://component-controls.com/blogs/custom-esm-pages

You can customize component-controls in many ways, in this article we will show how you can add custom page tabs to your ESM documentation pages.

The ESM page tabs are customizable by page type - where the default page type for your ESM files is story.

By default, component-controls uses a page template (ClassicPage) that contains a good mixture of blocks to document your stories.

Example

You can find the source for the example custom esm pages

Live gatsby site

Live nextjs site

Adding new tabs

The page tabs can be customized in the buildtime configuration file of your project.

The following example will use two ESM page tabs - the default ClassicPage and a new TestingPage - this will be for all pages of type story

module.exports = {
  ...
  pages: {
    story: {
      tabs: {
        page: '@component-controls/pages/ClassicPage',
        test: '@component-controls/pages/TestingPage',
      },
    },
  },
};

Customize installed pages

You can customize various page props if you configure the page tab as an array. The following example uses a custom title, as well as empty 'containers' for the page:

module.exports = {
  ...
  pages: {
    story: {
      tabs: {
        canvas: [
          "@component-controls/pages/CanvasPage",
          { title: "Render", container: null, variant: "" },
        ],
        page: '@component-controls/pages/ClassicPage',
      },
    },
  },
};

Create a custom page

You can also create custom pages and add them to a tab on the ESM documentation pages.

The following example creates a custom page that renders the current story in all the available themes. This can be useful to quickly preview any coloring/spacing issues for your components that are specific to a theme.

import React from 'react';
import { TabConfiguration } from '@component-controls/core';
import {
Story,
Description,
} from '@component-controls/blocks';
import { ThemeProvider } from 'theme-ui';
import { BlockContainer } from '@component-controls/components';
import { useThemes } from '../src/components/useThemes';

const ThemesPage = () => {
  const themes = useThemes();
  return (
    <>
      <Description />
        {themes.map(({ name, theme }) => (
          <BlockContainer key={`themed_component_${name}`} title={name} id={name} sx={{ mt: 0 }}>
            <ThemeProvider  theme={theme}>
              <Story id="."sx={{ mb: 0, ...theme?.styles?.root }}/>
            </ThemeProvider>
          </BlockContainer>
        ))}
    </>
  );
}

export default {
  title: 'Themes',
  component: ThemesPage
} as TabConfiguration

and to add this page to your configuration

const { defaultBuildConfig } = require('@component-controls/core');

module.exports = {
  ...
  pages: {
    story: {
      tabs: {
        ...defaultBuildConfig.pages.story.tabs,
        test: '@component-controls/pages/TestingPage',
        themes: require.resolve('./ThemesPage.tsx'),
      },
    },
  },
};

The full sample project, documenting theme-ui is available for inspiration.

source code

live site

Migrate documentation sites from docz

Atanas Stoyanov — Wed, 09 Sep 2020 20:25:50 +0000

docz is an easy to use, zero-config documentation system using Gatsby.

component-controls provides migration compatibility for many of the documentation features in docz. And component-control can also use gatsby as a static site generator.

A migration example can be found here:

source code

live site

The sample project was bootstraped from gatsby-starter-docz-netlifycms

Add gatsby theme

First we will add gatsby and the component-controls gatsby-theme-stories to the project

yarn add gatsby @component-controls/gatsby-theme-stories

Configuration file path

By default, the docz configuration file is kept in the project main folder, while component-controls uses a .config sub-folder. We will configure the gatsby stories theme in gatsby-config.js.

module.exports = {
  plugins: [
    {
      resolve: '@component-controls/gatsby-theme-stories',
      options: {
        //doczrc.js file located in project main folder
        configPath: '.',
      },
    },
  ],
};

Launch scripts

Next, you should add the gatsby develop and gatsby build scripts to the project's package.json

  "scripts": {
    //component-controls scripts
    "build-docs": "gatsby clean && gatsby build",
    "start": "gatsby develop -p 9022",
    //docz original scripts
    "develop": "docz dev",
    "build": "docz build --dest public",
    "serve": "docz serve"
  },

Configuration fields

component-controls can read directly the docz configuration file, the following is an example doczrc.js:

the component-controls build-time and run-time configurations are merged, and we can use a single configuration file (doczrc.js) for both build-time and run-time configurations.

export default {
  files: 'docs/**/*.mdx',
  title: 'Docz & Netlify CMS',
  menu: [
    {
      name: 'Quick Start',
      menu: [''],
    },
    {
      name: 'Getting Started',
      menu: ['Manual Installation'],
    },
    {
      name: 'Configuration',
    },
  ],
};

files field is mapped to the stories configuration field.
title field is mapped to the siteTitle configuration field.
description field is mapped to the siteDescription configuration field.
menu field is used in component-controls in a similar way to docz. You will need to fill the menu field in a document to attach it to a menu item.

Components

Playground

component-controls provides a similar component to docz's Playground. The Playground component can be imported from @component-controls/blocks.

---
name: Playground
menu: Components
---

import { Playground } from '@component-controls/blocks';
import { Button } from '../src/components/Button'

# Button

<Playground>
  <Button kind="secondary">Click me</Button>
</Playground>

You can also wrap the component code in a Story component to display the source code.

---
name: Playground
menu: Components
---

import { Playground, Story } from '@component-controls/blocks';
import { Button } from '../src/components/Button'

# Button

<Playground>
  <Story name='example'>
    <Button kind="secondary">Click me</Button>
  </Story>  
</Playground>

Props

component-controls provides a similar component to docz's Props. The Props/PropsTable component can be imported from @component-controls/blocks.

---
name: Props
menu: Components
---

import { Props } from '@component-controls/blocks';
import { Button } from '../src/components/Button'

# Button

<Props of={Button} />