DEV Community

Matheus de Camargo Marques
Matheus de Camargo Marques

Posted on

Building a Consensus-Driven Evolutionary Plugin Architecture in Elixir (CEPA)

#elixir #architecture #distributed #phoenix

1. Core Architecture & Architectural Pillars

Software platforms for complex, highly regulated domains (such as legaltech) only scale when code coupling is entirely eliminated in favor of explicit, versioned data contracts backed by evolutionary consensus.

JusrisOS was designed around a fundamental lesson: monolithic code coupling destroys domain extensibility. In complex ecosystems, plugins should never directly invoke other plugins or query each other's database schemas. Instead, our system evolves through Consensus-Driven Evolutionary Plugin Architecture (CEPA) and a strict compile-time enforced layer hierarchy:


[ Pure Primitives (Support) ]
│
▼
[ Microkernel (Kernel) ]
│
▼
[ Domain Plugins (Isolated) ]
│
▼
[ Shell SDK ]
Enter fullscreen mode Exit fullscreen mode

Key Pillars of the Architecture

  1. Consensus-Driven Evolutionary Plugin Architecture (CEPA): No more "move fast and break things." Changes to contracts shared between plugins require explicit versioning and non-breaking additive progression (version + agreement). Each plugin evolves at its own pace without bringing down adjacent services.
  2. Shared-Nothing Microkernel: Dependencies flow strictly in one direction verified at compile time. Plugins are completely isolated in self-contained folders containing their own schemas, migrations, LiveViews, and co-located tests. Deleting the plugin folder removes 100% of the plugin (enabling a fork-per-client / clone-plugin strategy).
  3. Phased Compilation Pipeline: The build separates Core/Shell, compiles plugins in parallel, and finishes with a boundary re-checking phase using manifest files and filesystem partitioning.
  4. Event-Driven Communication + Outbox Pattern: All inter-plugin interaction occurs via versioned events (v1.<plugin>:<event>) with explicit contract declarations. Consumption is driven by local, idempotent projections with snapshot backfills, backed by atomic local writes using an outbox pattern.
  5. True Local-First Engine: Desktop clients operate entirely offline using embedded SQLite for relational queries, Mnesia for fast caching/presence/preferences, and zero-knowledge field/payload encryption (Vault/Cloak) with modern Key Derivation Functions (KDF).
  6. Reconciling Master Node: When network connectivity is present, an upstream Master Node (Canonical Postgres + BEAM Cluster) reconciles local writes via an adaptive backoff sync engine with domain-specific conflict resolution.
  7. Web Shell & Marketplace: The product entry point is a shell (dynamic navigation sidebar, tab system, widgets, user preferences) featuring an AppStore for plugins: catalog, dynamic activation/deactivation with automatic migrations, dependency resolution, and user consent management.

2. CEPA — Consensus-Driven Evolutionary Plugin Architecture

CEPA guarantees that Plugin A never silently breaks Plugin B. When Plugin A needs to evolve a shared contract, it declares a new contract version, maintains backward compatibility for existing published types, and allows Plugin B to migrate on its own schedule.

2.1 The 4 Contract Layers

Layer Contract Mechanism How It Evolves via Consensus
1. PluginBehaviour Kernel ↔ Plugin callbacks verified at compile-time. Additive: New optional callbacks + version bump. Existing callbacks are immutable without a deprecation cycle.
2. Versioned Events Plugin ↔ Plugin (only inter-plugin communication path). Frozen on publish: Payload schema is immutable. Semantic changes require declaring v{n+1}.<plugin>:<event>.
3. Ports (@behaviour) Plugin ↔ Infra (KMS, Storage, Payments, APIs). Additive: New optional callbacks in Ports. Unbounded adapters selected dynamically via config/environment.
4. Context API Plugin ↔ Its own LiveViews & co-located tests. Internal to the plugin: Free to change internally, as long as the co-located context_api.ex mirror and tests match. Never exported or called by other plugins.

2.2 End-to-End Plugin Lifecycle

  1. Scaffolding: Running mix jusris_os.gen.plugin <name> creates a single co-located folder holding entrypoints, schemas/, live/, priv/repo/migrations/, and test/.
  2. Contract Declaration: Inside its init/1 callback, the plugin declares its manifest (slug, dependencies for topological sorting, Mnesia tables) and registers contracts using EventContract.declare/2 and HookRegistry.register/2.
  3. Activation: The AppManager executes migrations, sets up Mnesia tables, invokes init/1, attaches hooks, and flushes pending snapshots via Snapshot.flush_pending/1. Deactivation is blocked while active dependents exist.
  4. Evolution Traffic Light:
    • 🟢 Green (Local Change): Modifying local schemas, UI, or internal functions. Additive column migrations deploy alongside code.
    • 🟡 Yellow (Contract Evolution): Additive contract changes, new event versions (v{n+1}), or optional callbacks. Requires a multi-version compatibility window.
    • 🔴 Red (Breaking Change): Modifying published event payloads, dropping columns currently read, or removing callbacks. Strictly prohibited without formal deprecation cycles.
  5. Retirement: Deactivating via AppStore and deleting the folder completely removes UI, schemas, migrations, and tests (100% cleanup).

2.3 Guided Example: Event + Projection (CRM → Finance)

Below is the canonical Shared-Nothing pattern: consumers maintain their own local projection table fed by producer events.


1. Crm.init/1
├── EventContract.declare("v1.crm:client_registered", required: [:id])
└── Snapshot.register_provider("crm:clients", Crm, "v1.crm:client_registered")
2. Finance.init/1
├── HookRegistry.register("v1.crm:client_registered", Finance)
└── Snapshot.request("crm:clients", Finance)
3. AppManager.bootstrap! ──> Snapshot.flush_pending()
├── Crm.snapshot("crm:clients") fetches existing clients (backfill)
└── Each record triggers an event dispatched directly to Finance
4. Crm.create_person(attrs)
├── Insert into `people` (CRM's local DB table)
└── Event.new("v1.crm:client_registered", payload) |> HookRegistry.dispatch()
5. Finance.handle_event(%Event{type: "v1.crm:client_registered", payload: p})
└── Projection.upsert(local_repo, LocalClient, [crm_client_id: p.id], ...)
└── Writes directly to `fin_clients` (Finance's OWN projection table)
6. Finance LiveViews query `fin_clients`
└── Even if CRM is deactivated, Finance retains its read-model data!
Enter fullscreen mode Exit fullscreen mode

3. Developer Guide & Rules of Consensus

3.1 Repository Map 


lib/jusris_os_core/
├── support/          # Pure primitives (Snowflake IDs, Crypto, Validators)
├── kernel/           # Microkernel (PluginBehaviour, AppManager, Event, Projection)
├── plugins// # Co-located domain plugins (Entrypoint, LiveViews, Schemas, Migrations, Tests)
└── ports/ & adapters/# Infrastructure contracts and unbounded implementations

lib/jusris_os/         # Runtime infrastructure (Sync Engine, Outbox, Mnesia, Vault)
lib/jusris_os_web/     # Shell UI (ShellLive, Dynamic Navigation, Widgets)
Enter fullscreen mode Exit fullscreen mode

3.2 Practical Rules of Consensus

  • Zero Direct Invocations: Never call functions or query schemas belonging to another plugin. Inter-plugin interactions must pass through versioned events and local projections.
  • Snowflake IDs & UPPERCASE Enums: Use Snowflake IDs for all primary keys and events. Ecto enums must be formatted in UPPERCASE.
  • Single-Folder Co-location: Schema, migrations, LiveViews, and unit tests must live within lib/jusris_os_core/plugins/<plugin_name>/.
  • Fail-Closed Commercial Entitlements: Commercial/paid plugins enforce license checks at activation via security manifests. Missing valid entitlements fail closed without degrading runtime stability.

4. Quality Gates & Enforcement

Before submitting any pull request, developers must pass the strict mix precommit verification suite:

mix precommit
# Runs TIA → compile --warnings-as-errors → format → check.cross_schema →
# fitness → credo → sobelow → mock_gate → colocated_test_suite
Enter fullscreen mode Exit fullscreen mode
  • Cross-Schema Gate (check.cross_schema): Verifies that no plugin references an Ecto schema or SQL table owned by another plugin.
  • Decreasing Mock Baseline: Enforces zero regression on domain mocks inside production code paths.
  • Contract Fit Reporter (mix jusris_os.interfaces_check): Validates co-located context_api.ex files against plugin entrypoints.

This article presents the architectural specs and academic foundation behind JusrisOS, an Elixir/BEAM microkernel platform for complex domains.

Top comments (0)