DEV Community: Temitope

The Death of the "Black Box": Why the Future of AI is Modular

Temitope — Thu, 07 May 2026 13:57:53 +0000

For the past few years, the narrative around AI has been dominated by the "Black Box"—massive, monolithic models that live in the cloud, gatekept by APIs, and disconnected from our actual workflows.

But as someone building at the intersection of history and engineering, I see a shift happening. We are moving away from the monolith and toward Modularity.

The Shift to Agentic Architecture
The real breakthrough isn’t just a "smarter" LLM; it’s the ability to wrap that intelligence in a framework that can act. Using** p-agent** as an orchestrator allows us to treat the LLM as just one component of a larger machine.

When you pair this with the Model Context Protocol (MCP), you solve the biggest hurdle in AI: Context. Instead of begging a model to "remember" your project structure, you give it a standardized pipe directly to your filesystem or database.

Why This Matters for Developers
Vendor Agility: If a better model comes out tomorrow (like the next iteration of Gemma), a modular stack lets you swap the "brain" without rebuilding your entire toolset.

Privacy by Design: By running orchestration locally, we move from "Trust us with your data" to "We never see your data".

Local-First Engineering: As we've seen with startups like Ex Machina Technologies, the goal is to build systems that work at the speed of local hardware, not the speed of an API queue.

Implementation: The Modular Loop

Here is the "Invisible Logic" of a modular agent. It’s not a single script; it’s a conversation between an orchestrator and its environment.

# The "Modular" approach: Separation of Brain and Body
from p_agent.core import Agent
from p_agent.providers import AnthropicProvider # or Gemma via Ollama

# 1. Define the Brain (Intelligence)
brain = AnthropicProvider(model="claude-3-5-sonnet")

# 2. Define the Body (Tools/Context via MCP)
assistant = Agent(
    name="Architect",
    provider=brain,
    tools=["filesystem-mcp", "postgres-mcp"] # Standardized interfaces
)

# 3. Execution
assistant.run("Audit my local database schema against the latest documentation.")

Final Thoughts: From Tools to Teammates
We are no longer just "using" AI; we are architecting digital colleagues. By leaning into open-source frameworks and standardized protocols, we ensure that the future of AI is transparent, private, and—most importantly—under our control.

This journey from a simple script to a production-ready agent isn't just a technical upgrade; it's a new philosophy of software.

Open-Source Intelligence: Building with Gemma and MCP

Temitope — Thu, 07 May 2026 13:27:40 +0000

Open-source models have reached a pivotal moment. With the release of Gemma, developers are no longer forced to choose between "smart" cloud-based models and "private" local ones. We finally have both.

In this guide, I’ll walk you through setting up a fully local agentic system using Gemma as the brain, p-agent as the orchestrator, and the Model Context Protocol (MCP) to bridge the gap between AI and your local data.

Why Gemma for Agents?
Gemma is built on the same technical foundations as Google’s Gemini models but tailored specifically for the open-source community. For agentic workflows, it offers:

Superior Logic: Its architecture excels at following complex, multi-step instructions—the "bread and butter" of agent planning.
Efficiency: The 9B and 27B variants run comfortably on consumer hardware while outperforming many models twice their size.
Total Privacy: This is the perfect setup for a "Knowledge Brain" where you want to query sensitive local files without your data ever leaving your machine.

The Stack: Local Intelligence in Action
To get this running, we'll use Ollama to serve Gemma locally. This allows p-agent to call it just like a cloud API, but with 100% data sovereignty.

1. Serve Gemma Locally
First, grab Ollama and pull the model:

ollama run gemma2

2. Configure p-agent to use Gemma
In your p-agent setup, we simply point the provider to your local Ollama instance. This gives your agent a private reasoning engine.

from p_agent.core import Agent
from p_agent.providers import OllamaProvider

# Set up the Gemma brain
gemma_provider = OllamaProvider(model="gemma2")

local_agent = Agent(
    name="GemmaAgent",
    instructions="You are a local researcher. Use MCP tools to analyze data.",
    provider=gemma_provider
)

Bridging the Gap with MCP

An agent is only as useful as the tools it can access. By registering a Filesystem MCP server, we allow Gemma to "reach out" and actually read your local documentation or code repositories.

from p_agent.mcp import MCPTool

# Connect Gemma to your local filesystem
fs_tool = MCPTool(server_url="http://localhost:8080")
local_agent.register_tool(fs_tool)

# Execute a complex local task
local_agent.run("Summarize the architectural decisions found in the /docs folder.")

Final Thoughts: What This Means for Us
The ability to run a model with Gemma’s reasoning capabilities on a local machine—orchestrated by a framework like p-agent—is a genuine game-changer. For developers and founders, it offers:

Cost Independence: No more per-token billing for your internal development tools.
Data Sovereignty: Your codebases and strategy documents stay exactly where they belong: on your hardware.
Rapid Prototyping: Build and test agentic loops all day without worrying about API rate limits or latency spikes.

Standardized protocols like MCP, paired with high-capability open models, are the foundation of the next generation of software.

From Prototype to Production: Deploying p-agent Workflows

Temitope — Thu, 07 May 2026 13:04:24 +0000

The journey from a main.py script to a production-ready AI service is often the hardest part of the "agentic" lifecycle. When building for the real world—especially in high-growth tech hubs like Lagos—developers need to balance power with cost-effectiveness and latency.

In this final installment, we’re looking at how to take the p-agent workflows we’ve built and wrap them in a scalable architecture that’s ready for users.

The Deployment Lifecycle
Moving to production means solving for three things: Persistence, Scalability, and Connectivity.

1. Session Persistence
In a local script, your agent's memory disappears when the process ends. In production, you need to maintain state across multiple user interactions. p-agent handles this by allowing you to inject session managers that store conversation history in a database rather than just RAM.

2. Scaling with MCP Microservices
In our previous tutorials, we ran MCP servers locally. For production, you can host your MCP servers as independent microservices.

The Benefit: Your "GitHub MCP" or "Database MCP" can run on a separate container, allowing your main p-agent orchestrator to remain lightweight.

The Standard: Because MCP is a protocol, your p-agent instance can connect to these remote tools over secure transports (like SSE or WebSockets).

3. Optimizing for Latency
Not every task requires GPT-4o. A professional architecture uses a "Routing Model" (like a smaller, faster LLM) to handle simple tool-calling tasks, reserving the "Reasoning Model" for complex problem-solving. This saves both time and API credits.

Implementation: The Production Wrapper
Here is how you might wrap a p-agent workflow into a FastAPI endpoint for deployment:

from fastapi import FastAPI
from p_agent.core import Agent
from p_agent.providers import OpenAIProvider

app = FastAPI()
provider = OpenAIProvider(model="gpt-4o-mini") # Cost-effective for routing

# Initialize a production-ready agent
deploy_agent = Agent(
    name="ProdAssistant",
    instructions="Process user requests efficiently using connected tools.",
    provider=provider
)

@app.post("/chat")
async def chat_endpoint(user_input: str, session_id: str):
    # Retrieve session context and run the agent
    response = deploy_agent.run(user_input, session_id=session_id)
    return {"reply": response.content}

Foundational Future: Building Locally, Scaling Globally
Frameworks like p-agent are lowering the barrier to entry for AI startups. Whether you are building the next big thing at a Lagos-based firm like Ex Machina Technologies or optimizing internal workflows for a global team, the focus remains the same: building modular, open-source-first systems.

By using p-agent and MCP, you aren't just building a feature; you are architecting a system that can evolve with the AI landscape.

What’s your biggest challenge when moving AI to production? Let’s troubleshoot in the comments!

Temitope Ajao, AI Engineering professional based in Lagos; founder of Ex Machina Technologies.

Building an LLM-as-a-Judge Evaluation Pipeline for Translation Quality

Temitope — Sun, 03 May 2026 10:45:02 +0000

Introduction

How do you know if your AI-generated translation is actually good?

Traditional metrics like BLEU scores measure word overlap — but they miss fluency, context, and cultural nuance entirely. A translation can score well on BLEU and still read like gibberish to a native speaker.

This is where LLM-as-a-Judge comes in — using a large language model to evaluate the quality of another model's output. In this tutorial, we'll build a practical evaluation pipeline that scores translation quality across multiple dimensions using Claude as the judge.

By the end, you'll have a working system you can plug into any translation workflow.
What We're Building
A Python-based evaluation pipeline that:

Accepts a source text + translated output.
Sends both to an LLM judge with a structured scoring prompt.
Returns scores for fluency, accuracy, and cultural appropriateness.
Logs results to a simple JSON file for tracking over time.

Prerequisites

Python 3.9+
An Anthropic API key (or OpenAI)
Basic familiarity with REST APIs and Python

pip install anthropic python-dotenv

Step 1: Set Up Your Project Structure

llm-judge-pipeline/
├── evaluator.py
├── prompts.py
├── logger.py
├── results/
│   └── evaluations.json
└── .env

Create your .env:

ANTHROPIC_API_KEY=your_api_key_here

Step 2: Design Your Evaluation Prompt
The quality of your judge depends almost entirely on your prompt. We want structured, consistent output — so we'll ask the model to respond in JSON.

Prompts.py

JUDGE_PROMPT = """
You are an expert translation evaluator with deep knowledge of linguistics and cultural context.

You will be given:
- SOURCE: The original text
- TRANSLATION: The translated output to evaluate

Evaluate the translation on three dimensions and return ONLY a JSON object:

{{
  "fluency": {{
    "score": <1-10>,
    "reason": "<one sentence explanation>"
  }},
  "accuracy": {{
    "score": <1-10>,
    "reason": "<one sentence explanation>"
  }},
  "cultural_appropriateness": {{
    "score": <1-10>,
    "reason": "<one sentence explanation>"
  }},
  "overall_score": <average of the three scores>,
  "recommendation": "<pass | review | reject>"
}}

SOURCE: {source}
TRANSLATION: {translation}
TARGET LANGUAGE: {target_language}
"""

Step 3: Build the Evaluator
evaluator.py

import os
import json
import anthropic
from dotenv import load_dotenv
from prompts import JUDGE_PROMPT

load_dotenv()

client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

def evaluate_translation(source: str, translation: str, target_language: str) -> dict:
    prompt = JUDGE_PROMPT.format(
        source=source,
        translation=translation,
        target_language=target_language
    )

    message = client.messages.create(
        model="claude-opus-4-5",
        max_tokens=1024,
        messages=[
            {"role": "user", "content": prompt}
        ]
    )

    raw_response = message.content[0].text

    try:
        result = json.loads(raw_response)
    except json.JSONDecodeError:
        import re
        json_match = re.search(r'\{.*\}', raw_response, re.DOTALL)
        if json_match:
            result = json.loads(json_match.group())
        else:
            raise ValueError("Could not parse JSON from model response")

    return result


def batch_evaluate(pairs: list[dict]) -> list[dict]:
    """Evaluate multiple source/translation pairs"""
    results = []
    for pair in pairs:
        result = evaluate_translation(
            source=pair["source"],
            translation=pair["translation"],
            target_language=pair["target_language"]
        )
        result["source"] = pair["source"]
        result["translation"] = pair["translation"]
        results.append(result)
    return results

Step 4: Add a Logger
logger.py

import json
import os
from datetime import datetime

RESULTS_FILE = "results/evaluations.json"

def log_evaluation(evaluation: dict):
    os.makedirs("results", exist_ok=True)

    existing = []
    if os.path.exists(RESULTS_FILE):
        with open(RESULTS_FILE, "r") as f:
            existing = json.load(f)

    evaluation["timestamp"] = datetime.utcnow().isoformat()
    existing.append(evaluation)

    with open(RESULTS_FILE, "w") as f:
        json.dump(existing, f, indent=2)

    print(f"✅ Logged evaluation. Overall score: {evaluation['overall_score']}/10")

Step 5: Run It
main.py

from evaluator import evaluate_translation
from logger import log_evaluation

source = "Please ensure the patient takes the medication twice daily with food."
translation = "Jọwọ rii daju pe alaisan mu oogun naa lẹmeji lojoojumọ pẹlu ounjẹ."
target_language = "Yoruba"

result = evaluate_translation(source, translation, target_language)
log_evaluation(result)

print(json.dumps(result, indent=2))

Sample Output

{
  "fluency": {
    "score": 9,
    "reason": "The translation reads naturally and follows Yoruba grammatical structure."
  },
  "accuracy": {
    "score": 8,
    "reason": "Core meaning is preserved; minor phrasing differences don't affect intent."
  },
  "cultural_appropriateness": {
    "score": 9,
    "reason": "Terminology is appropriate for a Nigerian Yoruba-speaking audience."
  },
  "overall_score": 8.67,
  "recommendation": "pass"
}

Step 6: Scale It with Batch Processing

from evaluator import batch_evaluate
from logger import log_evaluation

pairs = [
    {
        "source": "Welcome to our platform.",
        "translation": "Kaabọ si pẹpẹ wa.",
        "target_language": "Yoruba"
    },
    {
        "source": "Your payment was successful.",
        "translation": "Isanwo rẹ ti ṣaṣeyọri.",
        "target_language": "Yoruba"
    }
]

results = batch_evaluate(pairs)
for result in results:
    log_evaluation(result)

When to Use This Pattern

LLM-as-a-Judge works best when:

You need nuanced evaluation beyond keyword matching
You're working with low-resource languages where reference datasets are scarce
You want explainable scores — not just a number, but a reason
You're building human-in-the-loop review systems

Limitations to Be Aware Of

Cost: Every evaluation is an API call — batch wisely
Judge bias: LLMs have their own language biases; calibrate against human evaluators
Consistency: Add temperature=0 for more deterministic scoring
Self-evaluation: Don't use the same model as judge and translator

What's Next

Add a dashboard to visualize score trends over time.
Integrate with GitHub Actions to auto-evaluate translations in CI/CD
Extend to multi-language pairs with language-specific rubrics
Add human feedback loops to fine-tune your judge prompt over time.

Conclusion
LLM-as-a-Judge is one of the most practical evaluation techniques available today — especially for language tasks where ground truth is hard to define. With just a well-crafted prompt and a structured output format, you can build an evaluation system that catches what traditional metrics miss.
The full code is available on GitHub