Black Shadow Team ©

Posted on Aug 27

Chapter 11 — Complete Technical Implementation

#ai

Chapter 11 — Technical Implementation Frontend/API architecture Orchestrator service Planner Memory service Retrieval service Model router Verification service Event/queue system Logging and monitorin

11.1 Introduction

This chapter converts the ACAI architecture from a conceptual design into a practical software engineering plan.

The objective is not to claim that the complete system already exists. Instead, this chapter specifies how a research team could build a working prototype using currently available software components.

A practical ACAI implementation should be modular:

                ┌───────────────┐
                │   Frontend    │
                └───────┬───────┘
                        │
                        ▼
                ┌───────────────┐
                │  API Gateway  │
                └───────┬───────┘
                        │
                        ▼
                ┌───────────────┐
                │ Orchestrator  │
                └───────┬───────┘
                        │
      ┌─────────────────┼─────────────────┐
      ▼                 ▼                 ▼
  Planner            Memory          Retrieval
      │                 │                 │
      └─────────────────┼─────────────────┘
                        ▼
                ┌───────────────┐
                │ Model Router  │
                └───────┬───────┘
                        │
              ┌─────────┼─────────┐
              ▼         ▼         ▼
            LLM-A     LLM-B     LLM-C
              └─────────┼─────────┘
                        ▼
                ┌───────────────┐
                │ Verification  │
                └───────┬───────┘
                        ▼
                ┌───────────────┐
                │   Response    │
                └───────────────┘

11.2 Recommended Prototype Architecture

A first implementation does not need a huge distributed infrastructure.

A practical prototype could use:

Frontend
↓
Backend API
↓
Orchestrator
├── Planner
├── Memory
├── Retrieval
├── Model Router
└── Verification
↓
Foundation Model

Once this works, individual services can be separated.

11.3 Frontend

The frontend provides the interface through which users interact with ACAI.

A modern web implementation could contain:

/pages
├── Home
├── Chat
├── Projects
├── Memory
├── Documents
└── Settings

The interface should expose only information users actually need.

For example, the user may see:

AI Assistant

[ Enter your request................ ]

[ Send ]

Status:
Planning → Retrieving → Reasoning → Verifying

The internal reasoning implementation should not be confused with exposing private chain-of-thought. A production interface can instead show concise process status, such as "Searching documents" or "Verifying response," without revealing hidden reasoning traces.

11.4 API Gateway

The API Gateway receives requests from the frontend.

Example endpoints:

POST /api/chat
POST /api/upload
GET /api/session
GET /api/memory
POST /api/feedback
GET /api/health

The gateway handles:

Authentication
Request validation
Rate limiting
Session identification
Routing
Error handling
11.5 Orchestrator

The Orchestrator is the central controller.

Its job is to coordinate the other services.

Conceptually:

Request
↓
Intent
↓
Plan
↓
Memory
↓
Retrieval
↓
Model
↓
Verification
↓
Response

The Orchestrator should not contain the implementation details of every subsystem.

Instead:

Orchestrator
│
├── Planner Service
├── Memory Service
├── Retrieval Service
├── Model Service
└── Verification Service

This keeps the architecture maintainable.

11.6 Planner Service

The Planner Service converts a large request into structured tasks.

Example:

Input:
"Build an AI document analysis application."

Output:

Task 1:
Requirement analysis

Task 2:
Document ingestion

Task 3:
Text extraction

Task 4:
Embedding generation

Task 5:
Vector search

Task 6:
LLM integration

Task 7:
Verification

Task 8:
Testing

Each task can contain:

task_id
description
priority
dependencies
status
required_capability
11.7 Task Graph

Instead of storing tasks only as a list, the planner can create a dependency graph.

      Requirement
           │
    ┌──────┴──────┐
    ▼             ▼
Document       Database
Processing         │
    │              │
    └──────┬───────┘
           ▼
      AI Pipeline
           │
           ▼
        Testing
           │
           ▼
      Deployment

This makes dependencies explicit.

11.8 Memory Service

The Memory Service provides persistent and temporary memory.

A practical implementation can separate:

Working Memory
Short-Term Memory
Long-Term Memory
Semantic Memory
Vector Memory

Example API:

POST /memory/store
POST /memory/search
GET /memory/{id}
DELETE /memory/{id}

A memory record could conceptually contain:

memory_id
user_id
session_id
content
embedding
importance
created_at
updated_at
source
11.9 Memory Storage

Different memory types may use different storage technologies.

Example:

Relational Database
│
├── Users
├── Sessions
└── Metadata

Vector Database
│
└── Semantic Memories

Object Storage
│
└── Documents / Files

Cache
│
└── Temporary Session Data

There is no requirement that one database technology must handle every type of information.

11.10 Retrieval Service

The Retrieval Service handles external and internal knowledge.

Pipeline:

Query
↓
Query Expansion
↓
Search
↓
Filtering
↓
Ranking
↓
Context Selection

A hybrid system can combine:

Keyword Search
+
Semantic Search
+
Metadata Filtering

This can be more flexible than relying on only one retrieval method.

11.11 Document Processing

Uploaded documents first pass through an ingestion pipeline.

File Upload
↓
File Validation
↓
Text Extraction
↓
Chunking
↓
Metadata Extraction
↓
Embedding
↓
Vector Storage

For example, a PDF may be converted into text segments before being indexed.

Each segment should retain useful metadata such as:

document_id
page_number
section
source
timestamp
chunk_id

This makes later citation and retrieval easier.

11.12 Context Builder

The Context Builder combines:

User Prompt
+
Relevant Memory
+
Retrieved Knowledge
+
Task Plan
+
Tool Results

into a structured model input.

Conceptually:

┌─────────────────────────┐
│ User Request │
├─────────────────────────┤
│ Relevant Memory │
├─────────────────────────┤
│ Retrieved Evidence │
├─────────────────────────┤
│ Task Requirements │
├─────────────────────────┤
│ Tool Results │
└─────────────────────────┘
│
▼
Model Input
11.13 Model Router

The Model Router determines which model should process a task.

It can evaluate:

Task Type
Model Capability
Context Requirement
Latency Target
Cost Limit
Availability

Example:

Programming
↓
Code-capable model

Long-context analysis
↓
Long-context model

Simple classification
↓
Small model

The router should also support a fallback model.

Primary Model
↓
Failure?
↓
Backup Model
↓
Continue
11.14 Tool Execution Layer

Some tasks require external tools.

Possible tools include:

Calculator
Search
Code Execution
Database Query
File Parser
Image Analysis

The Tool Layer should use explicit permissions.

Model
↓
Tool Request
↓
Permission Check
↓
Tool Execution
↓
Validated Result
↓
Model

The model should not automatically receive unrestricted access to the host system.

11.15 Verification Service

The Verification Service receives the model's draft and performs quality checks.

Draft
↓
Claim Extraction
↓
Evidence Comparison
↓
Consistency Check
↓
Policy / Safety Check
↓
Final Review

Where applicable, verification should use independent evidence rather than simply asking the same generation process whether its own answer is correct.

11.16 Response Service

After verification, the Response Service prepares the user-facing output.

It handles:

Formatting
Citations
Markdown
Error messages
Structured outputs
Streaming

Example:

Internal Result
↓
Response Formatter
↓
User Interface
11.17 Event-Driven Architecture

As the system grows, asynchronous events can reduce coupling.

Example:

Request Received
↓
Event Queue
↓
Planner
↓
Retrieval
↓
Model
↓
Verification

Possible events:

REQUEST_CREATED
PLAN_CREATED
MEMORY_RETRIEVED
DOCUMENT_RETRIEVED
MODEL_COMPLETED
VERIFICATION_COMPLETED
RESPONSE_CREATED

This approach can make distributed deployments easier to scale.

11.18 Caching

Caching can reduce repeated work.

Potential cache targets:

Embeddings
Retrieval results
Model metadata
Frequently accessed documents
Session information

Example:

Request
↓
Cache?
┌┴───────┐
Yes No
│ │
▼ ▼
Return Compute

Caching must account for freshness so that outdated information is not returned when current information is required.

11.19 Observability

A production implementation should provide three major observability capabilities:

Logs

What happened?

Metrics

How often and how quickly did it happen?

Traces

How did a single request travel through the system?

Example:

Request #123

API
↓ 20ms
Planner
↓ 80ms
Memory
↓ 15ms
Retrieval
↓ 220ms
LLM
↓ 1.8s
Verification
↓ 300ms
Response

This makes performance bottlenecks easier to identify.

11.20 Error Handling

Every service should return structured errors.

Example:

{
"error_code": "MODEL_UNAVAILABLE",
"message": "Primary model unavailable",
"retryable": true
}

The Orchestrator can then decide whether to:

Retry
↓
Fallback
↓
Ask User
↓
Return Error

Blindly retrying every failure can increase cost and latency, so retry policies should have limits.

11.21 Prototype Deployment

A beginner-friendly prototype can initially run on one machine:

Frontend
Backend
Database
Vector Store
LLM

After the prototype becomes stable, components can be separated:

Frontend Server
│
▼
API Server
│
├── Planner
├── Memory
├── Retrieval
├── Router
└── Verification
│
▼
Model Server

This staged approach avoids unnecessary infrastructure complexity during early research.

11.22 Implementation Sequence

A practical development sequence is:

Basic Chat API ↓
Foundation Model ↓
Planner ↓
Retrieval ↓
Memory ↓
Verification ↓
Model Router ↓
Tool Layer ↓
Evaluation ↓
Production Hardening

The important engineering principle is to make each stage work before adding the next major subsystem.

11.23 Minimum Technical Prototype

The first demonstrable ACAI prototype could therefore be:

            USER
              │
              ▼
           Web UI
              │
              ▼
           API
              │
              ▼
         Orchestrator
          /    |    \
         /     |     \
    Planner  Memory  Retrieval
         \     |     /
          \    |    /
              ▼
         Model Router
              │
              ▼
            LLM
              │
              ▼
         Verification
              │
              ▼
           Response

This is sufficiently small to build and sufficiently modular to expand into a larger research platform.

11.24 Chapter Summary

This chapter translated the ACAI concept into a practical software architecture. The proposed implementation uses an API Gateway, Orchestrator, Planner, Memory Service, Retrieval Service, Model Router, Tool Layer, Verification Service, and Response Service.

The most important implementation principle is incremental construction. A small working prototype should be created first, measured against a baseline, and then expanded component by component.

That approach makes it possible to determine which ACAI features actually provide measurable value rather than assuming that architectural complexity automatically produces better AI performance.

End of Chapter 11

Stay tuned for Chapter: 12 Complete End-to-End System Architecture.

🚀 Connect with Black Shadow Team Across the Web! 🌐

We are actively sharing our latest cybersecurity research, AI safety insights, ethical hacking content, and tech updates across multiple platforms. Follow and subscribe to stay updated with our official channels:

📝 Articles & Research Papers:

Medium: https://medium.com/@blackshadowteam.net

Substack: https://blackshadowteam.substack.com

Dev.to: https://dev.to/black_shadow_team

HackerNoon: https://hackernoon.com/u/black-shadow-team

Hashnode: https://hashnode.com/@black-shadow-team

Blogspot: https://black-shadow-team.blogspot.com/

💻 Code & Open Source:

GitHub: https://github.com/blackshadowteamnet-netizen

WordPress: https://profiles.wordpress.org/blackshadowteam

📱 Social Media & Updates:

X (Twitter): https://x.com/BlackShadoTeam