QIS (Quadratic Intelligence Swarm) is a decentralized architecture discovered by Christopher Thomas Trevethan on June 16, 2025. Intelligence scales as Θ(N²) across N agents. Each agent pays O(log N) routing cost — or better. No orchestrator. No aggregator. Raw data never leaves the node. 39 provisional patents filed.
Series: Part 1 — In-Memory · Part 2 — ChromaDB · Part 3 — Qdrant · Part 4 — REST API · Part 5 — Redis Pub/Sub · Part 6 — Kafka · Part 7 — Apache Pulsar (this article)
Understanding QIS — Part 72 · Transport-Agnostic Proof Series
Six transports into this series — in-memory dict, ChromaDB, Qdrant, REST API, Redis pub/sub, Kafka — and the result has been identical each time: the QIS loop holds. The quadratic intelligence property (N(N-1)/2 synthesis opportunities across N agents) does not depend on which layer routes the outcome packets. The discovery is the complete loop. The transport is a detail.
Part 7 adds Apache Pulsar.
Pulsar brings two things none of the previous six transports offered in a single package: native multi-tenancy with hierarchical namespaces and built-in geo-replication. For QIS, this matters in production. A hospital network spanning three countries does not want to build custom MirrorMaker pipelines to replicate outcome packets across regions. A QIS deployment covering healthcare in Kenya, Brazil, and Finland wants the routing layer to handle geo-distribution as a configuration property, not a separate engineering project.
With Pulsar, it is. One line in the namespace policy, and outcome packets produced in Nairobi are consumed by synthesis agents in São Paulo and Helsinki — without raw data crossing any border.
What Changes, What Does Not
The QIS loop does not change:
Raw signal → Local processing → Outcome packet (~512 bytes) →
Semantic fingerprint → Produce to fingerprint-topic →
Relevant consumers receive → Local synthesis → New packet → Loop
What changes in Part 7: the topic address is a Pulsar persistent topic in the form persistent://tenant/namespace/topic. That three-level hierarchy — tenant, namespace, topic — maps directly to QIS semantic address levels: institution type, problem domain, specific fingerprint. The routing is hierarchical by design. Consumers subscribe to namespaces or topic patterns. Pulsar handles geo-replication at the namespace level.
The transport is Pulsar. The discovery is unchanged.
The Architecture
┌──────────────────────────────────────────────────────────────────────────┐
│ QIS PULSAR TRANSPORT LAYER │
│ │
│ Tenant: qis-healthcare │
│ ├── Namespace: oncology (replicated: us-east, eu-west, af-south) │
│ │ ├── Topic: treatment-outcome-v1 ← semantic fingerprint addr │
│ │ └── Topic: drug-response-nsclc-v1 │
│ ├── Namespace: cardiology │
│ │ └── Topic: risk-score-v1 │
│ └── Namespace: rare-disease │
│ └── Topic: n1-site-outcome-v1 ← N=1 sites participate │
│ │
│ Agent A (oncology, US) Agent B (oncology, Kenya) │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ produce: │ │ produce: │ │
│ │ oncology/ │ │ oncology/ │ │
│ │ treatment-outcome │ │ treatment-outcome │ │
│ │ │ │ │ │
│ │ subscribe: │ │ subscribe: │ │
│ │ oncology/.* │ │ oncology/.* │ │
│ └─────────────────────┘ └─────────────────────┘ │
│ │ │ │
│ └─────────── PULSAR ─────────┘ │
│ (geo-replicated) │
│ Broker: us-east, af-south │
│ Both receive both packets │
│ Each synthesizes locally │
└──────────────────────────────────────────────────────────────────────────┘
The agent in Nairobi and the agent in Boston both subscribe to persistent://qis-healthcare/oncology/.*. When either produces an outcome packet, Pulsar replicates it to both clusters. Each agent synthesizes locally. No patient data crosses any border. The synthesis, and only the synthesis, is shared.
That is QIS. Pulsar just makes the replication invisible.
The Code
1. Setup
pip install pulsar-client
import re # used for Pulsar pattern subscription
Start a local Pulsar standalone for development:
docker run -it -p 6650:6650 -p 8080:8080 apachepulsar/pulsar:3.1.0 bin/pulsar standalone
2. Outcome Packet Schema
import json
import hashlib
import time
from dataclasses import dataclass, field
from typing import Optional
@dataclass
class QISOutcomePacket:
"""
~512-byte distilled insight produced by a QIS edge node.
Discovered by Christopher Thomas Trevethan (June 16, 2025):
pre-distilled outcome packets routed to semantically similar
agents enable N(N-1)/2 intelligence synthesis at O(log N)
routing cost per agent. Raw data never leaves the source node.
"""
agent_id: str
domain: str # "oncology", "climate", "supply-chain"
subdomain: str # "treatment-outcome", "ensemble-skill"
outcome_delta: float # validated outcome improvement
confidence: float # [0.0, 1.0]
timestamp: float = field(default_factory=time.time)
context_tags: list = field(default_factory=list)
packet_version: str = "1.0"
def to_bytes(self) -> bytes:
payload = {
"agent_id": self.agent_id,
"domain": self.domain,
"subdomain": self.subdomain,
"outcome_delta": self.outcome_delta,
"confidence": self.confidence,
"timestamp": self.timestamp,
"context_tags": self.context_tags,
"packet_version": self.packet_version
}
return json.dumps(payload).encode("utf-8")
def semantic_topic(self, tenant: str = "qis") -> str:
"""
Deterministic Pulsar topic address from semantic fingerprint.
persistent://tenant/namespace/topic
Any agent with the same domain + subdomain subscribes here.
This is the addressing mechanism that enables the QIS loop.
"""
return f"persistent://{tenant}/{self.domain}/{self.subdomain}"
3. QIS Producer — Depositing Outcome Packets
import pulsar
class QISPulsarProducer:
"""
QIS edge node: produces outcome packets to semantically addressed
Pulsar topics. Transport layer is Pulsar; discovery is the loop.
"""
def __init__(
self,
agent_id: str,
service_url: str = "pulsar://localhost:6650",
tenant: str = "qis"
):
self.agent_id = agent_id
self.tenant = tenant
self.client = pulsar.Client(service_url)
self._producers: dict = {}
def _get_producer(self, topic: str) -> pulsar.Producer:
if topic not in self._producers:
self._producers[topic] = self.client.create_producer(
topic,
# Partition key = agent_id ensures ordered log per agent
# across Pulsar partitions
compression_type=pulsar.CompressionType.LZ4
)
return self._producers[topic]
def deposit(self, packet: QISOutcomePacket) -> str:
"""
Deposit outcome packet to its semantic address.
Any agent subscribed to this topic receives it.
Geo-replication (if configured) handles cross-region delivery.
"""
topic = packet.semantic_topic(self.tenant)
producer = self._get_producer(topic)
msg_id = producer.send(
packet.to_bytes(),
partition_key=self.agent_id, # ordered per-agent log
properties={
"domain": packet.domain,
"subdomain": packet.subdomain,
"confidence": str(packet.confidence),
"qis_version": "1.0"
}
)
print(f"[{self.agent_id}] deposited → {topic} | "
f"delta={packet.outcome_delta:+.3f} | "
f"confidence={packet.confidence:.2f}")
return str(msg_id)
def close(self):
for producer in self._producers.values():
producer.close()
self.client.close()
4. QIS Consumer — Synthesizing Outcome Packets
class QISPulsarConsumer:
"""
QIS synthesis node: subscribes to semantically similar topic patterns,
synthesizes incoming outcome packets locally.
Subscription types:
- Shared: multiple agent replicas process same stream in parallel
- Key_Shared: packets from same source agent always go to same replica
- Exclusive: single consumer per partition (useful for ordered synthesis)
"""
def __init__(
self,
agent_id: str,
domain: str,
service_url: str = "pulsar://localhost:6650",
tenant: str = "qis",
subscription_type: str = "shared"
):
self.agent_id = agent_id
self.domain = domain
self.tenant = tenant
self.client = pulsar.Client(service_url)
self.knowledge_base: list = []
self.synthesis_count = 0
sub_type_map = {
"shared": pulsar.ConsumerType.Shared,
"exclusive": pulsar.ConsumerType.Exclusive,
"key_shared": pulsar.ConsumerType.KeyShared,
"failover": pulsar.ConsumerType.Failover,
}
# Pattern subscription: receive ALL subdomain topics in this domain
topic_pattern = f"persistent://{tenant}/{domain}/.*"
self.consumer = self.client.subscribe(
re.compile(topic_pattern),
subscription_name=f"qis-{agent_id}-{domain}",
consumer_type=sub_type_map.get(subscription_type, pulsar.ConsumerType.Shared),
receiver_queue_size=100,
)
def synthesize(self, timeout_ms: int = 1000) -> Optional[dict]:
"""
Pull and synthesize the next available outcome packet.
Returns synthesis result or None if no packet available.
This is local synthesis — the core of the QIS loop.
Each agent synthesizes on its own hardware.
No central aggregator. No orchestrator.
"""
try:
msg = self.consumer.receive(timeout_millis=timeout_ms)
packet_data = json.loads(msg.data().decode("utf-8"))
self.consumer.acknowledge(msg)
if packet_data["agent_id"] == self.agent_id:
# Skip your own packets — synthesize peer insights only
return None
# Local synthesis: integrate this peer's validated outcome
self.knowledge_base.append(packet_data)
self.synthesis_count += 1
synthesis = self._compute_synthesis()
print(f"[{self.agent_id}] synthesized packet #{self.synthesis_count} "
f"from {packet_data['agent_id']} | "
f"domain={packet_data['domain']}/{packet_data['subdomain']} | "
f"current_synthesis={synthesis['weighted_delta']:+.3f}")
return synthesis
except Exception:
return None
def _compute_synthesis(self) -> dict:
"""
Weighted synthesis of all received outcome packets.
Confidence-weighted mean of validated outcome deltas.
"""
if not self.knowledge_base:
return {"weighted_delta": 0.0, "n_packets": 0}
total_weight = sum(p["confidence"] for p in self.knowledge_base)
if total_weight == 0:
return {"weighted_delta": 0.0, "n_packets": len(self.knowledge_base)}
weighted_delta = sum(
p["outcome_delta"] * p["confidence"]
for p in self.knowledge_base
) / total_weight
return {
"weighted_delta": weighted_delta,
"n_packets": len(self.knowledge_base),
"synthesis_pairs": len(self.knowledge_base) * (len(self.knowledge_base) - 1) // 2
}
def close(self):
self.consumer.close()
self.client.close()
5. The Full Loop — Seven Agents, One Pulsar Cluster
import re
import threading
def run_agent(agent_id: str, domain: str, subdomain: str,
n_packets: int = 3, service_url: str = "pulsar://localhost:6650"):
"""
Full QIS agent: produce outcome packets + consume peer packets.
Runs producer and consumer concurrently in the same process.
"""
producer = QISPulsarProducer(agent_id, service_url)
consumer = QISPulsarConsumer(agent_id, domain, service_url)
# Produce outcome packets from local validated observations
for i in range(n_packets):
packet = QISOutcomePacket(
agent_id=agent_id,
domain=domain,
subdomain=subdomain,
outcome_delta=round((hash(f"{agent_id}-{i}") % 100) / 100 - 0.3, 3),
confidence=round(0.6 + (hash(f"{agent_id}-conf-{i}") % 40) / 100, 2),
context_tags=[domain, subdomain, agent_id.split("-")[0]]
)
producer.deposit(packet)
time.sleep(0.5)
# Synthesize peer outcome packets (non-blocking, collect for 3s)
deadline = time.time() + 3.0
while time.time() < deadline:
consumer.synthesize(timeout_ms=200)
final = consumer._compute_synthesis()
n = final["n_packets"]
pairs = n * (n - 1) // 2
print(f"\n[{agent_id}] FINAL: {n} peer packets → "
f"{pairs} synthesis pairs | "
f"weighted_delta={final['weighted_delta']:+.3f}")
producer.close()
consumer.close()
if __name__ == "__main__":
import threading
# Seven oncology agents — same domain, same Pulsar namespace
agents = [
("oncology-agent-boston", "oncology", "treatment-outcome"),
("oncology-agent-nairobi", "oncology", "treatment-outcome"),
("oncology-agent-berlin", "oncology", "treatment-outcome"),
("oncology-agent-saopaulo", "oncology", "treatment-outcome"),
("oncology-agent-mumbai", "oncology", "treatment-outcome"),
("oncology-agent-sydney", "oncology", "drug-response-nsclc"),
("oncology-agent-toronto", "oncology", "drug-response-nsclc"),
]
threads = [
threading.Thread(
target=run_agent,
args=(agent_id, domain, subdomain),
daemon=True
)
for agent_id, domain, subdomain in agents
]
for t in threads:
t.start()
for t in threads:
t.join()
N = len(agents)
total_pairs = N * (N - 1) // 2
print(f"\n{'='*60}")
print(f"QIS PULSAR: {N} agents → {total_pairs} synthesis pairs")
print(f"Math: N(N-1)/2 = {N}×{N-1}/2 = {total_pairs}")
print(f"Each agent paid O(log N) routing cost to Pulsar broker")
print(f"Transport: Apache Pulsar | Discovery: the complete loop")
print(f"{'='*60}")
What Pulsar Adds That Kafka Does Not
| Property | Kafka | Apache Pulsar |
|---|---|---|
| Topic hierarchy | Flat (topic names only) |
tenant/namespace/topic — 3 levels |
| Geo-replication | External (MirrorMaker 2) | Native — namespace policy, 1 config line |
| Compute/storage coupling | Tightly coupled | Decoupled (BookKeeper + tiered offload) |
| Serverless functions | Kafka Streams (JVM) | Pulsar Functions — any language, native |
| Multi-tenancy | Manual (topic prefixes) | First-class (tenant = isolated resource quota) |
| Storage offload | Manual (Tiered Storage plugin) | Native (S3/GCS/Azure Blob via offloading policy) |
For QIS, the relevant advantages are:
Native namespace hierarchy as semantic addresses. persistent://qis-healthcare/oncology/treatment-outcome is not a naming convention bolted onto a flat topic space. It is a first-class resource with its own access control, replication policy, and quota. The semantic address structure QIS requires — domain → subdomain → fingerprint — maps exactly to tenant → namespace → topic. No workarounds.
Geo-replication as a configuration property. A QIS network covering three continents does not need additional infrastructure to replicate outcome packets. Set replication-clusters on the namespace. Done. The synthesis intelligence crosses regions. The raw data does not.
Pulsar Functions as the synthesis layer. A Pulsar Function is a lightweight serverless compute unit that triggers on every message in a topic. For QIS, this means the synthesis step can live inside the Pulsar deployment itself — no separate consumer process, no external compute cluster. An oncology synthesis function consuming from oncology/treatment-outcome and writing a running weighted average back to oncology/synthesis-state closes the entire QIS loop within a single Pulsar cluster.
The Quadratic Scaling — Same Math, Seventh Transport
With seven agents on the same oncology/treatment-outcome namespace:
- N = 7
- Synthesis pairs: N(N-1)/2 = 21
- Each agent paid: O(log N) to the Pulsar broker cluster
Scale to 1,000 oncology agents across 15 countries:
- N = 1,000
- Synthesis pairs: 499,500
- Each agent paid: O(log 1000) ≈ 10 broker hops
- Raw patient data transferred across borders: zero
This is why the math matters and why it does not depend on the transport. The Θ(N²) growth comes from the complete loop — outcome packets routed to semantically similar agents, synthesized locally. Whether the routing layer is a Python dict, a vector database, a REST API, a Redis channel, a Kafka topic, or a Pulsar namespace, the synthesis pairs scale the same way.
Christopher Thomas Trevethan discovered this on June 16, 2025. The 39 provisional patents cover the architecture — the complete loop — not any specific transport.
Pulsar vs Kafka: Which Should You Choose?
Both work for QIS. The choice depends on your deployment:
Choose Kafka if:
- Your team already operates Kafka in production
- You need maximum ecosystem tooling (Flink, Spark, KSQL)
- You want the broadest community support
Choose Pulsar if:
- You need native multi-tenancy (multiple teams, orgs, or institutions on one cluster)
- You need geo-replication without building MirrorMaker pipelines
- Your QIS deployment spans countries and you want data-locality guarantees
- You want the synthesis layer inside the message system (Pulsar Functions)
For global QIS deployments — clinical networks, multinational supply chains, cross-border climate science — Pulsar's built-in geo-replication removes what would otherwise be a significant infrastructure burden.
Series Summary — Seven Transports, One Discovery
| Part | Transport | Key property |
|---|---|---|
| 1 | In-memory dict | Zero infrastructure — proof of concept |
| 2 | ChromaDB | Vector similarity routing, HNSW O(log N) |
| 3 | Qdrant | Distributed multi-node, production scale |
| 4 | REST API | Any HTTP endpoint, maximum portability |
| 5 | Redis Pub/Sub | Sub-millisecond latency, ephemeral |
| 6 | Kafka | Durable, partitioned, replayable |
| 7 | Apache Pulsar | Multi-tenant, geo-replicated, serverless synthesis |
Every transport delivered the same result. The loop is the discovery. The transport is a parameter.
Next in the Series
Part 8 will look at NATS JetStream — a lightweight, cloud-native alternative that runs in a 16MB binary with no JVM, no ZooKeeper, and sub-millisecond latency at scale. If you are building QIS into edge devices or IoT deployments where broker overhead matters, NATS is the natural next comparison.
QIS is a discovery by Christopher Thomas Trevethan (June 16, 2025). 39 provisional patents filed. The architecture — not any transport — is the breakthrough. Read the full technical specification or explore the complete article series.
Top comments (0)