Part 2 of my agent build series
Let me set the scene again.
Part 1 shipped. One tool, one loop, no memory — and I was honest about all three limits in the post. A few days later I ended up reading through Google's writeup on how they built Dev Signal, an internal multi-agent system that goes from "scan Reddit for trending questions" to "draft a technical blog post" to "remember your writing style for next time." Root orchestrator, three specialist agents, and a long-term memory layer sitting underneath all of it.
I closed the tab and immediately started drafting a title in my head: "I gave Lumina memory and a team of agents."
Same instinct as last time. Same problem as last time — I hadn't written a line of it yet.
So instead of the announcement post, here's the actual plan — architecture, code, and the parts of Google's pattern I'm not stealing as-is because their own comment section already poked holes in them.
![Lumina multi-agent architecture: a supervisor node routes to search, verify, and synth agents sharing short-term state, with the synth agent reading and writing long-term memory in a vector store]
Infrastructure and Model Setup
Lumina today is Bun + TypeScript. The multi-agent version is a separate Python service sitting next to it — LangGraph for the graph, LangChain for the memory tools, same OpenRouter model underneath so I'm not paying for two providers.
# lumina_agents/graph.py
from langgraph.graph import StateGraph, END
from langchain_openai import ChatOpenAI
from typing import TypedDict, Annotated
import operator
llm = ChatOpenAI(
model="anthropic/claude-sonnet-4-6",
base_url="https://openrouter.ai/api/v1",
temperature=0,
)
class AgentState(TypedDict):
messages: Annotated[list, operator.add]
query: str
sources: list
verified_claims: list
final_answer: str
AgentState is the short-term working memory every node reads and writes. It's the thing that replaces the ad-hoc object I was passing through agent-runner.ts by hand in the TS version — LangGraph threads it through the graph for me instead.
Memory Ingestion Logic
The goal isn't "store everything Lumina ever sees." It's: capture the handful of things a user actually corrects or repeats — preferred source recency, terse vs. detailed answers — and make those available on the next session without re-asking.
Long-term Memory
A vector store, written to explicitly, never automatically:
# lumina_agents/memory.py
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain_core.tools import tool
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
memory_store = Chroma(
collection_name="lumina_user_memory",
embedding_function=embeddings,
persist_directory="./memory_db",
)
@tool
def save_preference(user_id: str, preference: str) -> str:
"""Save an explicit user preference to long-term memory."""
memory_store.add_texts(
texts=[preference],
metadatas=[{"user_id": user_id}],
)
return f"Saved: {preference}"
@tool
def load_preferences(user_id: str, query: str, k: int = 3) -> list[str]:
"""Retrieve the most relevant stored preferences for this user."""
results = memory_store.similarity_search(
query, k=k, filter={"user_id": user_id}
)
return [r.page_content for r in results]
Two tools, same split Google's pattern uses — one for writing, one for reading. The difference is save_preference only fires when a node explicitly calls it after an unambiguous signal ("always sort by recency"), not automatically at the end of every turn. Given the retrieval-precision problem below, I'd rather have less memory that's trustworthy than more memory I have to second-guess.
Short-term Memory
This is just AgentState. No separate service, no persistence — it resets when the graph run ends:
def verify_node(state: AgentState) -> AgentState:
# reads what search_node already put in state, no DB round-trip
sources = state["sources"]
overlap = check_source_overlap(state["query"], sources)
return {"verified_claims": overlap}
The boundary matters for the same reason it did in Google's writeup: state answers "what happened this run," memory answers "what do I know about this user across every run." Conflating them is how you end up either re-asking the same question every session or leaking one session's context into a completely different conversation.
Specialist 1: Search Agent
This one's not new — it's Lumina's existing web_search tool, ported over as a node instead of the whole loop:
# lumina_agents/agents/search_agent.py
from langchain_core.tools import tool
from lumina_agents.tools.tavily import tavily_search
@tool
def web_search(query: str, depth: str = "basic") -> list[dict]:
"""Search the web and return numbered, deduped results."""
results = tavily_search(query, depth=depth)
return [{"index": i + 1, "title": r["title"], "url": r["url"], "content": r["content"]}
for i, r in enumerate(results)]
def search_node(state: AgentState) -> AgentState:
response = llm.bind_tools([web_search]).invoke(state["messages"])
if response.tool_calls:
results = web_search.invoke(response.tool_calls[0]["args"])
return {"sources": results}
return {}
Same citation-numbering rule I learned the hard way in Part 1 applies here — the index gets assigned the moment a result comes back, not whenever the model gets around to referencing it.
Specialist 2: Verify Agent
New. Lumina v0 trusted whatever came back first. This node checks whether the sources actually agree before anything gets synthesized into an answer:
# lumina_agents/agents/verify_agent.py
def verify_node(state: AgentState) -> AgentState:
sources = state["sources"]
prompt = f"""Given these sources: {sources}
Identify which claims are supported by 2+ independent sources.
Flag any claim supported by only one source as UNVERIFIED."""
response = llm.invoke(prompt)
return {"verified_claims": response.content}
Deliberately dumb prompt, on purpose — the point of this node existing is that "did I check" becomes a visible step in the graph instead of an implicit assumption baked into the synthesis prompt.
Specialist 3: Synth Agent
Writes the final answer, and is the only node that reads from long-term memory — because tone and format preferences only matter at the point where you're producing output, not while you're gathering it:
# lumina_agents/agents/synth_agent.py
def synth_node(state: AgentState, user_id: str) -> AgentState:
prefs = load_preferences.invoke({"user_id": user_id, "query": state["query"]})
prompt = f"""Verified findings: {state['verified_claims']}
User preferences on record: {prefs}
Write the final answer, following those preferences where they apply."""
response = llm.invoke(prompt)
return {"final_answer": response.content}
The Root Orchestrator
The supervisor is the piece I'm most cautious about. An LLM deciding "route this to search vs. verify vs. synth" is classification with a probabilistic model attached — it will misroute sometimes, and that's not a bug to patch, it's a property to design around. So the routing function checks explicit signal first and only falls back to the model for genuinely ambiguous cases:
# lumina_agents/graph.py (continued)
def route(state: AgentState) -> str:
if not state.get("sources"):
return "search"
if not state.get("verified_claims"):
return "verify"
if state.get("verified_claims") and not state.get("final_answer"):
return "synth"
return END
graph = StateGraph(AgentState)
graph.add_node("search", search_node)
graph.add_node("verify", verify_node)
graph.add_node("synth", synth_node)
graph.set_entry_point("search")
graph.add_conditional_edges("search", route)
graph.add_conditional_edges("verify", route)
graph.add_conditional_edges("synth", route)
app = graph.compile()
No LLM call in route() at all, for now. State shape decides the next node deterministically. I'll only reach for LLM-based routing if a real case shows up that state shape can't disambiguate — not by default, because it's less code to write upfront.
What I'm Not Copying As-Is
Google's own comment section did the work of stress-testing this pattern before I had to, and two points are worth taking seriously instead of shipping the architecture and hoping:
-
Embedding similarity isn't a precision instrument. One commenter had actually measured it — semantic search struggled to cleanly separate stylistic preferences that were near-opposite of each other. A
load_preferencescall can hand back the wrong preference with a confident-looking score. Nothing pulled from memory changes the final answer's substance without at least a recency check behind it. -
Unstructured content flowing into long-term memory is a prompt-injection surface. If
save_preferenceever gets called on text the model read off the open web instead of something the user explicitly typed, that's an open door. For now, only user-authored turns can trigger a memory write — search results never do.
Summary
Three specialists — search, verify, synth — routed by explicit state instead of LLM judgment, with a short-term AgentState for in-run handoffs and a long-term vector store for anything a user explicitly wants remembered across sessions. Not the full Dev Signal pattern, and not meant to be — the parts I kept are the ones that held up under scrutiny; the parts I changed are the ones that didn't.
Code's still at github.com/Saurabhsing21/Lumina — the memory branch goes up once this is running end-to-end, not before.
If you've run embedding-based memory retrieval in production and it held up better than I'm expecting, or you've got a cleaner way to handle explicit-vs-LLM routing — tell me in the comments. I check daily.
#ai #agents #opensource #llm #langgraph #buildinpublic
Top comments (0)