We’ve all been there:
you build an LLM-powered application, and for the first ten minutes, it feels magical. Then a user updates their preferences, changes their target audience, or tweaks a campaign goal, and your agent completely forgets everything it learned two minutes ago. Traditional applications solve this by running relational database migrations and stuffing massive JSON objects into system prompts. But when building BrandPulse—a platform coordinating automated digital out-of-home (DOOH) advertising campaigns—watching our database bloat with brittle state logic became an engineering bottleneck we needed to eliminate.
Instead of writing custom CRUD wrappers around user preferences, I ripped out our session management tables and integrated Hindsight. Below is how we decoupled agent state from rigid database schemas using isolated streaming memory banks.
The Problem: Stateless LLMs and Rigid Databases
When an agent needs to remember operational context—like a brand's specific tone, target regions, or strict safety boundaries—traditional databases force you into painful trade-offs:
The Migration Trap: Every time your agent logic evolves to track a new constraint (e.g., automated safety filtering tags), you have to rewrite SQL schemas and update backend serialization models.
Prompt Inflation: Pulling raw rows into every chat turn or API call eats up token budgets and introduces high-latency context stuffing.
Cross-Talk Hazards: Shared table scopes lead to bleeding user data if tenant isolation isn't enforced cleanly at the query level.
By looking at how agent memory works via Vectorize, I realized that agent state shouldn't be treated like static rows in a ledger. It needs to act as a living stream of retained entities and reflections.
Architectural Blueprint
In our FastAPI service, every brand gets an isolated workspace or "bank" mapped to a deterministic identifier (e.g., brand-dior). Instead of handling manual lookups, incoming profile updates flow directly into the Hindsight engine.
The integration relies on three fundamental primitives:
Retain: Ingests unstructured brand specifications and constraints into the memory store.
Recall: Pulls specific factual history when requested by incoming queries.
Reflect: Synthesizes high-level strategic reasoning and programmatic placement suggestions tailored to the brand's exact profile.
According to the official Hindsight documentation, these memory streams automatically index semantic relationships over time, letting your agent reason over historical preferences rather than forcing keyword matches.
Code-In-Action: The FastAPI Integration
Here is a look at how clean our backend profile ingestion route became once we dropped the old database tables in favor of direct Hindsight calls:
Python
import os
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from hindsight_client import Hindsight
app = FastAPI(title="BrandPulse AI API")
app.add_middleware(
CORSMiddleware,
allow_origins=[""],
allow_credentials=True,
allow_methods=[""],
allow_headers=["*"],
)
HINDSIGHT_BASE_URL = os.environ.get("HINDSIGHT_BASE_URL", "https://api.hindsight.vectorize.io")
HINDSIGHT_API_KEY = os.environ.get("HINDSIGHT_API_KEY", "")
client = Hindsight(base_url=HINDSIGHT_BASE_URL, api_key=HINDSIGHT_API_KEY)
class BrandProfile(BaseModel):
brand_name: str
industry: str
target_audience: str
campaign_goals: str
brand_tone: str
keywords: str
blocked_topics: str
target_locales: str
@app.post("/api/brand/login")
def save_brand_profile(profile: BrandProfile):
bank_id = f"brand-{profile.brand_name.strip().lower().replace(' ', '-')}"
profile_content = (
f"Brand Name: {profile.brand_name}. Industry: {profile.industry}. "
f"Target Audience: {profile.target_audience}. Campaign Goals: {profile.campaign_goals}. "
f"Brand Tone: {profile.brand_tone}. Keywords: {profile.keywords}. "
f"Blocked Safety Topics: {profile.blocked_topics}. Target Locales: {profile.target_locales}."
)
# Push the profile state into Hindsight memory
client.retain(bank_id=bank_id, content=profile_content, context="brand-profile-login")
# Trigger reflection to produce custom programmatic strategies
reflect_res = client.reflect(
bank_id=bank_id,
query=f"Synthesize strategic marketing suggestions and programmatic digital screen placements in {profile.target_locales} matching the tone '{profile.brand_tone}' and goals '{profile.campaign_goals}'."
)
return {
"status": "success",
"bank_id": bank_id,
"strategic_recommendation": getattr(reflect_res, 'text', str(reflect_res))
}
Before & After Comparison
Before (Relational CRUD Tables):
Adding a new preference filter meant writing migration scripts, altering ORM models, and manually injecting variables into giant string templates.
Context queries returned rigid, static strings that lacked semantic flexibility when user inputs varied slightly.
After (Streaming Memory Banks):
Profile updates stream straight into Hindsight via client.retain(), instantly indexing new constraints without touching database tables.
Reflection calls dynamically synthesize contextual business strategies, keeping system prompts lean and relevant.
Dead Ends and Hard Lessons
Watch Out for Silent Network Failures: Early in development, unhandled API timeouts during memory calls caused silent application fallbacks. Wrapping calls in explicit error handlers ensured our service surfaces connectivity issues immediately.
Keep Bank IDs Deterministic: Relying on random UUIDs for memory scopes made debugging messy. Switching to predictable strings like brand-{name} kept our multi-tenant logs clean and isolated.
Reflect, Don't Just Search: Relying purely on keyword search gave us flat historical snippets. Using the reflection endpoint allowed the agent to synthesize actual strategic output from raw memory streams.
https://github.com/vectorize-io/hindsight
https://hindsight.vectorize.io/
Top comments (0)