DEV Community

Cover image for How I Built a Marketing Memory Loop With Hindsight
Nikitha
Nikitha

Posted on

How I Built a Marketing Memory Loop With Hindsight

#ai #hindsight #marketing #agents

A stateless marketing agent is dangerous with capital. When you prompt a standard language model to formulate a multi-channel growth strategy, it will generate budget distributions, draft copy angles, and suggest cost-per-acquisition targets. But the moment that campaign ends, everything learned disappears. If a creative hook fatigued after 18 days, or broad-match search terms burned budget on unqualified leads, a stateless agent will recommend the exact same mistake next month.

I built MarkSight around a simple technical premise: campaigns should never start from zero. Instead of treating strategy formulation as disposable text generation, MarkSight runs an autonomous closed-loop execution pipeline backed by persistent memory. Past campaign outcomes, platform conversion baselines, behavioral skepticism models, and tactical heuristics persist across runs.

Here is how I designed and built this memory loop using Hindsight.

What I Wanted MarkSight to Remember

When human performance marketing leads manage ad spend, their mental model is rarely a blank slate. They rely on four distinct categories of institutional knowledge:

  1. Episodic Campaign Outcomes: Real post-mortems containing spend, realized return on ad spend (ROAS), cost-per-acquisition (CPA), conversion rates, and qualitative summaries of what worked or failed.
  2. Platform Benchmarks: Empirical baselines per vertical and platform (for instance, enterprise B2B lead generation costs on LinkedIn versus high-intent Google Search).
  3. Tactical Heuristics: Operational rules of thumb derived over time, such as minimum liquidity thresholds required for platform bidding algorithms to exit exploration mode.
  4. Audience Behavioral Profiles: Decision-maker psychographics, including common objections, skepticism patterns, and messaging formats that actually resonate.

Ordinary conversational context windows fail here. Shoving dozens of past campaign post-mortems into a system prompt degrades reasoning, wastes token budget, and requires manual curation every time someone starts a new strategy run. What I needed was an external memory layer that can selectively retrieve relevant past experience when a new campaign is initialized, and cleanly retain post-mortem observations once a campaign flight concludes.

This distinction between transient session context and durable recall is central to modern agent memory architectures.

How MarkSight Is Structured

MarkSight is split into a modular full-stack architecture designed for observable agent execution:

  • Frontend (client/): React 19 single-page app with Tailwind CSS v4 housing the Strategy Studio, Execution Visualizer, Memory Bank Explorer, and Flight Simulator.
  • Backend Server (server/src/index.js): Express v5 server on port 5050 hosting REST endpoints and static assets.
  • Memory Engine (server/src/memoryEngine.js): Singleton managing persistence (store.json), seed baselines (defaultMemories.json), hybrid retrieval scoring, and cloud sync.
  • Agent Orchestrator (server/src/marketingAgent.js): Coordinates the 6-stage execution pipeline from input decomposition to memory commit.
  • Streaming Pipeline (server/src/routes/agentRoutes.js): Server-Sent Events (SSE) streaming endpoint (POST /api/agent/run-stream) with synchronous fallback (POST /api/agent/run).

Where Hindsight Fits

In MarkSight, memory operations map directly to the core primitives defined in the Hindsight documentation:

  • Recall: Querying relevant historical context by vertical, channels, audience, and objectives before strategy formulation.
  • Retain: Committing structured post-mortem nodes and performance actuals after a campaign flight concludes.
  • Reflect: Synthesizing high-level heuristics and updating confidence weights across the memory topology based on metric variance.

The environment connects to the Hindsight runtime using HINDSIGHT_API_KEY and HINDSIGHT_API_URL parameters. The local engine maintains an active file store (store.json) for zero-latency local execution while staging documents for upstream Hindsight bank synchronization.

The Memory Loop

The execution lifecycle in MarkSight is explicitly closed-loop, advancing through six stages:

[Stage 1: User Input] 
  ↳ Decompose parameters & evaluate channel liquidity threshold
[Stage 2: Marketing Agent] 
  ↳ Formulate testable hypotheses based on audience psychographics
[Stage 3: Hindsight Memory Retrieval] 
  ↳ 5-dimension hybrid vector & keyword retrieval over memory bank
[Stage 4: Benchmark & Gap Analysis] 
  ↳ Compare proposed goals to historical baselines & extract risk radar
[Stage 5: Strategy Recommendation] 
  ↳ Synthesize budget splits, A/B creative blueprints & anti-pattern guardrails
[Stage 6: Memory Update & Feedback Loop] 
  ↳ Commit hypothesis registry & trigger post-mortem learning on flight completion
Enter fullscreen mode Exit fullscreen mode

When a campaign runs, it queries the memory bank in Stage 3. When the campaign finishes (via actuals logging or the built-in 30-day flight simulator), Stage 6 fires a feedback loop: it calculates variance between predicted and actual metrics, writes an episodic post-mortem node, adjusts heuristic confidence scores, and updates the Collective IQ score.

Code-Backed Explanation

Let us look at how this is implemented in the repository.

1. 5-Dimension Hybrid Retrieval Scoring

Inside server/src/memoryEngine.js, memory retrieval does not rely purely on unconstrained text embeddings. Instead, the queryRelevantMemories method scores candidate memory nodes across five explicit dimensions: vertical match (25%), channel overlap (25%), audience and keyword tokens (25%), objective alignment (15%), and stored confidence score (10%):

// server/src/memoryEngine.js (excerpt from queryRelevantMemories)
const queryTokens = this.tokenize(`${campaignName} ${targetAudience} ${constraints} ${vertical} ${objective}`);

const scored = this.memories.map(node => {
  // 1. Vertical Match (25 pts)
  const verticalScore = (node.vertical?.toLowerCase() === vertical?.toLowerCase()) ? 25 : 5;

  // 2. Channel Overlap (25 pts)
  const intersection = (channels || []).filter(c => 
    (node.channels || []).some(nc => nc.toLowerCase() === c.toLowerCase())
  );
  const channelScore = Math.min(25, Math.round((intersection.length / Math.max(1, channels.length)) * 25));

  // 3. Audience & Keyword Match (25 pts)
  const nodeCorpus = `${node.title || ''} ${node.targetAudience || ''} ${node.learnings || ''}`.toLowerCase();
  const hits = queryTokens.filter(t => nodeCorpus.includes(t)).length;
  const keywordScore = Math.round(Math.min(1, hits / Math.min(queryTokens.length, 6)) * 25);

  const confidenceScore = Math.round((node.confidence || 0.9) * 10);
  const totalScore = verticalScore + channelScore + keywordScore + objectiveScore + confidenceScore;
  return { ...node, relevanceScore: Math.min(100, totalScore) };
});
Enter fullscreen mode Exit fullscreen mode

This ensures that a B2B SaaS campaign targeting LinkedIn and Google Search retrieves memories sharing those operational constraints, rather than generically similar text.

2. The Feedback Loop and Heuristic Weighting

When a campaign outcome is recorded, recordOutcomeAndLearn in server/src/memoryEngine.js evaluates whether the strategy outperformed or lagged expectations. It shifts existing heuristic confidence weights and writes a post-mortem:

// server/src/memoryEngine.js (excerpt from recordOutcomeAndLearn)
const roasVariance = Number((((actualMetrics.roas - predictedMetrics.roas) / Math.max(0.1, predictedMetrics.roas)) * 100).toFixed(2));
const isOutperforming = roasVariance >= 0;

// Shift heuristic confidence weights: +0.02 on positive variance, -0.03 on negative
const confidenceDelta = isOutperforming ? 0.02 : -0.03;
this.memories.forEach(m => {
  if (m.type === 'heuristic') {
    m.confidence = Number(Math.min(0.99, Math.max(0.50, (m.confidence || 0.9) + confidenceDelta)).toFixed(2));
  }
});

// Commit episodic post-mortem node to store.json
this.memories.push({
  id: `mem_ep_flight_${Date.now().toString(36)}`,
  type: 'episodic',
  title: `Post-Mortem: ${campaignTitle} (${roasVariance >= 0 ? '+' : ''}${roasVariance}% ROAS)`,
  vertical,
  channels,
  metrics: { ...actualMetrics },
  confidence: isOutperforming ? 0.96 : 0.88,
  learnings: qualitativeLearnings || `Flight completed with ${roasVariance}% variance.`,
  tags: ['flight-postmortem', 'closed-loop-learning']
});
this.save();
Enter fullscreen mode Exit fullscreen mode

By dynamically adjusting heuristic confidence, rules that consistently yield accurate predictions gain retrieval priority, while rules that lead to negative variance are penalized.

3. Real-Time Pipeline Orchestration

In server/src/marketingAgent.js, the agent links each stage sequentially. Stage 3 feeds Stage 4 and Stage 5 directly:

// server/src/marketingAgent.js (excerpt from executeStage3)
async executeStage3(campaignInput, stage2Output) {
  const retrieval = this.memoryEngine.queryRelevantMemories({
    vertical: campaignInput.vertical,
    channels: campaignInput.channels,
    objective: campaignInput.objective,
    targetAudience: campaignInput.targetAudience
  });

  return {
    stageNumber: 3,
    id: 'stage_3_memory',
    name: 'Hindsight Memory Retrieval',
    status: 'COMPLETED',
    output: {
      episodic: retrieval.episodic.slice(0, 3),
      benchmarks: retrieval.benchmarks.slice(0, 2),
      heuristics: retrieval.heuristics.slice(0, 3)
    }
  };
}
Enter fullscreen mode Exit fullscreen mode

The output of Stage 3 is passed into Stage 4 to construct the risk radar and Stage 5 to assemble creative blueprints and anti-pattern guardrails.

Concrete Campaign Example

To observe the memory loop in action, consider a representative run in the MarkSight interface:

  • Campaign: B2B Cloud Infrastructure Launch ($35,000 budget, 30 days)
  • Vertical & ICP: Developer Tools; Engineering Managers and VP Infrastructure
  • Channels: LinkedIn + Google Search
  • Objective: Qualified Free Trials & Product Signups

What the Memory Layer Contributes:

  1. Liquidity Law Enforcement: With $35,000 across 2 channels ($17,500/channel), heuristic mem_hr_001 validates that spend exceeds the $2,000/month threshold required to avoid algorithmic starvation.
  2. Benchmark Grounding: Node mem_bm_003 notes that developer search commands $8–$18 CPC but converts at >5% when documentation sitelinks are featured.
  3. Audience Skepticism Defense: Node mem_aud_001 warns that technical leads reject superlatives, injecting an anti-pattern guardrail: avoid generic adjectives; highlight verified benchmarks.
  4. Creative Synthesis: Variant A blueprint proposes an architectural diagram hook: "How infrastructure teams eliminate deployment friction", linked to API schemas.

Before vs After Memory

Dimension Stateless Marketing Generation MarkSight Closed-Loop Memory
Cold Starts Begins every prompt from zero assumptions. Evaluates against vertical baseline benchmarks.
Channel Allocation Arbitrarily splits budget across requested channels. Enforces the Liquidity Law to prevent algorithmic starvation.
Messaging Selection Recommends generic ad copy and slogans. Injects audience skepticism profiles and verified past angles.
Error Handling Repeats previously failed tactics and formats. Enforces anti-pattern guardrails derived from past post-mortems.
After Campaign Flight Transcript is lost once session concludes. Outcome variance is calculated and retained as a new memory node.

What Was Difficult

The most nuanced technical challenge was not vector storage; it was curating what deserves retention.

Early in development, serializing raw prompt transcripts directly into memory bloated the store with conversational fluff. Unstructured transcripts do not improve agent judgment; structured, typed observations do.

I structured memory into four deliberate classes: episodic, benchmark, heuristic, and audience. Each node enforces strict typing: spend, ROAS, CPA, conversion rates, learnings, and anti-patterns. Filtering out noise at the write boundary kept the memory topology clean, observable, and queryable.

Another challenge was handling network resilience. External cloud memory calls should never block the local UI or crash an active session. In server/src/memoryEngine.js, I isolated syncToCloud as a non-blocking asynchronous call with a strict timeout:

// server/src/memoryEngine.js (excerpt from syncToCloud)
async syncToCloud(memoryNode) {
  if (!this.hindsightApiKey) return;
  try {
    await fetch(`${this.hindsightApiUrl}/v1/memories`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${this.hindsightApiKey}`
      },
      body: JSON.stringify({ memory: memoryNode }),
      signal: AbortSignal.timeout(3500)
    });
  } catch {
    // Graceful fallback for local execution
  }
}
Enter fullscreen mode Exit fullscreen mode

If the external endpoint is unreachable or credentials are not yet configured, the system continues operating locally from store.json without failing the active session.

Lessons Learned

  1. Memory Must Alter Decisions: An agent storing memories that never change downstream parameters is just maintaining an expensive log file. In MarkSight, retrieved nodes directly govern budget splits, CPA targets, and anti-pattern rules.
  2. Hybrid Scoring Beats Pure Vector Search: For performance marketing, semantic similarity alone is insufficient. Exact matching on channels, vertical constraints, and numerical confidence scores must weight the retrieval score alongside keyword tokens.
  3. Observability Builds Trust: When an agent suggests allocating 65% of budget to LinkedIn, the user must understand why. Displaying retrieved node IDs and relevance scores in an inline Stage Inspector drawer transforms black-box advice into inspectable engineering.
  4. Failure Is Higher-Value Than Success: Positive learnings are helpful, but anti-pattern nodes—such as identifying that broad-match search queries burned budget on low-intent clicks—consistently save more capital.

Limitations and What's Next

MarkSight currently uses deterministic rule synthesis and structured template heuristics for strategy formulation rather than live external LLM narrative calls. While the scaffolding, prompt structure, and data pipelines are fully implemented, wiring the synthesizer directly to production LLM narrative generation remains an active development item.

Additionally, the 30-day flight simulator uses a stochastic variance model ($\pm 12\text{--}18\%$) to simulate market volatility. While manual actuals can be logged via POST /api/campaigns/log-actuals, direct OAuth integration into Google Ads and LinkedIn Campaign Manager APIs to automatically ingest live delivery numbers is the logical next architectural phase.

Top comments (0)