A Creative Intelligence Agent
Marketing dashboards are very good at answering one question:
What happened?
Spend increased. ROAS dropped. CTR improved. A campaign started to fatigue.
But that is not the question a creative team actually asks.
They want to know:
Why did this creative work, and what should we make next?
That answer does not exist in a metrics table.
An ad is a collection of creative decisions: the hook, the first three seconds, the persona, the messaging angle, the offer, the CTA, the visual style, and the audio.
The real problem is to connect those decisions to performance.
Over the last few weeks, I built a creative intelligence agent to do exactly that.
The principle behind the system is simple:
Compute the deterministic parts yourself - structure, statistics, confidence, and evidence and then let the agent reason over them.
Calling the model was not the hard part.
The hard part was building the system around it so the analysis remained reliable, idempotent, cost-controlled, and explainable.
The Creative Genome
The atomic unit of the system is the creative genome.
A multimodal model watches the actual ad and returns a structured extraction instead of a paragraph.
The extraction includes:
- The hook and its transcript
- When the product appears
- When the CTA appears
- The messaging angle
- Claims made in the ad
- Visual style
- Persona
- Audio
This gives me a structured representation of one creative.
But a JSON blob per ad is not enough.
If I have a thousand ads, I need to answer questions across all of them:
- Which hooks perform best?
- Which personas work with which offers?
- Which combinations are starting to fatigue?
- Which angles are competitors using that the brand is not?
So the genome is expanded into creative tags.
A large nested JSON document is difficult to aggregate.
Tags are easy to group, filter, and join with performance.
If the genome describes one creative, the tags describe the creative language of the account.
The surrounding data model includes:
-
creative patterns -
creative kb summaries -
competitor ad analysis creative analysis jobs- Embeddings in Chroma for semantic search
- Memory used by the agent
I kept embeddings in Chroma because semantic retrieval is a different workload from relational and analytical queries. Postgres remains responsible for system state, while ClickHouse handles performance analysis. Chroma handles nearest-neighbor search and lets the embedding index be rebuilt or changed without coupling it to either database schema.
The System
The architecture has three main parts:
- A worker that extracts creative genomes
- A daily job that computes patterns
- An agent that reasons over the resulting evidence
The extraction worker handles the slow path.
For every ad, it resolves the asset, sends the image or video to the multimodal model, validates the structured response, and stores the genome and its tags.
A single extraction can take 30 to 90 seconds.
The daily job handles the analytical path.
It joins genomes with performance and computes:
- Winning and losing patterns
- Fatigue curves
- Competitor gaps
- An insight digest
- The updated creative knowledge base
The agent sits on top of this system.
It uses prompting, context, memory, tool calling, and an execution harness to turn precomputed evidence into an answer or a creative recommendation.
It uses tools to retrieve the structured knowledge required for the current question.
Do Not Pay the Model Twice
Video extraction is the slowest and most expensive operation in the system.
Before running it, the worker checks whether the same creative has already been analyzed.
Every creative receives a deterministic fingerprint:
sha256("creative_id | video_id | body | title")
The worker then follows one of three paths:
- Same fingerprint and same ad: skip
- Same fingerprint but another ad: copy the existing genome
- No match: run the model
With fingerprinting, one extraction can serve every ad that uses the same creative and save the costing of re-analyzing.
The Queue Is Postgres
The obvious choice for slow asynchronous work is something like Celery or SQS.
I used a Postgres table with:
FOR UPDATE SKIP LOCKED
Postgres was already the system of record.
The claim is transactional, priority is an ORDER BY, and recovery is expressed as a query.
SKIP LOCKED is the important part.
If two workers try to claim jobs at the same time, each one skips rows already locked by the other. They receive different jobs without double-processing the same row.
That means scaling the worker does not require redesigning the queue.
Leases Make Failure Boring
A job moves through a small state machine:
queued -> leased -> done
On failure, it returns to queued.
After three failed attempts, it moves to dead, with a replayable record so it is not silently lost.
Idempotency is not an optimization here.
It is the reason retries remain safe.
The system also degrades instead of failing completely.
If a video cannot be resolved, it analyzes the thumbnail or hook frame and marks the genome as a fallback.
A partial genome is still more useful than no genome.
Compute First, Narrate Second
The daily job computes the evidence before the agent sees it.
For every pattern, it calculates values such as:
- Pooled ROAS
- Spend-weighted win rate
- Lift against the account baseline
- Sample size
- Confidence interval
A pattern supported by three ads should not be presented with the same confidence as one supported by hundreds.
There is another limitation: these are observational patterns, not causal conclusions.
If ugc_handheld creatives have higher ROAS, the hook style may be helping. But the same creatives may also have been shown to a different audience, paired with a stronger offer, placed in another campaign, or given more spend. The tag is correlated with performance; it is not automatically the cause of that performance.
I treat these patterns as evidence for what to investigate or test next. Stronger causal claims require controlled experiments or comparisons that account for those confounding variables.
The same rule applies to the rest of the system.
Fatigue is calculated from CTR decay against cumulative impressions.
Embedding clusters are found mathematically.
Competitor gaps are computed before they are described.
The model explains the result.
It does not invent the result.
The Agentic Layer
The agent turns evidence into an answer.
It is built from five pieces: the prompt, the context, memory, tools, and the harness that runs them.
The prompt sets the rules of reasoning.
The first rule outranks the rest: never state a number the tools did not return.
It also fixes the evidence discipline described below — separating measured fact from interpretation, and allowing a recommendation only when the facts support one.
The context is assembled per question, not left to the model to recall.
Into each run I inject the brand's knowledge base summary, its business context, the patterns already computed for the account, any memories worth recalling, and the results of the tools called so far.
The model reasons over an assembled window, not a blank slate.
Memory lets a useful conclusion outlive the conversation that produced it.
Findings from past chats are distilled into small cards and recalled by similarity, so the agent carries forward what it already learned about this brand instead of replaying entire transcripts.
Tools are the only way the agent touches the system.
They are thin readers over what the background jobs already produced:
- tag performance — which attributes win, with lift and confidence
- patterns — the detected winning, losing, and competitor-gap patterns
- genome lookup — the full analysis of one ad
- semantic search — the nearest creatives by content
- competitor intelligence — competitor genomes and the gaps against them
Two tools do not read; they create — drafting a structured concept from the winning patterns, and generating the creative itself.
Everything else is a read. The expensive work already happened overnight.
The harness runs the loop and keeps it honest.
Closing the Loop
The system is not only an analysis layer.
It forms a creative loop:
Analyze
-> Find patterns
-> Draft a concept from those patterns
-> Generate the creative
-> Run it
-> Analyze it again
The same attributes used to explain why a creative worked can be used to specify the next one.
The agent takes what happened, turns it into structure, retrieves the right evidence, reasons over it, and produces the next action.
Getting a model to say something intelligent is the easy part.
The real engineering work is building the reliable, idempotent, cost-controlled, and explainable agentic system around it.
Top comments (0)