DEV Community: Kuldeep Singh

How I Build a Creative Agent That Learns From Ad & its Performance

Kuldeep Singh — Tue, 14 Jul 2026 12:29:00 +0000

A Creative Intelligence Agent

Marketing dashboards are very good at answering one question:

What happened?

Spend increased. ROAS dropped. CTR improved. A campaign started to fatigue.

But that is not the question a creative team actually asks.

They want to know:

Why did this creative work, and what should we make next?

That answer does not exist in a metrics table.

An ad is a collection of creative decisions: the hook, the first three seconds, the persona, the messaging angle, the offer, the CTA, the visual style, and the audio.

The real problem is to connect those decisions to performance.

Over the last few weeks, I built a creative intelligence agent to do exactly that.

The principle behind the system is simple:

Compute the deterministic parts yourself - structure, statistics, confidence, and evidence and then let the agent reason over them.

Calling the model was not the hard part.

The hard part was building the system around it so the analysis remained reliable, idempotent, cost-controlled, and explainable.

The Creative Genome

The atomic unit of the system is the creative genome.

A multimodal model watches the actual ad and returns a structured extraction instead of a paragraph.

The extraction includes:

The hook and its transcript
When the product appears
When the CTA appears
The messaging angle
Claims made in the ad
Visual style
Persona
Audio

This gives me a structured representation of one creative.

But a JSON blob per ad is not enough.

If I have a thousand ads, I need to answer questions across all of them:

Which hooks perform best?
Which personas work with which offers?
Which combinations are starting to fatigue?
Which angles are competitors using that the brand is not?

So the genome is expanded into creative tags.

A large nested JSON document is difficult to aggregate.

Tags are easy to group, filter, and join with performance.

If the genome describes one creative, the tags describe the creative language of the account.

The surrounding data model includes:

creative patterns
creative kb summaries
competitor ad analysis
creative analysis jobs
Embeddings in Chroma for semantic search
Memory used by the agent

I kept embeddings in Chroma because semantic retrieval is a different workload from relational and analytical queries. Postgres remains responsible for system state, while ClickHouse handles performance analysis. Chroma handles nearest-neighbor search and lets the embedding index be rebuilt or changed without coupling it to either database schema.

The System

The architecture has three main parts:

A worker that extracts creative genomes
A daily job that computes patterns
An agent that reasons over the resulting evidence

The extraction worker handles the slow path.

For every ad, it resolves the asset, sends the image or video to the multimodal model, validates the structured response, and stores the genome and its tags.

A single extraction can take 30 to 90 seconds.
The daily job handles the analytical path.
It joins genomes with performance and computes:

Winning and losing patterns
Fatigue curves
Competitor gaps
An insight digest
The updated creative knowledge base

The agent sits on top of this system.

It uses prompting, context, memory, tool calling, and an execution harness to turn precomputed evidence into an answer or a creative recommendation.

It uses tools to retrieve the structured knowledge required for the current question.

Do Not Pay the Model Twice

Video extraction is the slowest and most expensive operation in the system.

Before running it, the worker checks whether the same creative has already been analyzed.

Every creative receives a deterministic fingerprint:

sha256("creative_id | video_id | body | title")

The worker then follows one of three paths:

Same fingerprint and same ad: skip
Same fingerprint but another ad: copy the existing genome
No match: run the model

With fingerprinting, one extraction can serve every ad that uses the same creative and save the costing of re-analyzing.

The Queue Is Postgres

The obvious choice for slow asynchronous work is something like Celery or SQS.

I used a Postgres table with:

FOR UPDATE SKIP LOCKED

Postgres was already the system of record.

The claim is transactional, priority is an ORDER BY, and recovery is expressed as a query.

SKIP LOCKED is the important part.

If two workers try to claim jobs at the same time, each one skips rows already locked by the other. They receive different jobs without double-processing the same row.

That means scaling the worker does not require redesigning the queue.

Leases Make Failure Boring

A job moves through a small state machine:

queued -> leased -> done

On failure, it returns to queued.

After three failed attempts, it moves to dead, with a replayable record so it is not silently lost.

Idempotency is not an optimization here.
It is the reason retries remain safe.

The system also degrades instead of failing completely.
If a video cannot be resolved, it analyzes the thumbnail or hook frame and marks the genome as a fallback.

A partial genome is still more useful than no genome.

Compute First, Narrate Second

The daily job computes the evidence before the agent sees it.

For every pattern, it calculates values such as:

Pooled ROAS
Spend-weighted win rate
Lift against the account baseline
Sample size
Confidence interval

A pattern supported by three ads should not be presented with the same confidence as one supported by hundreds.

There is another limitation: these are observational patterns, not causal conclusions.

If ugc_handheld creatives have higher ROAS, the hook style may be helping. But the same creatives may also have been shown to a different audience, paired with a stronger offer, placed in another campaign, or given more spend. The tag is correlated with performance; it is not automatically the cause of that performance.

I treat these patterns as evidence for what to investigate or test next. Stronger causal claims require controlled experiments or comparisons that account for those confounding variables.

The same rule applies to the rest of the system.

Fatigue is calculated from CTR decay against cumulative impressions.

Embedding clusters are found mathematically.

Competitor gaps are computed before they are described.

The model explains the result.

It does not invent the result.

The Agentic Layer

The agent turns evidence into an answer.

It is built from five pieces: the prompt, the context, memory, tools, and the harness that runs them.

The prompt sets the rules of reasoning.

The first rule outranks the rest: never state a number the tools did not return.

It also fixes the evidence discipline described below — separating measured fact from interpretation, and allowing a recommendation only when the facts support one.

The context is assembled per question, not left to the model to recall.

Into each run I inject the brand's knowledge base summary, its business context, the patterns already computed for the account, any memories worth recalling, and the results of the tools called so far.

The model reasons over an assembled window, not a blank slate.

Memory lets a useful conclusion outlive the conversation that produced it.

Findings from past chats are distilled into small cards and recalled by similarity, so the agent carries forward what it already learned about this brand instead of replaying entire transcripts.

Tools are the only way the agent touches the system.

They are thin readers over what the background jobs already produced:

tag performance — which attributes win, with lift and confidence
patterns — the detected winning, losing, and competitor-gap patterns
genome lookup — the full analysis of one ad
semantic search — the nearest creatives by content
competitor intelligence — competitor genomes and the gaps against them

Two tools do not read; they create — drafting a structured concept from the winning patterns, and generating the creative itself.

Everything else is a read. The expensive work already happened overnight.

The harness runs the loop and keeps it honest.

Closing the Loop

The system is not only an analysis layer.

It forms a creative loop:

Analyze
  -> Find patterns
  -> Draft a concept from those patterns
  -> Generate the creative
  -> Run it
  -> Analyze it again

The same attributes used to explain why a creative worked can be used to specify the next one.

The agent takes what happened, turns it into structure, retrieves the right evidence, reasons over it, and produces the next action.

Getting a model to say something intelligent is the easy part.

The real engineering work is building the reliable, idempotent, cost-controlled, and explainable agentic system around it.

"From 30-Minute Manual Deployments to 5-Minute Automation: My CI/CD Journey"

Kuldeep Singh — Sat, 28 Jun 2025 07:26:41 +0000

my-app/ ├── .github/ │ └── workflows/ │ └── deploy.yml ├── public/ ├── src/ ├── package.json ├── next.config.js ├── README.md └── ...

Code

Step 2: Creating the GitHub Actions Workflow

Here's the workflow file I created:


yaml
name: Deploy to EC2

on:
  push:
    branches: [ main ]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Deploy to Server
        uses: appleboy/ssh-action@v1.0.3
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          key: ${{ secrets.PRIVATE_SSH_KEY }}
          script: |
            echo "🎉 Starting Birthday App deployment..."

            # Create app directory if it doesn't exist
            mkdir -p ~/birthday-app
            cd ~/birthday-app

            # Clone or update the repository
            if [ -d ".git" ]; then
              echo "📦 Updating existing repository..."
              git pull origin main
            else
              echo "📦 Cloning repository..."
              git clone https://github.com/Kuldeep2602/CI_CD.git .
            fi

            # Navigate to the app directory
            cd my-app

            # Install dependencies
            echo "📋 Installing dependencies..."
            npm install

            # Build the application
            echo "🔨 Building Next.js application..."
            npm run build

            # Install PM2 if not already installed
            if ! command -v pm2 &> /dev/null; then
              echo "📦 Installing PM2..."
              npm install -g pm2
            fi

            # Stop existing app if running
            pm2 stop birthday-app 2>/dev/null || true

            # Start the application with PM2
            echo "🚀 Starting Birthday App with PM2..."
            pm2 start npm --name "birthday-app" -- start
            pm2 save

            # Show status
            pm2 status
            echo "✅ Birthday App deployment completed successfully!"
Step 3: Setting Up GitHub Secrets
To securely handle sensitive information, I added these secrets to my GitHub repository:

Go to your Repository → Settings → Secrets and variables → Actions
Add the following secrets:
HOST: The public IP of your EC2 instance (e.g., 13.60.232.88)
USERNAME: The SSH username (e.g., ubuntu)
PRIVATE_SSH_KEY: Your private SSH key (the content of your .pem file)
Step 4: Setting Up the EC2 Instance
I prepared my EC2 instance with:

bash
# Update system packages
sudo apt update && sudo apt upgrade -y

# Install Node.js and npm
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt install -y nodejs

# Install Git
sudo apt install -y git

# Install PM2 globally
sudo npm install -g pm2

# Set up SSH authorized_keys for GitHub Actions
mkdir -p ~/.ssh
chmod 700 ~/.ssh
# Add your public key to ~/.ssh/authorized_keys
nano ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
Step 5: Using Ngrok for Public Access
To make my application publicly accessible without a domain:

bash
# Install Ngrok
wget https://bin.equinox.io/c/bNyj1mQVY4c/ngrok-v3-stable-linux-amd64.tgz
tar xvzf ngrok-v3-stable-linux-amd64.tgz
sudo mv ngrok /usr/local/bin/

# Configure Ngrok
ngrok config add-authtoken YOUR_AUTH_TOKEN

# Start Ngrok to expose your Next.js app (running on port 3000)
ngrok http 3000
The Results
After implementing this CI/CD pipeline, I achieved:

83% faster deployments: From 30 minutes to just 5 minutes
Zero deployment errors: Eliminated the 20% failure rate
Zero downtime: PM2 ensures smooth application restarts
Confidence in deploying: No more fear of breaking the production environment
Challenges I Faced
SSH Key Format: Initially, GitHub Actions couldn't connect to my EC2 instance because of incorrect SSH key formatting. I had to ensure the key was properly formatted with proper line breaks.

IP Address Mismatch: I mistakenly used an old IP address in my GitHub secrets, causing connection timeouts. Always double-check your host IP when configuring secrets!

PM2 Persistence: I learned that without pm2 save, the process list wasn't persisted across server restarts.

Lessons Learned
Automate Everything: Every manual step is an opportunity for error
Security First: Use secrets for sensitive information
Zero-Downtime Deployments: PM2 is invaluable for Node.js applications
Testing is Essential: Next step is to integrate automated testing
Next Steps
Add automated testing to the workflow
Set up monitoring and alerts
Implement a staging environment
Replace Ngrok with a proper domain and HTTPS
Conclusion
Building this CI/CD pipeline has transformed my development workflow and significantly improved productivity. The ability to push code and have it automatically deployed within minutes is incredibly powerful.

Have you implemented CI/CD for your projects? What tools and approaches did you use? Let me know in the comments!

You can check out the live application here: https://86d2-13-60-232-88.ngrok-free.app/

GitHub Repo: https://github.com/Kuldeep2602/CI_CD

Code

This blog post is now correctly formatted for DEV.to with:

1. Proper frontmatter metadata
2. Correctly formatted directory structure using code blocks
3. All code samples properly formatted with language specification
4. Current date (June 28, 2025) included
5. Your GitHub username (Kuldeep2602) incorporated
6. All content sections maintained with proper headings
7. Helpful tips and command examples throughout