DEV Community: Magenta Qin

Loop Engineering Visualized: A World Cup Fan Journey with SerpApi and Codex

Magenta Qin — Wed, 29 Jul 2026 18:50:38 +0000

Prompt Engineering focuses on shaping a model call, while Loop Engineering focuses on what happens after that call: how an agent observes the result, updates its state, chooses the next action, and decides whether to continue.

The difference sounds simple. But it is hard to feel until you build something where the model has to keep going:

A conceptual Loop Engineering loop, from trigger and state to tools, observations, memory updates, and the stop condition.

The easiest way to understand a loop is to watch one unfold.

So I built a small World Cup fan demo with Codex and SerpApi to make Loop Engineering visible.

The product idea is intentionally simple: a fan starts with one match, and the agent follows the journey from there. It identifies an interesting player, discovers related matches, searches for highlights, reads fan debates, and keeps deciding what to explore next.

Eventually, it produces a trace showing how the entire journey unfolded. But under the surface, this is not just a football app. The fan journey is the agent loop.

Every discovery becomes a new context. Every action changes the state. And every result influences what the agent does next.

The Demo

The demo is called World Cup Fan Journey , and the full source code is available in the Github repository.

The World Cup Fan Journey starts with a natural-language match request and turns it into an interactive path through matches, players, highlights, and fan discussions.

The journey starts with a simple natural-language request:

Show me France vs Morocco

or in Chinese:

我想看挪威对法国的比赛

The app interprets the request, identifies the football matchup, and initializes the first journey state. From there, the agent starts building a fan pack around the match:

Live match context and upcoming games
Highlights and social content
Fan debates and trending stories
Related players and AI-generated match analysis

The first iteration turns a match request into a fan pack grounded in live sports data, videos, news, trends, and AI-generated analysis.

The important interaction is what happens after the user clicks a player.

For example, if the user follows Kylian Mbappé, the goal changes. The app no longer needs to explain only France vs Morocco. It now needs to explore Mbappé’s football trail: recent matches, videos, debates, memes, and related players.

Following a player starts a new iteration: the goal shifts from exploring one match to building that player’s wider football trail.

The player clicks to update the goal. From there, the agent enters another loop: it searches, observes the results, updates the journey state, and decides what to retrieve next.

Following Mbappé changes the goal and expands the journey into three new match trails, each with fresh content and new players the user can follow into the next iteration.

The user chooses the direction. The loop handles the exploration.

Finally, when the user clicks Generate my football graph , the demo stops exploring and turns the accumulated journey into a Loop Engineering trace.

That final output is not meant to be a normal social graph. It is meant to answer:

What did the agent know, what did it observe, what did it do next, and why?

Pipeline vs Loop

A pipeline follows a predefined sequence, while this demo loops through goal, planning, search, observation, and state updates until the user chooses the next direction.

A normal pipeline follows a predefined path:

Fetch the match → fetch videos → fetch players → render the page.

The next step is already known. However, this demo works differently.

The agent starts with a goal and builds a journey state around it. Search results introduce new matches, debates, and related players. These observations expand the state and create new directions the journey can take.

If the user follows Mbappé, the goal changes. The agent plans which of Mbappé's matches to explore, retrieves related football content, and adds new players and observations to the journey. The user can then follow another player, changing the goal again.

The loop continues:

Goal → Plan → Search → Observe → Update State → Choose Next Direction → Goal Changes → Plan Again

This is a human-in-the-loop agent loop. The agent handles planning and exploration, while the user decides which direction is worth following.

A pipeline says:

Do A, then B, then C.

This loop asks:

Given the current goal and journey state, what should we explore next?

Mapping the Demo to the Loop

The easiest way to map the demo to the loop is to follow it iteration by iteration.

Each iteration starts with a trigger and a goal. The agent plans what to explore, calls tools, observes the results, and updates the journey state.

In this demo, the next iteration begins when the user chooses a new direction.

Trigger → Goal → Plan → Tools → Observe → Update State → Next Trigger

The demo unfolds as a human-in-the-loop journey: each user choice changes the goal and starts another iteration, while the same loop structure repeats underneath.

Iteration 1: Start with a Match

The first trigger is the user's search:

Show me France vs Morocco.

The goal is:

Build a fan pack for France vs Morocco.

The app parses the request and uses sports, YouTube, news, trends, and player data to assemble the first fan journey.

The resulting observations include the match status, score, available videos, discussions, and related players. Those results become the current journey state.

At the end of the first iteration, the system has not just rendered a page. It has created possible directions for the next iteration.

Iteration 2: Following Mbappé Changes the Goal

The user clicks Follow Kylian Mbappé. That click becomes the next trigger. In the code, this action starts a new planning step:

The followPlayer() flow turns a user click into a new plan, enriches the selected matches, and expands the current journey state.

The LLM decides which three recent or trending matches are worth exploring:

planPlayerExpansion() asks the LLM to select three recent or trending matches for the followed player and return them as a structured exploration plan.

The plan is then grounded with external search data by SerpApi. It retrieves the surrounding sports results, videos, news, and trends.

enrichPlannedMatch() grounds the LLM’s plan with sports results, videos, and discussions from SerpApi before adding the new match trail to the journey state.

The agent observes three new match trails and discovers more related players. The state expands. Now the user has another set of possible directions.

The user chooses the direction. The agent explores it.

Iteration 3: The Loop Continues

Suppose the user follows Erling Haaland.

The same loop structure runs again, but with a different goal and a different state.

The agent plans Haaland's match trail, retrieves new context, observes the results, and adds new match and player nodes to the journey.

This is why the demo is not a fixed pipeline. The structure repeats, but the content of each iteration depends on the current goal and accumulated state.

Iteration 4: Another Direction, Same Loop

The user follows Martin Odegaard.

Again:

New trigger → new goal → new plan → new observations → expanded state

The loop structure stays stable. The journey does not.

By this point, the system has accumulated multiple player trails, discovered matches, and possible next actions.

Final Iteration: Stop and Generate the Trace

The user clicks:

Generate my football graph.

This becomes the stop condition.

The agent stops exploring and turns the accumulated journey into a Loop Engineering trace.

The final output shows how each iteration changed:

the goal
the current state
the tools used
the observations
the memory carried forward

The graph is therefore not just a record of which players the user followed.

It is a visualization of how the loop changed from one iteration to the next.

The final trace reveals how each user action changed the goal, expanded the state, and started the next iteration.

Why SerpApi is Useful

For this demo, SerpApi acts as the observation layer. The LLM can plan what to explore, but the football world keeps changing: match status and scores change, new videos appear, and news and fan discussions evolve.

The loop needs a way to observe that external world.

SerpApi provides that context through several search APIs:

Google Sports API : match scores, status, stage, venue, and time
YouTube Search API : highlights and shorts
Google News API and Google Trends API: debates, narratives, and rising topics
Instagram Profile API : player profile context
Google Images API : player photos and visual fallbacks

The LLM decides what to explore. SerpApi gives the loop fresh context from the outside world.

Building It with Codex

I built this demo with Codex, but the hardest part was not writing the code.

At the beginning, even the product goal was vague.

I knew I wanted to build a World Cup demo that made Loop Engineering visible. I did not yet know what that experience should look like.

The first version reflected that uncertainty.

The first version exposed the loop as an engineering dashboard, making the architecture visible but the user experience difficult to understand.

It exposed almost every engineering concept directly: loop state, observations, memory, constraints, tool calls, validators, and a replay timeline.

Technically, the ideas were there. But as a product, it was painful to use.

A user had to understand Loop Engineering before they could understand the demo. The interface was explaining the architecture instead of letting the user experience it.

So the development process became iterative.

I would inspect the UI, explain what felt confusing, and ask Codex to change the product around that observation. Codex would read the existing codebase, make a targeted change, run the build, and give me another version to react to.

Build → Observe → Rethink the goal → Patch → Verify → Repeat

After several iterations, the product idea became much clearer.

Instead of showing the loop as an engineering dashboard, the demo would hide most of the machinery behind a simple fan journey:

Start with a match. Follow a player. Discover another match. Follow another player.

The user experiences the loop first.

Only at the end does the app reveal the Loop Engineering trace and show how each action changed the goal, state, observations, and next iteration.

That shift changed the entire interface.

The redesigned interface hides the engineering machinery behind a simple fan journey, letting users experience the loop before revealing the trace underneath it.

The interesting part is that Codex was not simply implementing a finished specification. The specification itself became clearer through the implementation loop.

Every version gave me a new observation. Those observations changed what I asked Codex to build next.

The workflow itself became loop-shaped.

Two Failures That Made the Loop Clearer

The most useful lessons came from the parts that did not work. Some were ordinary UI bugs. Others exposed weaknesses in how I was thinking about the loop itself.

1. The First Graph Showed Relationships, Not Iterations

My first version ended with a graph of players and matches. It showed that Mbappé, Haaland, and Ødegaard were connected to different matches, but it did not explain why the journey moved from one player to the next.

It showed relationships, but not:

What triggered each iteration
How the goal changed
What the agent observed
What state was carried forward

It was a relationship graph, not a loop trace.

So I redesigned the final output around iterations. The new trace shows the goal, current state, tools, observations, and memory for each step.

The redesigned trace organizes the journey by iteration, revealing how each trigger changed the goal, state, observations, and memory carried into the next step.

A visualization should explain the concept, not just decorate the data.

This change also clarified the product itself. The fan journey was only the surface. The real output was the sequence of state transitions underneath it.

2. Search Relevance Is Not Factual Correctness

At one point, a video title implied that Mbappé had scored in a match where he had not.

The player-trail search included his name:

findYoutubeVideo(
  `${planned.title} ${player.name} shorts memes`,
  "short"
);

That made sense for retrieval. The goal was to find content related to Mbappé. But a relevant title could still imply incorrect facts, such as the wrong score or a goal he never scored.

The bug was simple:

The application treated search-derived copy as verified sports data.

I added a deterministic fact guard before displaying those titles. Sports data verifies the score and match events; search and the LLM only shape how the content is presented.

Sports data owns the facts. Search and the LLM shape the narrative.

The broader Loop Engineering lesson is:

A relevant observation is not always trustworthy enough to update factual state.

What I Would Improve Next

The demo makes the loop visible, but it is still a prototype. If I wanted to harden it, I would focus on three things.

1. Unify the Loop State Model

The demo currently has two state models because the product evolved in stages.

The earlier replay-mode prototype derives a display snapshot from SessionState and PersistentMemory:

const snapshot = buildLoopStateSnapshot(session, memory);

The final fan journey uses RabbitHoleState as its main application state:

const [state, setState] =
  useState<RabbitHoleState | null>(null);

This split reflects the development history: the replay dashboard came first, and the fan journey came later.

If I hardened the project, I would unify both flows around one sequence of loop-state transitions. The fan UI, execution logic, and final trace should all read from the same underlying history instead of adapting between separate representations.

The trace should come from the loop that actually ran, not from a parallel model built to explain it.

That would make each iteration easier to inspect, persist, and replay.

2. Add Production Guardrails

The demo mostly relies on the user to decide when to stop.

A production loop should also enforce operational limits, including maximum iterations, API timeouts, retry budgets, and graceful fallback behavior when a tool fails.

Instead of retrying forever, the loop should be able to stop and return the best partial result available.

A stop condition is also a reliability boundary.

3. Persist the Journey Across Sessions

Right now, the fan journey is mainly session-based.

I would persist followed players, expanded trails, accepted content, and other useful memory so that a new session does not start from zero. That would let the loop carry context across a longer fan journey, rather than only across interactions on one page.

The Main Takeaway

Everyone is talking about Loop Engineering right now, but people do not always mean exactly the same thing. Some use the term to describe the iteration inside a single AI agent: observe, update state, choose the next action, and repeat. Others use it at a higher level to describe systems that coordinate multiple agent runs over time.

Neither view is necessarily wrong. They are looking at the same idea from different levels of abstraction.

In this demo, I focused on the micro view. The loop is human-in-the-loop: the user chooses a direction, the agent explores it, new observations expand the state, and the next user action starts another iteration.

The hard part is not making the model act once. It is designing how goals, observations, state, and decisions evolve from one iteration to the next.

That was also why football turned out to be such a useful example. The outside world keeps changing. A match moves from upcoming to live to full-time. Scores change. New videos appear. News and fan discussions evolve. The model cannot simply remember that world correctly. The loop has to be observed again.

For this demo, SerpApi's Google Sports APIbecame the main source of match state: scores, match status, teams, schedules, and other sports context. That fresh data could then be written back into the loop state before the next iteration continued.

Calling a tool is not enough. The result has to become a new observation, and that observation has to update the state the next iteration will use.

If you are building an AI agent or workflow around live sports data, try the SerpApi Google Sports API and see what kind of loop you can build around changing match state.

How I Built a Star Wars Grogu Product Research Agent with Codex, Lark, and SerpApi

Magenta Qin — Tue, 28 Jul 2026 09:42:36 +0000

Cross-border e-commerce sellers often spend hours comparing the same products across different Amazon marketplaces. Prices, reviews, and seller signals vary by country, but the process is still largely manual.

I wanted to see how far I could automate it with a small AI agent built using Codex, SerpApi, and Lark.

Suppose I ask in Codex:

Compare Grogu products in the US and Japan

The workflow is straightforward:

search Amazon marketplaces
fetch product details
summarize seller signals
send a report to Lark

I kept the stack intentionally simple:

Python 3.12
uv
Pydantic
SerpApi Python SDK
OpenAI structured outputs
Lark custom bot webhook
Codex for both development and as the conversational interface

Running the Agent from Codex

Once everything was wired together, I could simply ask Codex:

Compare Grogu products in the US and Japan

Codex understood the request, triggered the CLI workflow, collected marketplace data via SerpApi, and delivered a structured report to Lark.

Here’s the entire flow in action:

0:00
/0:16
1×

The Workflow

The overall flow looks like this:

Starting with a natural-language request in Codex, the agent first validates that the question is related to cross-border Amazon research.

OpenAI then translates the request into a structured command. SerpApi handles both product discovery and detailed product retrieval, while OpenAI extracts seller-focused insights from the collected data.

Finally, the results are packaged into a Lark card and delivered via a Custom Bot webhook.

I intentionally did not start with FastAPI, background jobs, or a database. For the MVP, the important question was simpler: Can I go from a natural-language product question to a useful cross-border product card?

Translating Natural Language into Commands

The natural-language entrypoint uses OpenAI to translate user requests into structured commands.

For example:

Compare Grogu products in the US and Japan

becomes:

query="Grogu" 
marketplaces=["us","jp"] 
output_mode="send_lark"

Using a strict schema keeps the pipeline predictable and much easier to debug. Instead of letting the model orchestrate everything, I only ask it to generate structured commands.

Searching Amazon with SerpApi

This project relies on two SerpApi endpoints.

The Amazon Search API is used to discover candidate products, while the Amazon Product API enriches them with much richer details.

Search results provide ranking context and thumbnails, while product pages contain detailed information such as images, availability, and descriptions.

Combining both produced much better product cards than using either endpoint alone.

Modeling Product Data

Amazon pages are messy.

Some products have ratings but no availability.

Some have images but no variants.

Some fields simply don't exist.

Missing data is normal, not an exception.

My first instinct was to say, "Why not model most fields as optional?" That is true, but it is not the whole solution.

The real issue was that SerpApi returns useful product data from several different places. Some fields are flat. Some fields are nested. Some fields have different names depending on whether they came from Amazon Search API or Amazon Product API. Some products should not be shown at all if they are missing the signals a seller actually needs.

To make the data usable, I:

normalized both endpoints into a common Product model;
filtered out products with weak seller signals;
merged Product API details back into search results;
treated missing fields as expected rather than failures.

The Product model still uses optional fields, because missing data is normal:

But optional fields alone were not enough. I also filtered search results before choosing products for detail lookup:

This was important for the Lark card. A cross-border seller does not want a table full of Rating: N/A and Reviews: N/A. Those rows make the card noisy and less actionable.

The next issue was nested and inconsistent JSON.

For price, SerpApi may return a string, a number-like value, or a nested object:

For availability, the Product API does not always use one stable field. I had to check availability, stock, and sometimes delivery:

Keeping the Agent Narrow

The most interesting part of the project isn't Grogu.

It's scope.

The entrypoint only supports cross-border Amazon product research.

If somebody asks:

What is today's weather?

the app simply rejects the request.

Requests outside the project scope are rejected locally before calling OpenAI.

For supported requests, OpenAI returns a strict command object:

This keeps the agent predictable. It translates requests into commands instead of improvising actions.

I deliberately avoided turning this into a general chatbot.The agent only knows one workflow: cross-border Amazon product research.That narrow scope makes the behavior easier to explain, test, and trust. It also keeps OpenAI API responsible for translation rather than improvisation.

Generating Seller-Friendly Insights and Delivering Them to Lark

Once product data has been collected, OpenAI generates seller-focused insights. I intentionally constrain the model to use only information returned by the APIs, avoiding hallucinated prices, ratings, or availability.

Instead of producing generic summaries, the analysis focuses on demand signals, social proof, pricing, and obvious risks — information that is much more useful for sellers.

The final result is delivered through a Lark Custom Bot webhook.

Error Handling

External APIs fail. That's normal.

I treated each layer independently.

If OpenAI fails, no command is generated.

If SerpApi fails, the analysis stops with a clear message.

If Lark delivery fails, the report can still be viewed locally.

This separation keeps failures localized and prevents one component from bringing down the entire workflow.

Conclusion

The most interesting lesson from this project wasn’t the Grogu theme or the Lark card.

It was learning where the boundaries should be.

The agent works because it stays narrow.

That narrowness makes the system easier to understand, debug, and trust.

And if you’re building tools for cross-border e-commerce, SerpApi’s Amazon API provide a surprisingly rich source of product data. They made this entire workflow possible.

If you’re working on product research, seller analytics, or marketplace intelligence, I’d recommend giving them a try.

Using SerpApi and DeepSeek to Break Down Dan Koe’s Content Strategy

Magenta Qin — Mon, 01 Jun 2026 10:30:49 +0000

Dan Koe is one of those creators who always seems to say the right thing at the right time. His videos consistently resonate around productivity, purpose, self-education, and building a meaningful life.

At some point, I started wondering: Is there a repeatable pattern behind his content? Or is it mostly intuition?

More importantly, can a creator's content strategy actually be broken down into something systematic and reproducible?

So I ran a small experiment.

Repository: https://github.com/serpapi/creator-lens

Using SerpApi, I collected metadata, video details, and transcripts from Dan Koe’s YouTube channel(top 100 videos). Then I used DeepSeek to analyze the patterns behind the content.

Instead of simply watching the videos, I tried to reverse-engineer his strategy across four dimensions: content themes, title patterns, publishing cadence, and recurring core beliefs.

What I found was far more structured, and far more revealing than I expected.

The Engineering Approach

To test this idea, I built a small internal tool called CreatorLens. The workflow was intentionally simple:

Use SerpApi to collect YouTube data
Normalize the data
Send the structured payload to DeepSeek
Generate strategy insights through AI analysis
Visualize the result in a dashboard

I didn’t use a database for this experiment. Instead, I wanted to see how far I could get with a lightweight, stateless pipeline built around APIs and structured prompts.

The stack looked like this:

User Input
↓
SerpApi YouTube Search API
↓
SerpApi YouTube Video API
↓
SerpApi YouTube Transcript API
↓
DeepSeek Analysis
↓
CreatorLens Dashboard

At a high level, SerpApi handled the data layer — searching videos, fetching metadata, and retrieving transcripts. And DeepSeek handled the reasoning layer — identifying recurring themes, extracting title patterns, clustering audience pain points, summarizing core beliefs, and generating strategy frameworks.

The interesting part wasn’t just collecting the data. It was turning unstructured YouTube content into something analyzable.

Collecting YouTube Data with SerpApi

The first challenge for me was turning a YouTube channel into structured, analyzable data.

Instead of manually opening videos one by one, I used SerpApi’s YouTube APIs to build a lightweight data pipeline. Just like what we mentioned before, the workflow looked like this:

YouTube Search API
→ fetch videos

YouTube Video API
→ enrich metadata

YouTube Transcript API
→ retrieve spoken content

DeepSeek
→ analyze creator strategy

As I wanted to keep the implementation itself simple and stateless, I built a single API route in Next.js that orchestrated the entire flow:

Search videos
→ fetch metadata
→ fetch transcripts
→ normalize data
→ send compact payload to DeepSeek

To keep the implementation clean, I wrapped the SerpApi JavaScript SDK into a small reusable helper:

import { getJson } from "serpapi";

function serpapi(params: Record<string, string>) {
  return getJson({ ...params, api_key: API_KEY });
}

From there, I split the data collection logic into three focused functions:

searchYouTubeVideos()
fetchVideoDetails()
fetchTranscript()

Each function handled a different layer of the YouTube dataset.

Searching Videos

The first function, searchYouTubeVideos(), uses SerpApi’s YouTube Search API to retrieve videos from a creator.

const data = await serpapi({
  engine: "youtube",
  search_query: creatorName,
});

From the response, I extracted the fields needed for analysis:

video ID
title
thumbnail
view count
publish date
duration

return results
  .slice(0, maxVideos)
  .map((v) => ({
    videoId: (v.link as string)?.split("v=")[1]?.split("&")[0] ?? "",
    title: (v.title as string) ?? "",
    thumbnail: (v.thumbnail as Record<string, string>)?.static ?? "",
    views: v.views,
    publishedDate: (v.published_date as string) ?? "",
    length: (v.length as string) ?? "",
  }))

I also added a small parseViews() helper to normalize values like:

"1.2M views"
"842K views"

into actual numeric values.

Fetching Video Details

But the fields for each video is not enough for me to analyze, so I added the second function, fetchVideoDetails(). Through SerpApi’s YouTube Video API, it gives me additional metadata.

const data = await serpapi({
  engine: "youtube_video",
  v: videoId,
});

This gives more detailed information like:

description
likes
more accurate view counts

return {
  description: (vr.description as string) ?? "",
  views: parseViews(vr.views),
  likes: parseViews(vr.likes),
};

I also intentionally wrapped the function in try/catch so the pipeline can fail gracefully if a single request breaks.

Fetching Transcripts

The most important part of the pipeline was transcript retrieval.

Using SerpApi’s YouTube Transcript API:

const data = await serpapi({
  engine: "youtube_video_transcript",
  v: videoId,
});

I extracted transcript snippets and joined them into plain text:

return snippets
  .map((s) => s.text)
  .join(" ");

This is where the project shifted from simple analytics into actual strategy analysis.

Titles and metrics tell you what performed.

Transcripts reveal:

**- recurring language and persuasion patterns

mental models and emotional framing
audience positioning**

Composing Three Fetch Functions Into One Orchestration Layer

Finally, I combined all three functions into a single orchestration layer.

The goal of this function is simple: given a creator name, return a normalized list of videos with everything DeepSeek needs for analysis.

export async function fetchVideosWithDetails(
  creatorName: string,
  maxVideos = 10
): Promise<VideoData[]> {

Internally, the pipeline does three things:

Creator name
→ search relevant YouTube videos
→ fetch video metadata
→ fetch video transcripts
→ return normalized VideoData[]

For each video, metadata and transcript are fetched in parallel:

const [details, transcript] = await Promise.all([
  fetchVideoDetails(v.videoId),
  fetchTranscript(v.videoId),
]);

This avoids unnecessary sequential API calls. The transcript does not depend on the metadata response, so both requests can run at the same time.

The final output is a structured VideoData[] array, where each item contains the video title, URL, metadata, and transcript. This gives DeepSeek a clean input format instead of forcing it to reason over scattered API responses.

At a high level, SerpApi handled the full data layer:

Creator Name
→ YouTube Search Results
→ Video Metadata
→ Transcripts
→ Structured Dataset

Once this dataset was ready, I could move from data collection to the analysis layer.

Using DeepSeek to Analyze Creator Strategy

Once the YouTube data was normalized into a structured dataset, the next challenge was turning raw video content into creator-level strategy insights.

My first attempt was the obvious one: truncate each transcript and send it directly to DeepSeek.

At first, this actually worked surprisingly well. But after testing longer videos, I started noticing an important limitation: important arguments, storytelling structure, and recurring ideas often appear later in the video. By aggressively slicing the transcript, the analysis became biased toward only the beginning of the content.

For example: a key insight might appear halfway through; the conclusion of an argument could be missing entirely; emotional framing develops gradually across the whole transcript.

So I tried the opposite approach — sending multiple full transcripts into a single DeepSeek prompt:

const payload = videos.map((v) => ({
  title: v.title,
  views: v.views,
  description: v.description
  transcript: v.transcript
}));

That ran into different problems. Multiple long transcripts significantly increased latency and context usage, while introducing noisy spoken content — repeated phrases, sponsor segments, filler words. As the number of videos grew, reasoning quality degraded because too much irrelevant context competed for attention.

Either way, I was hitting the same wall: a context compression problem. Truncating loses signal; combining adds noise.

Switching to a Two-Step Analysis Pipeline

So I redesigned the workflow into a hierarchical pipeline: instead of compressing information before understanding it, DeepSeek would understand each video independently first, then perform pattern analysis on those summaries.

Step 1
Full transcript
→ DeepSeek
→ compact VideoSummary

Step 2
VideoSummary[]
→ DeepSeek
→ creator strategy analysis

This architecture improved analysis quality significantly because semantic compression happened after DeepSeek fully understood each individual video.

The first stage extracted structured insights from each transcript independently.

The second stage analyzed recurring strategic patterns across the entire channel using semantically dense summaries rather than raw text.

The pipeline also became easier to scale because per-video summarization could run in parallel.

Designing Structured AI Contracts

One important realization during development was that prompt engineering was less about “asking better questions” and more about designing reliable AI interfaces.

Instead of treating DeepSeek like a chatbot, I treated it more like a structured analysis engine with strict input and output contracts.

The prompts effectively became schema-driven interfaces between different stages of the pipeline.

Step 1 Prompt: Per-Video Semantic Compression

The first prompt focused on compressing a full YouTube video into a structured semantic representation.

export function buildVideoSummaryPrompt(
  video: VideoData
): string {
  const context = JSON.stringify({
    videoId: video.videoId,
    title: video.title,
    views: video.views,
    publishedDate: video.publishedDate,
    description: video.description,
    transcript: video.transcript,
  });

  return `
You are a content analyst.

Analyze the YouTube video below and return a compact JSON summary.

Video data:

${context}

Return ONLY valid JSON with exactly these fields:
{
  "videoId": "",
  "title": "",
  "views": 0,
  "publishedDate": "",
  "mainTopic": "",
  "keyPoints": [],
  "titlePattern": "",
  "hook": "",
  "whyItWorks": "",
  "audienceProblems": [],
  "beliefs": []
}
`;
}

Instead of generating conversational summaries, the model was forced to extract structured strategic signals such as:

hooks
title patterns
audience pain points
creator beliefs
positioning strategies
narrative structures This transformed long transcripts into compact VideoSummary objects optimized for downstream reasoning.

Step 2 Prompt: Cross-Video Strategy Analysis

Once all videos were summarized, the second prompt analyzed creator-level patterns across the channel.

At this stage, DeepSeek no longer needed raw transcripts. Instead, it reasoned over structured semantic abstractions.

export function buildAnalysisPrompt(
  summaries: VideoSummary[]
): string {
  return `
You are a content strategy analyst.

Based on the per-video summaries below,
identify the creator's overall content strategy.

Video summaries:

${JSON.stringify(summaries, null, 2)}

Return ONLY valid JSON with exactly these fields:
{
  "mainNiche": "",
  "topContentThemes": [],
  "titlePatterns": [],
  "coreBeliefs": [],
  "audiencePainPoints": [],
  "publishingCadence": "",
  "videoAnalysis": [],
  "strategyReport": ""
}
`;
}

This created a hierarchy of reasoning:

raw transcript
→ semantic summary
→ creator-level strategy analysis

The second-stage analysis became surprisingly effective at identifying patterns I wouldn't have caught by watching:

Recurring hooks — emotional triggers reused across videos
Identity-based positioning — how he frames the audience to themselves
Reusable title formulas — structural templates behind high-performers
Recurring beliefs — philosophical anchors that show up again and again
Audience anxieties — the underlying fears the content addresses
Publishing rhythm — cadence patterns across multi-year timeframes At that point, the project stopped feeling like "YouTube analytics." It started feeling more like reverse-engineering a creator's communication system.

Parallelizing the Pipeline

Since each video could be summarized independently, the first stage was parallelized using Promise.all():

const summaries = await Promise.all(
  videos.map((v) => summarizeVideo(v))
);

This significantly improved throughput while avoiding unnecessary sequential API calls.

Even when analyzing many videos, the pipeline remained reasonably fast because transcript summarization happened concurrently.

Calling the DeepSeek API

For the implementation, I used DeepSeek’s chat completion API with JSON response formatting enabled.

const res = await fetch(DEEPSEEK_API_URL, {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${API_KEY}`,
  },
  body: JSON.stringify({
    model: "deepseek-chat",
    messages: [
      { role: "system", content: SYSTEM_MESSAGE },
      { role: "user", content: prompt },
    ],
    response_format: {
      type: "json_object",
    },
  }),
});

I also added a strict system instruction:

const SYSTEM_MESSAGE =
  "You are a JSON-only assistant. Always respond with valid JSON only.";

Enabling:

response_format: {
  type: "json_object",
}

significantly reduced malformed outputs.

Parsing and Normalizing the AI Response

Even with structured prompting, LLM responses can still fail occasionally.

So instead of assuming the response would always be valid JSON, I added defensive parsing logic.

DeepSeek sometimes returned markdown wrappers such as:

...

So I stripped those wrappers before parsing:

function stripMarkdown(content: string): string {
  const match = content.match(
    /```
{% endraw %}
(?:json)?\s*([\s\S]*?)
{% raw %}
```/
  );

  return match
    ? match[1].trim()
    : content.trim();
}

Then I safely parsed the response:

try { return JSON.parse(cleaned); } catch { return {}; }
Finally, I normalized all fields with fallback values:

mainNiche: parsed.mainNiche ?? "Unknown",
topContentThemes: parsed.topContentThemes ?? [],
titlePatterns: parsed.titlePatterns ?? [],

This made the overall pipeline much more resilient.
Even if the model returned partial or malformed output, the dashboard could still render safely instead of crashing entirely.

What the Pipeline Found

Running the full pipeline on Dan Koe's top 100 videos surfaced a strategy that was more systematic than I expected. Across 35.2M total views and 352K average views per video, four patterns kept reappearing.

His real topic isn't AI, or productivity, or skills. It's transformation. The pipeline grouped his 100 videos into five clusters: rapid life transformation, productivity and focus, AI and the future of work, skill development, and mindset and discipline. On paper, five different subjects. In practice, five different costumes for the same promise: you can become someone else, and fast. A title like "How To Use AI Better Than 99% Of People" looks like an AI video. But strip the AI away and what's actually being sold is become elite. The subject rotates; the topic doesn't.

His titles run on two engines. Either he's promising you something impossibly fast — "How to fix your entire life in 1 day," "How To Become So Self-Disciplined It Feels Illegal" — or he's telling you you're doing it wrong: "I'm begging you to start writing essays," "You are learning the wrong skills." Desire on one side, fear on the other. And once you notice it, you start seeing it in every title. The mechanic underneath both is the same — the title plants a question your brain can't answer without watching. Fix my life in a day? How? Wrong skills? Which ones? That question is the hook. The click is just the brain trying to close the loop.

Publishing cadence is consistent but not frequent. Roughly one video per month, with occasional gaps of up to three months. He's not playing the volume game. Each video is a deliberate piece of content carrying high production value, which fits the audience he's building — viewers who want depth, not constant noise.

The core beliefs are remarkably stable across years. Four ideas appear again and again, regardless of topic: radical change is possible through focused action; personal agency and discipline are the keys to success; AI and modern tools can dramatically accelerate growth; continuous learning and adaptability are essential. These aren't claims Dan Koe argues for — they're claims he argues from. Every video assumes them and builds on them. That's what makes the content feel coherent across hundreds of videos: it's not a feed of topics, it's a worldview being expressed across many surfaces.

The most interesting finding wasn't any single one of these — it was the combination.

The framework underneath the channel is something like: identify a specific audience pain point (feeling stuck, lacking discipline, fearing obsolescence) → offer a contrarian or high-impact solution → package it with a hyperbolic or pattern-interrupt title → deliver actionable advice that reinforces the four core beliefs.

That's not intuition. That's a repeatable system. And it's exactly the kind of thing that's nearly impossible to see by watching the videos one at a time — but obvious once you compress 100 of them into structured summaries and analyze them together.
Final Thoughts
At first, I thought I was simply building a YouTube analytics tool.

But once transcripts, metadata, and structured AI analysis started working together, the project became something much more interesting: a creator strategy reverse-engineering engine.

One of the biggest insights from this experiment was that LLMs become significantly more powerful when they are treated as structured reasoning systems rather than conversational chatbots.

The largest improvement did not come from changing models.

It came from redesigning the architecture around reasoning workflows:

full-context transcript understanding
semantic compression
hierarchical reasoning
structured AI contracts
resilient parsing
parallel orchestration
For this experiment, I also intentionally kept the backend completely stateless.

There was:

no database
no queue system
no background jobs
Every request fetched fresh YouTube data directly from SerpApi, normalized it in memory, performed multi-stage DeepSeek analysis, and returned a dashboard-ready response.That kept the architecture surprisingly simple while still producing rich creator-level insights.

More importantly, it reinforced something I’ve been thinking about recently:

Modern AI applications increasingly behave less like traditional CRUD software and more like orchestration pipelines built around reasoning models. And honestly, that was probably the most exciting part of the entire project.

If you want to build similar AI-powered analysis tools on top of real-time search and creator data, SerpApi provides a very clean developer experience for working with YouTube search, video metadata, and transcripts.

The demo app in this article uses the following APIs from SerpApi:

SerpApi YouTube Search API
SerpApi YouTube Video API
SerpApi YouTube Transcript API Give them a try in your own AI-native applications and analytical workflows.

Use which: Interface or Type Alias in Typescript?

Magenta Qin — Wed, 04 Jan 2023 05:01:21 +0000

This blog mainly discusses the key differences of interface vs type, and share how I would use them in my daily software development routine.

Describe the shape of an object or a function signature

Both can be used to describe the shape of an object or a function signature.

However, I mostly use interface to describe the object, and use type to describe a function signature, as I can clearly know it’s a definition for function from arrow function, which is impossible to realize in interface.

interface Point {
  x: number;
  y: number;
}

type SetPoint = (x: number, y: number) => void;

Another scenario where I use interface is for 「Index Signatures」. For example, in dynamic generated input, we don’t know all the names of a type’s properties ahead of time, but we do know the shape of the values.

interface Input {
  [index: string]: string | number;
}

Extending and Intersection Types

At first sight, extending and intersection are similar ways to combine types. However, the principle difference between the two is how conflicts are handled.

For interface, If you declare a property with the same key, it will overrides the one in the derived interface. However, it should be compatible with the previous one, or typescript compiler will throw an error.

// This works pretty well, as it is compatible
interface Converter {
  convert: (value: number) => string;
}

interface StringNumberConverter extends Converter {
  convert: (value: number | string) => string;
}

// Oops...this will throw an error, as it is incompatible
interface Converter {
  convert: (value: number) => string;
}

interface StringNumberConverter extends Converter {
  convert: (value: string) => string;
}

For type, properties with the same key don’t have to be compatible. You can declare whatever you like.

type Converter = {
  convert: (value: number) => string;
}

type StringNumberConverter = Converter & {
  convert: (value: string) => string;
}

Another difference you should pay attention is the meaning they represent. When I use interface, I want it stands for the「Parent-Child」relationship. By contrast, when I use type, it only stands for 「Composition」relationship.

// 「Parent-Child」relationship. Cat derives from Animal.
interface Animal {
  name: string;
}

interface Cat extends Animal {
  color: string;
}

// 「Composition」relationship. Color and Size are two seperate species
type Color = {
  name: string
}
type Size = {
  size: 'small' | 'medium' | 'large'
}

type ColorSize = Color & Size

Class Implements

A class can implement an interface or type alias, but it can not implement Union type, as in typescript, a class can only implement an object type or intersection of object types with statically known members.

So in daily development, if this type is designed to be implemented by class, I will use interface over type.

type PartialPoint = { x: number; } | { y: number; };

// can not implement a union type
class SomePartialPoint implements PartialPoint {
  x = 1;
  y = 2;
}

More than Object Types

Unlike an interface, type alias can also be used for other types. Frequently used are such as union type and tuples.

type StringNumberPair = [string, number];
function doSomething(stringHash: [string, number]) {
  const [inputString, hash] = stringHash;
  console.log(inputString); // string
  console.log(hash);  // number        
}

Declaration Merging

interface is open, and type alias is close within itself. An interface can be defined multiple times, and will be treated as a single interface with members of all declarations being merged.

type Bear = {
  name: string;
}
// Error：Duplicate Identifier
type Bear = {
  age: number;
}

// works pretty well
interface Bear {
  name: string;
}

interface Bear {
  age: number;
}

const b: Bear = {
  name: '',
  age: 0
}

Working With `typeof` Type Operator

In most cases, if you want to work with typeof type operator, choose type over interface.

function f() {
  return { x: 10, y: 3 };
}

type P = ReturnType<typeof f>;

/**
** type P = {
    x: number;
    y: number;
}
**/

Conclusion

When the next time you are confused which to use, try this cheatsheet!

Implement Dark Mode in Vue/React UI Component Library

Magenta Qin — Thu, 29 Dec 2022 13:00:16 +0000

Unlike other blogs only telling you how to implement Dark Mode using React Hooks to toggle theme, this blog will explain how Dark Mode will be implemented across the whole Design to Code stages from a bottom-top side. The stages include:

Design token and style variable naming conventions
Css variables and Javascript color variables
Global dark mode implementation
Customize component-level dark mode color

Design token and style variable naming conventions

Dark and Light mode share the same token name. Let's take the ProgressIndicator component as an example. The color of the circle changes with the Dark/Light mode, but they share the same Design Token--Fill.

Speaking of the css variable naming convention, we add prefix to differentiate them.

For the color that will change with the mode, we name Themeprefix.

// dark/index.styl
$Fill = rgba(255, 255, 255, 0.2)
$SecondaryFill = rgba(255, 255, 255, 0.1)

// light/index.styl
$Fill = rgba(0, 0, 0, 0.08)
$SecondaryFill = rgba(0, 0, 0, 0.04)

// theme/index.styl
$ThemeFill = var(--ThemeFill, $Fill)
$ThemeSecondaryFill = var(--ThemeSecondaryFill, $SecondaryFill)

For the colors that will not change with the dark or light mode, we name Always as the prefix of the Token.

// always/index.styl
$AlwaysWhiteFill = rgba(255, 255, 255, 0.99)

CSS variables and Javascript color variables

As we've already provided stylus variables, do we need css variables?

The answer is YES.

Stylus variables are a part of CSS preprocessors, and need to be compiled into CSS to be understood by the browser.

$Divider: #ddd;

.main-header {
  border: 1px solid $Divider;
}

For example, the code would be compiled into:

.main-header {
  border: 1px solid #ddd;
}

Once the code compiles, the variables are gone.

CSS variables, however, is natively supported within CSS. It doesn’t need to be compiled, and can be directly used.

:root {
  --Divider: #ddd;
}

.main-header {
  border: 1px solid var(--Divider);
}

So what is the biggest advantage of CSS variables vs preprocessor variables?

Through CSS variables, styles are changed in runtime instead of compilation time. This will bring following benefits:

BENEFIT 1: Enable us to reset style in runtime, which is impossible with preprocessor variables.

For example, in grid layout, media query could change the variables:

:root {
  --width: 30%;
}
@media (max-width: 450px) {
  :root {
    --width: 60%;
  }
}

Another example is that if you use preprocessor variables to switch UI mode, you have to provide two different preprocessor variables to the same value, and you have to write many duplicated code to tell the browser to use the right color, which enlarges the code size and make it hard to manage styles. e.g.

// dark mode
$DarkDivider: #eee;

// light mode
$LightDivider: #ddd;

// component example
.divider {
    border: 1px solid $LightDivider;
  &.dark {
    border: 1px solid $DarkDivider;
  }
}

However, if you use css variables, you don’t have to provide two different variables for the same value. And you don’t have to write duplicated code. When you switch class name, In light mode, browser will use #ddd, and in dark mode, browser will use #eee.

// light style
--Divider: #ddd;

// Dark style
--Divider: #eee;

// index.css
:root, .light-mode {
  --Divider: #ddd;
}

:root, .dark-mode {
  --Divider: #eee;
}

BENEFIT 2: You could manipulate them in JavaScript. In component library, you could only allow users to pass Color Token instead of pass any rgba or hex values, which brings chaos and hard to manage styles.

// colors.js
const ThemeGray = `var(--ThemeGray)`

// demo.vue
textStyle.color = colors.ThemeGray

Despite differences between preprocessor variables and less variables, they can work together. Let’s have a look!

Global dark mode implementation

In most time, we apply theme changes globally. You could either implement in the component library like Vant, or recommend your users to use ready-made solution like @nuxtjs/color-mode
Let’s see how they are working under the hood.

Vant Solution—ConfigProvider

To enable ConfigProvider, you should register component globally.

import { createApp } from 'vue';
import { ConfigProvider } from 'vant';

const app = createApp();
app.use(ConfigProvider);

Then you can enable dark mode like this:

<van-config-provider theme="dark">
  ...
</van-config-provider>

ConfigProvider is essentially a component. It will watch the theme props, and dynamically add or remove class to document.documentElement.

Vant provides css variables for dark theme. When you add class van-them-black, it will use these css variables to replace light theme css variables.

Nuxt Solution—ColorMode Module

Nuxt module enables us to share custom solutions as npm packages without adding unnecessary boilerplate.

@nuxtjs/color-mode mainly provide two plugins (plugin.clientand plugin.server）and one script. Let’s see how they are working together.

To be mentioned, on the server side, it will use preference options color as initial colorMode, and the colorMode class has not been added to the template, and the css variables are loaded asynchronously. So like below, even you set the app to be dark mode, you will see the preview is still in light mode.

So when will the dark mode class be added?

On the client side, the browser begins to execute script. It gets the exact colorMode from localStorage or system color or the forced set color. Then it will set className to document.documentElement, which will let the browser know to use the relevant mode css variables.

Then, client plugin begins to watch color mode changes from user and watch colorMode changes from the router options.

Customize component-level dark mode color

Instead of using the dark mode color provided by the component library, sometimes we want to customize the dark mode color for certain component.

The easiest way to do this is to wrap dark mode class outside the component class. You can directly use dark-mode class, but it would be a little bit disordered, as dark-mode class is different from other classes.

Why don’t we provide a stylus function prefers-color-schemeto users?

prefers-color-scheme(scheme)
  themes = light dark
  error('invalid color scheme: <light | dark>') unless scheme in themes

  if scheme == light
    &, .light-mode &
      {block}
  else if scheme == dark
    .dark-mode &
      {block}

When we want to customize the dark mode color for certain component, we simply use:

.reds-button-new.outlined
  +prefers-color-scheme(dark)
    background-color red

This will greatly improve the code readability.

That’s all. Feel free to ask me any questions about dark mode.

DEV Community: Magenta Qin

Loop Engineering Visualized: A World Cup Fan Journey with SerpApi and Codex

The Demo

Pipeline vs Loop

Mapping the Demo to the Loop

Iteration 1: Start with a Match

Iteration 2: Following Mbappé Changes the Goal

Iteration 3: The Loop Continues

Iteration 4: Another Direction, Same Loop

Final Iteration: Stop and Generate the Trace

Why SerpApi is Useful

Building It with Codex

Two Failures That Made the Loop Clearer

1. The First Graph Showed Relationships, Not Iterations

2. Search Relevance Is Not Factual Correctness

What I Would Improve Next

1. Unify the Loop State Model

2. Add Production Guardrails

3. Persist the Journey Across Sessions

The Main Takeaway

How I Built a Star Wars Grogu Product Research Agent with Codex, Lark, and SerpApi

Running the Agent from Codex

The Workflow

Translating Natural Language into Commands

Searching Amazon with SerpApi

Modeling Product Data

Keeping the Agent Narrow

Generating Seller-Friendly Insights and Delivering Them to Lark

Error Handling

Conclusion

Using SerpApi and DeepSeek to Break Down Dan Koe’s Content Strategy

The Engineering Approach

Collecting YouTube Data with SerpApi

Searching Videos

Fetching Video Details

Fetching Transcripts

Composing Three Fetch Functions Into One Orchestration Layer

Using DeepSeek to Analyze Creator Strategy

Switching to a Two-Step Analysis Pipeline

Designing Structured AI Contracts

Step 1 Prompt: Per-Video Semantic Compression

Step 2 Prompt: Cross-Video Strategy Analysis

Parallelizing the Pipeline

Calling the DeepSeek API

Parsing and Normalizing the AI Response

What the Pipeline Found

Use which: Interface or Type Alias in Typescript?

Describe the shape of an object or a function signature

Extending and Intersection Types

Class Implements

More than Object Types

Declaration Merging

Working With typeof Type Operator

Conclusion

Implement Dark Mode in Vue/React UI Component Library

Design token and style variable naming conventions

CSS variables and Javascript color variables

Global dark mode implementation

Vant Solution—ConfigProvider

Nuxt Solution—ColorMode Module

Customize component-level dark mode color

Working With `typeof` Type Operator