GSC Gap Analysis: From 15 Queries to Automated Publishing

Originally published on AIdeazz — cross-posted here with canonical link.

My first attempt at an AI content pipeline failed to generate a single useful article for AIdeazz.xyz. The "gap analysis" was the problem. I fed Google Search Console (GSC) data directly to Claude, asking it to identify content gaps. Claude, being an LLM, dutifully hallucinated "gaps" based on the keywords I provided, often suggesting articles on topics I already covered or that were irrelevant to my niche. The output was generic, unpublishable fluff. The core issue: I treated GSC as a direct prompt source, not a data signal requiring interpretation by a specialized agent.

The "Gap" is Not a Missing Keyword

When GSC shows 15 queries for a topic, it doesn't mean you're missing an article. It means you're already ranking for something related to those queries, but likely not optimally. A true "gap" isn't a missing keyword; it's a missing intent fulfillment. My initial approach was to ask Claude, "What content should I write based on these 15 queries?" This is like asking a chef to build a house based on a list of ingredients. The chef knows ingredients, not construction.

The solution was to build a dedicated GSC analysis agent. This agent doesn't just look at keywords; it looks at clusters of keywords, their average position, click-through rate (CTR), and impressions. It then cross-references these clusters with my existing content.

Here's the logic:

1. Extract GSC Data: Daily export of queries, pages, impressions, clicks, CTR, and position.
2. Cluster Queries: Group similar queries using a simple TF-IDF vectorization and K-means clustering. For example, "Oracle Cloud AI agents," "Groq LLM deployment," "AI production pipeline" might cluster together.
3. Map to Existing Content: For each cluster, identify which existing articles on AIdeazz.xyz (if any) are already ranking for queries within that cluster. I use a simple cosine similarity between the cluster's centroid vector and the vector representation of my article titles and first paragraphs.
4. Identify Underperforming Clusters: A cluster is a "gap" if:
   • It has high impressions (e.g., >500 in the last 30 days).
   • Its average position is between 8 and 20 (meaning I'm on page 1 or 2, but not top 3).
   • Its CTR is below 1.5% for positions 8-10, or below 0.8% for positions 11-20.
   • Crucially: There is no existing article with a high similarity score (e.g., >0.7) to the cluster, or the existing article is also underperforming for that cluster.

This refined definition of a "gap" means I'm targeting areas where I already have some authority but am failing to capture clicks, or where I have a nascent presence that could be amplified. It's about optimization, not just creation.

The Multi-Agent Orchestration

My content pipeline is a series of specialized agents, not a single monolithic LLM call. It runs on Oracle Cloud Infrastructure (OCI) using OCI Functions and a PostgreSQL database for state management.

1. GSC Agent (Python, OCI Function):

   • Connects to GSC API, pulls data.
   • Performs query clustering and gap analysis as described above.
   • Identifies 1-3 top "gap" topics daily.
   • For each topic, it generates a concise prompt for the outlining agent: "Write an article outline for [topic cluster description] targeting users searching for [top 3 queries in cluster]. Focus on [specific angle derived from low CTR/position analysis]."
   • Stores the generated prompt and GSC context in the database.

2. Outlining Agent (Claude 3 Haiku, OCI Function):

   • Receives the prompt from the GSC agent.
   • Generates a detailed article outline (H2s, H3s, key points for each section).
   • The prompt includes instructions like: "Use a skeptical, practitioner-focused tone. Include specific numbers, error messages, and cost figures. Avoid startup clichés."
   • Stores the outline in the database, linked to the GSC context.

3. Drafting Agent (Claude 3 Sonnet, OCI Function):

   • Receives the outline.
   • Generates the full article draft, section by section. I found Sonnet to be a good balance of quality and cost for drafting. Haiku often missed nuance, Opus was overkill for a first draft.
   • Crucially, it's instructed to not invent facts or numbers. If it needs a specific number or example, it inserts a placeholder like [NEED: specific Groq cost per 1M tokens for Llama 3 8B]. This forces a human review step and prevents factual errors.
   • Stores the draft in the database.

4. Review & Enrichment Agent (Human, Telegram Bot):

   • A Telegram bot notifies me when a draft is ready.
   • I review the draft, fill in placeholders, add personal anecdotes, and refine the tone. This is where the "Elena Revicheva" voice comes in.
   • I also check for factual accuracy and ensure it meets the "no startup clichés" rule.
   • This step is non-negotiable. Fully automated content is detectable and often lacks the unique perspective that builds trust.

5. Publishing Agent (Python, OCI Function):

   • Once I approve the article in the Telegram bot, this agent takes over.
   • Dev.to API: Publishes the article to my Dev.to profile. Dev.to is a great platform for reaching developers, and their API is straightforward. I include canonical URLs pointing back to AIdeazz.xyz.
   • AIdeazz.xyz Cache: The article is stored in a static HTML file on my Oracle Object Storage bucket, which serves as the backend for AIdeazz.xyz. This is a static site, so no complex CMS is needed. The publishing agent simply writes the HTML file.
   • Sitemap Update: Updates the sitemap.xml file on AIdeazz.xyz and pings Google Search Console for faster indexing.

Groq and Claude Routing: Cost vs. Quality

I route LLM calls dynamically. For outlining, where speed and cost are paramount, I initially tried Groq with Llama 3 8B. It was fast, but the outlines were often too simplistic. Claude 3 Haiku, while slower, produced significantly better structured and more detailed outlines, justifying the slightly higher cost.

For drafting, Claude 3 Sonnet is my current choice. I experimented with Groq for drafting, but the output quality for complex technical topics was inconsistent. It often required more human editing, negating the cost savings. Opus is too expensive for initial drafts, especially when I'm still iterating on the prompt engineering for the outlining agent.

The routing logic is simple:

• GSC Agent: No LLM. Pure Python.
• Outlining Agent: Claude 3 Haiku.
• Drafting Agent: Claude 3 Sonnet.
• Publishing Agent: No LLM. Pure Python.

This setup costs me roughly $0.50 - $1.50 per article for LLM calls, depending on article length. The OCI Functions are on the free tier for most of the execution time. The biggest cost is my own time for review and enrichment, which I've optimized to about 15-30 minutes per article.

The AIdeazz.xyz Caching Strategy

My website, AIdeazz.xyz, is a static site served directly from Oracle Object Storage. This is incredibly cheap and scales infinitely. When the publishing agent creates an article, it generates a static HTML file and uploads it to a specific bucket path (e.g., articles/gsc-gap-analysis.html).

This approach has several benefits:

• Zero Database Overhead: No MySQL, no PostgreSQL for the website itself.
• CDN-like Performance: Object Storage is globally distributed, offering low latency.
• Security: No dynamic code, no attack surface for SQL injection or XSS.
• Cost: Object Storage costs pennies per GB. My entire site is less than 100MB.

The publishing agent also updates a manifest.json file which my simple JavaScript frontend uses to list articles. This keeps the site dynamic in appearance without needing a backend server.

Real Constraints and Future Iterations

The biggest constraint is still the human in the loop. While the agents handle the heavy lifting, my unique voice and specific technical insights are what make the content valuable. I'm exploring ways to fine-tune a smaller LLM on my past articles and writing style to reduce my review time, but I'm wary of losing authenticity.

Another constraint is the quality of GSC data itself. It's aggregated and sometimes delayed. My "gap" analysis is only as good as the data it receives. I'm considering integrating other data sources like Ahrefs or Semrush, but that adds cost and complexity, which I'm avoiding as a bootstrapped operation.

The current pipeline generates 3-5 articles per week. This is a sustainable pace for me, allowing for quality control. Scaling beyond this would require more sophisticated prompt engineering to reduce human intervention, or accepting a lower quality bar, neither of which I'm eager to do.

Frequently Asked Questions

Q: Why not use a single, more powerful LLM like Claude Opus for the entire pipeline?
A: Cost and control. Opus is significantly more expensive. By using specialized agents with smaller, cheaper models (Haiku for outlining, Sonnet for drafting), I optimize for cost while maintaining quality for each specific task. It also allows for easier debugging and iteration on individual agent prompts.

Q: How do you handle factual accuracy and prevent hallucinations, especially for technical content?
A: The drafting agent is explicitly instructed to insert [NEED: specific fact] placeholders for any information it cannot confidently generate. This forces a human review step where I fill in the precise details, error messages, or cost figures. This prevents publishing incorrect information.

Q: What's the biggest challenge in maintaining this automated content pipeline?
A: Prompt engineering drift. As LLMs evolve or I refine my understanding of what makes good content, the prompts for the outlining and drafting agents need constant iteration. Without careful versioning and testing, output quality can degrade quickly.

Q: Why Oracle Cloud Infrastructure (OCI) instead of AWS/Azure/GCP?
A: Cost-effectiveness and personal experience. OCI's always-free tier for functions, object storage, and databases provides a robust foundation with zero monthly cost for my current scale. As a single mother bootstrapping AIdeazz, every dollar saved on infrastructure is critical. I also have prior experience with OCI, which reduced the learning curve.

Q: How do you ensure the content published on Dev.to and AIdeazz.xyz doesn't lead to duplicate content penalties from Google?
A: I use canonical URLs. The Dev.to posts include a canonical_url field pointing back to the original article on AIdeazz.xyz. This tells search engines that AIdeazz.xyz is the primary source, preventing any duplicate content issues.

— Elena Revicheva · AIdeazz · Portfolio