Automating GSC Gap Analysis to Publishing: 15 Queries, 1 Pipeline

Originally published on AIdeazz — cross-posted here with canonical link.

My blog traffic stalled. For six months, aideazz.xyz hovered at 1,200 organic clicks per month, despite consistent manual publishing. My content strategy, if you could call it that, was "write about whatever I just built." It wasn't working. I needed a system that could identify what Google wanted me to write about, draft it, and publish it, all without my direct intervention. The goal: a fully automated AI content pipeline, from Google Search Console (GSC) gap analysis to automated publishing on Dev.to and my own site.

Defining "Content Gap" with 15 GSC Queries

The term "content gap" is often used loosely. For a site like mine, with a modest 1,200 organic clicks/month, it doesn't mean "missing an entire pillar topic." It means I'm ranking on page 3 for a query where I should be on page 1, or I'm getting impressions for a query with zero clicks. My initial GSC export was 1,500 queries. Manually sifting through that is a full-time job.

My agent, GSC_Analyzer, was built to identify these micro-gaps. It pulls GSC data via the API, specifically focusing on query, clicks, impressions, position, and page. The core logic for identifying a "gap" is a combination of three factors:

1. High Impressions, Low Clicks: Queries with >50 impressions and <5 clicks. This indicates Google thinks my content is relevant, but users aren't clicking. My current content isn't satisfying their intent.
2. Position 11-30: Queries where my average position is between 11 and 30. These are "on the cusp" queries. A targeted article could push them to page 1.
3. Zero Clicks, High Impressions for Relevant Pages: Queries with 0 clicks but >100 impressions, where the associated page is broadly relevant to my site's focus (e.g., "Oracle Cloud AI," "Groq API," "multi-agent systems").

The agent filters the 1,500 queries down to a manageable 15-20 "gap" candidates per week. This is the critical step. Without this aggressive filtering, the downstream LLM costs explode, and the output quality plummets.

Orchestrating the Drafting Agent with Claude 3.5 Sonnet

Once GSC_Analyzer identifies the top 15 queries, it passes them to Article_Drafter. I initially tried using a single prompt for all 15 queries, but the results were generic and unfocused. The solution was a multi-agent approach, where Article_Drafter acts as an orchestrator.

For each of the 15 identified queries, Article_Drafter performs the following:

1. SERP Analysis: It uses a custom Python script to scrape the top 10 search results for the target query. This provides real-time competitive analysis, identifying common themes, headings, and entities. I'm not using a commercial API for this; a simple requests and BeautifulSoup setup works for now, with IP rotation handled by Oracle Cloud's network policies.
2. Content Brief Generation: Based on the SERP analysis and the GSC query itself, it generates a detailed content brief. This brief includes:
   • Target keyword (the GSC query)
   • Target audience (developers, technical founders)
   • Key themes to cover (extracted from top-ranking articles)
   • Headings/subheadings (suggested structure)
   • Entities to mention (e.g., "Oracle Cloud Infrastructure," "Groq LPU," "Anthropic Claude")
   • Tone (technical, practical, no hype)
   • Word count target (800-1500 words)
3. LLM Call (Claude 3.5 Sonnet): The brief is then sent to Claude 3.5 Sonnet. I chose Sonnet for its balance of cost and quality for long-form content generation. Opus is too expensive for this volume, and Haiku often struggles with the depth required. The prompt explicitly states the need for a skeptical, practitioner-focused tone and to avoid clichés. It also includes a "negative prompt" section, listing phrases and concepts to avoid.
4. Draft Review & Iteration: The initial draft is then passed through a Critique_Agent. This agent checks for:
   • Adherence to the brief (e.g., word count, headings covered)
   • Technical accuracy (a basic check against a knowledge base of my previous articles and common technical terms)
   • Readability (Flesch-Kincaid score target of 40-60)
   • Presence of negative prompt terms. If the draft fails any of these checks, it's sent back to Claude 3.5 Sonnet with specific revision instructions. This loop typically runs 1-2 times.

The average cost per article draft, including SERP analysis and 1-2 critique cycles, is $0.85. This is a critical metric. If it exceeded $2, the economics of automated publishing would break down for my current traffic levels.

Automated Publishing to Dev.to and aideazz.xyz

Once Article_Drafter produces a satisfactory draft, it's handed off to Publisher_Agent. This agent has two primary functions:

1. Dev.to API Integration: It uses the Dev.to API to create a new post. The article content is converted to Markdown, a relevant tag is selected (e.g., ai, oraclecloud, llm), and a canonical URL pointing back to aideazz.xyz is set. This is crucial for SEO, preventing duplicate content penalties. The published flag is set to true.
2. aideazz.xyz Caching and Display: The article is also stored in my Oracle Autonomous Database. My aideazz.xyz site is a static site generated by Hugo, but I have a small API endpoint that pulls these articles from the database and renders them as dynamic content. This allows me to have a "fresh" section on my site without rebuilding the entire static site for every new article. The article is then immediately available on aideazz.xyz as well.

The entire process, from GSC data pull to article publication, takes approximately 30-45 minutes per article. I currently run this pipeline once a week, generating 1-2 articles based on the highest-priority GSC gaps.

Infrastructure: Oracle Cloud, Groq, and Custom Agents

My entire AI content pipeline runs on Oracle Cloud Infrastructure (OCI).

• Compute: OCI Ampere A1 Compute instances for running Python agents. These are cost-effective for CPU-bound tasks like GSC data processing and orchestrating LLM calls.
• Database: Oracle Autonomous Database (Always Free Tier) for storing GSC data, article drafts, and published content. This provides a robust, serverless backend.
• Networking: OCI VCNs and security lists to manage API access and IP rotation for SERP scraping.
• LLM Routing: I use a custom router to direct LLM calls. While Claude 3.5 Sonnet is the primary model for drafting, I've experimented with Groq for faster, shorter tasks like initial brief generation or quick summaries. The router dynamically selects the LLM based on task requirements and cost constraints. Groq's LPU is incredibly fast, but its context window and reasoning capabilities aren't yet on par with Claude for complex drafting.
• Agent Framework: My agents are custom Python scripts, not a commercial framework. I started with a simple if/else structure and gradually modularized it into distinct agents (GSC_Analyzer, Article_Drafter, Critique_Agent, Publisher_Agent) communicating via a shared database and message queue (OCI Streaming).

The total infrastructure cost for this pipeline, excluding LLM API calls, is under $15/month on OCI, largely due to leveraging the Always Free Tier and Ampere A1's efficiency.

The Results: A 30% Increase in Organic Traffic

After three months of running this automated pipeline, aideazz.xyz organic traffic has increased by 30%, from 1,200 to 1,560 clicks per month. This isn't a "hockey stick" growth, but it's consistent, sustainable growth without my direct writing effort. The articles generated by the pipeline are ranking for the targeted GSC queries, often moving from page 2-3 to page 1.

The most valuable lesson: "content gap" is a precise, data-driven concept, not a vague marketing term. For a small site, it's about optimizing existing impressions and positions, not chasing broad, competitive keywords. And automation, even with its initial setup cost, is the only way to scale this kind of granular optimization.

Frequently Asked Questions

Q: How do you prevent the AI from generating repetitive or low-quality content?
A: The Critique_Agent is crucial. It enforces specific quality metrics (readability, adherence to brief, negative prompt terms) and sends drafts back for revision. Also, the GSC gap analysis ensures each article targets a distinct, data-backed need.

Q: What if the AI drafts something factually incorrect or misleading?
A: For highly sensitive topics, manual review is still necessary. For my technical blog, the Critique_Agent performs basic accuracy checks against a knowledge base. However, a human eye is still the ultimate arbiter for critical factual correctness. I spot-check 1 in 5 articles.

Q: Why not use a commercial content automation platform?
A: Cost and control. Commercial platforms often have high monthly fees and less flexibility in agent orchestration, LLM routing, and custom logic for GSC analysis. Building it myself on OCI allows me to optimize for my specific constraints and keep costs minimal.

Q: How do you handle image generation or other media for the articles?
A: Currently, the pipeline only generates text. Images are added manually if needed, or I use placeholder images. Integrating an image generation model (e.g., DALL-E 3) is a future enhancement, but it adds significant cost and complexity.

Q: What's the biggest challenge you faced in building this pipeline?
A: Defining the "content gap" logic precisely enough to yield high-quality, actionable queries without overwhelming the LLM. Iterating on the GSC filtering criteria took the most time and experimentation.

— Elena Revicheva · AIdeazz · Portfolio