Automating GSC Gap Analysis to Publishing: 15 Queries, 1 Pipeline

Originally published on AIdeazz — cross-posted here with canonical link.

My first attempt at an AI content pipeline failed to move the needle. I spent $120 on Claude 3 Opus in a month, generating 30 articles, and saw exactly zero impact on organic traffic. The problem wasn't the LLM output quality; it was the input. My "content gap analysis" was a manual scan of Google Search Console (GSC) for queries with impressions but low clicks, then picking what felt right. This approach is a coin flip, not a strategy.

The real constraint wasn't the AI's ability to write, but my ability to identify what content would actually perform. With 15 GSC queries showing impressions but no clicks, "content gap" isn't a vague concept; it's a specific, measurable opportunity. I needed a system that could identify these gaps programmatically, generate content, and publish it, all without my direct intervention. This is how I built it for AIdeazz, running on Oracle Cloud Infrastructure (OCI) with Groq and Claude 3 Opus, publishing to Dev.to and caching on my own site.

Identifying the True Content Gap: Beyond Manual GSC Scans

The initial failure taught me that "content gap" is not a subjective feeling. It's a data point. Specifically, it's a GSC query where impressions > 0 and clicks = 0 for a given URL, or a query with high impressions and a low click-through rate (CTR) where existing content isn't adequately addressing the user intent.

My agent, GSC_Analyzer_Agent, pulls data daily from GSC via its API. I focus on queries for aideazz.xyz and dev.to/aideazz. The agent filters for:

1. Queries with impressions > 50 and clicks = 0 over the last 30 days.
2. Queries with impressions > 100 and CTR < 1% where the ranking URL isn't directly optimized for that query.

For each identified gap, the agent extracts the query, the current ranking URL (if any), and the average position. This output, a structured JSON object containing {'query': '...', 'current_url': '...', 'avg_position': '...'} for each gap, becomes the prompt for the next stage. This is where the "15 GSC queries" become actionable tasks, not just data points.

Orchestrating LLMs for Article Generation

The core of the content generation is a multi-agent system. I use a custom router to select the LLM based on task complexity and cost. For initial brainstorming and outline generation, I often route to Groq for speed and lower cost. For the full article draft, Claude 3 Opus is my current preference due to its long context window and nuanced writing style, especially for technical topics.

The Article_Generator_Agent receives the gap analysis output. For each gap, it performs these steps:

1. Outline Generation (Groq): Prompt: "Generate a detailed outline for an article targeting the keyword '[query]'. Include 5-7 main sections and 3-5 sub-sections per main section. Focus on practical advice for developers."
2. Drafting (Claude 3 Opus): Prompt: "Write a comprehensive, technical article based on the following outline: [outline JSON]. Incorporate specific examples, code snippets (if applicable), and address common pitfalls. The tone should be direct and practitioner-focused. Target audience: experienced developers and technical founders. Word count: 1500-2000 words."
3. SEO Optimization (Claude 3 Haiku): Once the draft is ready, a SEO_Optimizer_Agent takes over. It uses Claude 3 Haiku (cheaper, faster for smaller tasks) to review the draft against the target query. It suggests title improvements, meta description, internal linking opportunities, and keyword density adjustments. This agent also generates 3-5 relevant questions and answers for the FAQ section, mimicking the structure I use in my own articles.

This multi-LLM approach allows me to optimize for cost and performance. Groq handles the fast, iterative tasks, while Claude 3 Opus delivers the high-quality, long-form content.

Automated Publishing to Dev.to and AIdeazz.xyz

Publishing is handled by the Publisher_Agent. This agent is responsible for two main tasks:

1. Dev.to API Integration: Dev.to offers a robust API for publishing articles. The agent converts the generated Markdown into the required JSON format, adds appropriate tags (derived from the article content and target query), and sets the published status to true. Error handling is critical here. I've implemented retries with exponential backoff for 429 Too Many Requests errors and specific logging for 401 Unauthorized (API key issues) or 422 Unprocessable Entity (malformed content).
2. AIdeazz.xyz Caching: While Dev.to is a great platform for reach, I also want the content on my own site for long-term SEO benefits and control. The agent uses a simple POST request to an internal API endpoint on aideazz.xyz. This endpoint receives the Markdown content, generates a unique slug, and stores it in my Oracle Autonomous Database. I use a lightweight SvelteKit frontend to render these cached articles. This ensures that even if Dev.to changes its policies or goes down, I retain ownership of the content.

The entire process, from GSC gap identification to automated publishing, is orchestrated by a central Python script running as a cron job on an OCI compute instance. It's designed to be idempotent; if an article for a specific GSC gap has already been published, it skips it.

Infrastructure and Cost Considerations

Running this pipeline requires careful resource management.

• OCI Compute: A single VM.Standard.E4.Flex instance (1 OCPU, 16GB RAM) is sufficient. Cost: ~$50/month.
• Oracle Autonomous Database: I use the Always Free tier for storing GSC data, article outlines, and published content metadata. This is a critical component for persistent storage without direct cost.
• LLM APIs: This is the variable cost.
  • Claude 3 Opus: ~$15/M tokens input, ~$75/M tokens output. A 2000-word article (approx. 3000 tokens) costs about $0.30-$0.50 per article for drafting.
  • Claude 3 Haiku: ~$0.25/M tokens input, ~$1.25/M tokens output. SEO optimization for a 2000-word article costs pennies.
  • Groq: ~$0.13/M tokens input, ~$0.13/M tokens output for LLaMA3 8B. Outline generation is extremely cheap, often less than $0.01.
• GSC API: Free.
• Dev.to API: Free.

My total monthly cost for this automated content pipeline, generating 10-15 articles, is currently around $60-$80, primarily driven by LLM usage. This is a significant improvement over the $120 I spent previously for zero results. The key is the precision in targeting actual content gaps, not just generating content for the sake of it.

The "Content Gap" is a Feedback Loop

The initial failure wasn't a failure of AI, but a failure of process. The "content gap" isn't a static target; it's a dynamic feedback loop. The GSC_Analyzer_Agent continuously monitors performance. If a newly published article for a specific query starts gaining impressions but still has low clicks, it triggers a re-evaluation. The system can then:

1. Suggest revisions: The SEO_Optimizer_Agent can be prompted to analyze the existing article and suggest specific improvements to title, meta description, or content sections to improve CTR.
2. Generate follow-up content: If the initial article only partially addresses a broad query, the system can identify sub-gaps and generate more focused articles.

This iterative process is crucial. Automated publishing isn't a "set it and forget it" solution; it's a "set it and continuously optimize it" system. The goal isn't just to publish, but to publish effectively.

Frequently Asked Questions

Q: How do you handle potential duplicate content issues between Dev.to and your own site?
A: Dev.to allows canonical URLs. My Publisher_Agent sets the canonical_url field in the Dev.to API request to point to the article's URL on aideazz.xyz. This signals to search engines that aideazz.xyz is the primary source.

Q: What if the LLM generates factually incorrect or outdated information for a technical topic?
A: This is a critical risk. For highly sensitive technical content, I have a human review step for the Article_Generator_Agent's output before it's passed to the Publisher_Agent. For less critical topics, I rely on the LLM's general knowledge and prompt engineering to emphasize accuracy and cite sources where possible. I'm also experimenting with RAG (Retrieval Augmented Generation) to ground LLM responses in specific, verified documentation.

Q: How do you prevent the pipeline from publishing low-quality or irrelevant articles?
A: The quality starts with the GSC_Analyzer_Agent's precision in identifying relevant gaps. The Article_Generator_Agent uses detailed prompts for structure and tone. I also implement a simple content quality score based on readability metrics (e.g., Flesch-Kincaid) and keyword density, which can trigger a re-draft by the LLM if the score is too low.

Q: What's the biggest challenge you faced in building this automated pipeline?
A: Orchestrating the different LLMs and ensuring smooth data flow between agents was the biggest challenge. Each LLM has its own API, rate limits, and prompt engineering nuances. Building a robust router and error handling for each API call took significant effort.

Q: How do you manage the LLM API keys securely?
A: All API keys are stored as environment variables on the OCI compute instance and accessed via a secure secrets management service (OCI Vault). They are never hardcoded in the application logic.

— Elena Revicheva · AIdeazz · Portfolio