DEV Community

Cover image for Don't Generate Every Story: Put a Relevance Gate Before AI Content Creation

Don't Generate Every Story: Put a Relevance Gate Before AI Content Creation

A content automation pipeline can become wasteful surprisingly early.

Imagine pulling stories from several external sources.

Every new item arrives.
Every item gets summarized.
Every item gets sent through an LLM.
Every item gets a social draft.
Maybe an image prompt is generated too.
Then somebody looks at the output and realizes half of it was never useful enough to publish.

The problem started much earlier than generation.

The system never decided which stories deserved generation in the first place.

A cleaner pipeline adds a relevance gate before expensive AI work begins.

Do not make generation the first decision

A simple content pipeline often starts like this:

Sources
   ↓
Collect items
   ↓
Generate article
   ↓
Generate social copy
   ↓
Review
Enter fullscreen mode Exit fullscreen mode

It works.

It is also doing expensive work before asking whether that work is needed.

A better sequence is:

Sources
   ↓
Collect items
   ↓
Normalize
   ↓
Relevance check
   ↓
Duplicate check
   ↓
Candidate queue
   ↓
Generate
   ↓
Review
Enter fullscreen mode Exit fullscreen mode

Now generation is reserved for candidates that have already passed cheaper checks.

Normalize before you score

Different publishers rarely describe content the same way.

One source may call the main field title.

Another returns headline.

Another has rich category metadata.

Another provides almost none.

Before relevance logic runs, create a consistent internal object.

A simplified model could look like:

type SourceItem = {
  id: string;
  sourceId: string;
  headline: string;
  summary?: string;
  url: string;
  canonicalUrl?: string;
  publishedAt?: string;
  categories: string[];
  body?: string;
};
Enter fullscreen mode Exit fullscreen mode

The exact fields will vary.

The goal is more important than the schema:

relevance logic should work against one predictable representation instead of learning every source format.

Separate cheap checks from expensive checks

Not every relevance decision needs an LLM.

Some candidates can be eliminated with inexpensive rules.

For example:

function passesBasicRules(item: SourceItem) {
  if (!item.headline) return false;

  if (isTooOld(item.publishedAt)) return false;

  if (blockedSources.has(item.sourceId)) return false;

  if (containsExcludedCategory(item.categories)) return false;

  return true;
}
Enter fullscreen mode Exit fullscreen mode

This is not sophisticated.

It does not need to be.

Every obviously irrelevant item removed here is an item that never needs a model call later.

Then score the candidates that remain

After basic filtering, assign relevance deliberately.

A content operation may care about things such as:

  • geography
  • topic
  • company or entity
  • business category
  • recency
  • strategic priority
  • audience fit

A simple scoring interface could look like:

type RelevanceScore = {
  score: number;
  reasons: string[];
};

async function scoreRelevance(
  item: SourceItem
): Promise<RelevanceScore> {
  // Rule engine, classifier, model call,
  // or a combination of them.
  return {
    score: 82,
    reasons: [
      "target geography",
      "relevant property topic",
      "recent publication"
    ]
  };
}
Enter fullscreen mode Exit fullscreen mode

The number itself is not magic.

What matters is making the decision explicit.

Now the workflow can say:

if (relevance.score < MIN_GENERATION_SCORE) {
  await markSkipped(item.id, relevance);
  return;
}
Enter fullscreen mode Exit fullscreen mode

The system knows why generation did not happen.

That is much easier to operate than silently dropping items somewhere downstream.

Deduplication should happen before generation too

External content feeds repeat stories constantly.

The same development may appear:

  • on the original publisher
  • in a syndicated copy
  • in another outlet's rewrite
  • in an updated version
  • under a slightly different headline

If all of those reach the generation layer, the model produces several variations of essentially the same content.

That wastes model calls and fills the review queue with duplicates.

A duplicate gate can use several signals:

Canonical URL
Headline similarity
Entity overlap
Publication time
Source relationship
Body similarity
Enter fullscreen mode Exit fullscreen mode

A simplified decision could be:

const duplicate = await duplicateDetector.findMatch(item);

if (duplicate) {
  await markDuplicate(item.id, duplicate.id);
  return;
}
Enter fullscreen mode Exit fullscreen mode

Again, the benefit is not just cost.

The editorial team receives a cleaner candidate queue.

Build a candidate queue, not a generation queue

Once an item has passed:

  • normalization
  • basic rules
  • relevance scoring
  • duplicate detection

it becomes a candidate.

That distinction is useful.

A candidate does not automatically mean:

Generate everything now.

It means:

This item is worth considering for content creation.

You can model that explicitly:

type Candidate = {
  itemId: string;
  relevanceScore: number;
  status:
    | "candidate"
    | "approved_for_generation"
    | "skipped"
    | "generated";
};
Enter fullscreen mode Exit fullscreen mode

Now editorial policy has somewhere to live.

Some candidates can still wait

Suppose six useful stories arrive within ten minutes.

Generating all six immediately may not make sense.

The operation might only need:

  • two website stories today
  • three social posts
  • one weekly summary

So rank the candidate queue.

Candidate A   94
Candidate B   88
Candidate C   84
Candidate D   79
Candidate E   74
Candidate F   71
Enter fullscreen mode Exit fullscreen mode

Generation capacity can then follow editorial priority.

This creates a much healthier relationship between automation and volume.

The system is no longer asking:

What can we generate?

It is asking:

What is worth generating now?

Make rejection observable

A skipped story should not disappear into nowhere.

Store the decision.

For example:

{
  "itemId": "story_482",
  "decision": "skipped",
  "reason": "low_relevance",
  "score": 41,
  "evaluatedAt": "2026-08-15T10:30:00Z"
}
Enter fullscreen mode Exit fullscreen mode

Or:

{
  "itemId": "story_517",
  "decision": "duplicate",
  "matchedItemId": "story_503"
}
Enter fullscreen mode Exit fullscreen mode

This gives operators useful answers later.

Why did this story not get generated?

Why did the queue suddenly become small?

Why are two publishers producing similar candidates?

Without decision records, automation becomes difficult to explain.

Generation should receive context, not raw source data

When a candidate finally reaches generation, the model should not be forced to rediscover everything the pipeline already knows.

Pass structured context.

For example:

type GenerationContext = {
  sourceItem: SourceItem;
  relevance: RelevanceScore;
  targetAudience: string;
  outputType: "article" | "social" | "summary";
  editorialNotes?: string[];
};
Enter fullscreen mode Exit fullscreen mode

Now generation starts from a much cleaner state.

The workflow already knows:

  • what the item is
  • why it matters
  • what output is needed
  • who the output is for

That usually improves consistency as much as it reduces wasted processing.

Heavy work belongs behind the gate

Generation is rarely the last expensive operation.

A content platform may also run:

  • translation
  • asset creation
  • rendering
  • enrichment
  • image processing
  • scheduled publishing

Those jobs can live behind worker queues.

The pipeline then becomes:

Source intake
      ↓
Normalize
      ↓
Filter
      ↓
Score
      ↓
Deduplicate
      ↓
Candidate
      ↓
Generate
      ↓
Human review
      ↓
Approved?
      ↓
Background workers
      ↓
Schedule / publish
Enter fullscreen mode Exit fullscreen mode

The expensive path starts only after the content has earned its way there.

Measure the funnel

Once the gates exist, measure them.

Useful numbers might include:

Items collected
↓
Items passing basic rules
↓
Relevant candidates
↓
Unique candidates
↓
Generated drafts
↓
Approved drafts
↓
Published items
Enter fullscreen mode Exit fullscreen mode

This tells you much more than:

We generated 5,000 pieces this month.
Enter fullscreen mode Exit fullscreen mode

A large generation count can hide a very inefficient pipeline.

A smaller generation count with a strong approval rate may be healthier.

A recent system where this mattered

We recently worked on an AI Real Estate Content Operations Platform that brought source intake, relevance scoring, deduplication, AI content generation, review, scheduling, and background processing into one operational workflow.

The goal was not to push every incoming source item directly into generation.

The system needed a controlled path from external information to content that was actually useful enough to review and publish.

The public project breakdown is here:

Related work:
https://ascentinnovate.com/work/ai-real-estate-content-operations-platform

Generate later than you think

When teams add AI to content operations, generation is naturally the feature that gets attention.

But a good pipeline makes several decisions before the model starts writing.

  • Is the source usable?
  • Is the story relevant?
  • Have we already covered it?
  • Is it worth producing now?
  • Which output does it actually need?

Those gates protect model spend, editorial attention, and publishing quality at the same time.

Do not generate everything you can collect.

Generate the things that have already earned a place in the queue.

Top comments (0)