How to optimize Next.js project for AI search engines

#ai #nextjs #node #opensource

AI search engines like ChatGPT, Perplexity, Claude, and Gemini are becoming a major discovery channel. When someone asks them "recommend me a good product or service", they pull data from structured content, dedicated crawlers, and formats like llms.txt to generate answers.

Optimizing for this is called Generative Engine Optimization (GEO) – and it requires a different approach than traditional SEO.

In this guide, I'll show you how to optimize your Node.js or Next.js app for 16 AI search engines using an open-source TypeScript engine called GEO AI Core.

For any Node.js project:
npm install geo-ai-core

For Next.js specifically:
npm install geo-ai-next

Define your content

The simplest way is to pass your content directly:

import { createGeoAI } from 'geo-ai-core';

const geo = createGeoAI({
  siteName: 'My Store',
  siteUrl: 'https://example.com',
  provider: {
    Products: [
      { title: 'Widget', url: '/products/widget', description: 'A great widget' },
    ],
    Blog: [
      { title: 'Hello World', url: '/blog/hello', description: 'First post' },
    ],
  },
});

For dynamic data (CMS, database, API), implement the ContentProvider interface:

import { createGeoAI, type ContentProvider } from 'geo-ai-core';

class StrapiProvider implements ContentProvider {
  async getSections(options?: { locale?: string }) {
    return [
      { name: 'Products', type: 'product', resources: await fetchProducts() },
      { name: 'Blog', type: 'page', resources: await fetchPosts() },
    ];
  }
}

const geo = createGeoAI({
  siteName: 'My Store',
  siteUrl: 'https://example.com',
  provider: new StrapiProvider(),
  crawlTracking: true,
});

Generate llms.txt

llms.txt is a structured file that helps AI crawlers understand your site – like a sitemap, but designed for LLMs.

// Standard version — resource links with descriptions
const llmsTxt = await geo.generateLlms(false);

// Full version — includes content, pricing, availability, variants
const llmsFullTxt = await geo.generateLlms(true);

Add SEO signals for AI

// Meta tags for AI content discovery
const metaTags = geo.generateMetaTags();

// HTTP Link header
const linkHeader = geo.generateLinkHeader();

// JSON-LD (WebSite, Product, Article)
const jsonLd = geo.generateJsonLd();

// Robots.txt block with AI bot rules
const robotsTxt = geo.generateRobotsTxt();

Next.js: Wire it up with middleware

If you're using Next.js, geo-ai-next makes this even simpler – just add the middleware:

// middleware.ts
import { geoAIMiddleware } from 'geo-ai-next';

export default geoAIMiddleware({
  siteName: 'My Store',
  siteUrl: 'https://example.com',
  provider: new MyProvider(),
  cache: '24h',
  injectLinkHeader: true,
});

export const config = {
  matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
};

This automatically serves /llms.txt and /llms-full.txt and injects Link headers into all responses.

Or use an App Router route handler:

// app/llms/route.ts
import { createLlmsHandler } from 'geo-ai-next';

export const { GET } = createLlmsHandler({
  siteName: 'My Store',
  siteUrl: 'https://example.com',
  provider: new MyProvider(),
});

That's it. Your Next.js app is now optimized for AI search engines.

AI-powered descriptions

GEO AI Core can also generate AI-optimized descriptions for your content via Claude or OpenAI:

import { AiGenerator } from 'geo-ai-core/ai';

const ai = new AiGenerator({
  provider: 'anthropic',
  apiKey: 'sk-...',
  model: 'claude-sonnet-4-20250514',
});

const description = await ai.generate({
  title: 'Premium Widget',
  content: 'A high-quality widget...',
  type: 'product',
  price: '$29.99',
});

Bulk generation with progress tracking:

const results = await ai.bulkGenerate(items, {
  batchSize: 5,
  maxItems: 50,
  onProgress: ({ completed, total }) => console.log(`${completed}/${total}`),
});

What AI crawlers are supported?

GPTBot, OAI-SearchBot, ClaudeBot, claude-web, Google-Extended, PerplexityBot, DeepSeekBot, GrokBot, meta-externalagent, PanguBot, Bytespider, Baiduspider, Amazonbot, Applebot.

Technical highlights