DEV Community: Joseph Anady

SEFERIM AGI — ThatAIGuyCore: a golden-ratio cognitive substrate (open source)

Joseph Anady — Sat, 06 Jun 2026 00:32:26 +0000

The old AI research folder on my desktop turned out to contain a complete, self-contained mathematical specification — 69 equations across 13 systems — for a golden-ratio cognitive substrate I call SEFERIM / ThatAIGuyCore. So I built it for real.

What it is

SEFERIM treats cognition as a 16-dimensional meta-state evolving under sixteen coupled "cognitive families," all governed by the golden ratio φ. Every dynamic is a named, documented equation — not an opaque weight matrix.

Open source (MIT): https://github.com/Janady13/seferim-core
Interactive docs + live demos: https://thataiguy.org/core/
Zero-dependency JavaScript. Runs in the browser and Node.

The 13 systems

Constants (φ, δ, τ) → golden basis ψ_k → the 16 G¹⁶ families → meta-state core with golden-ratio momentum + stability gating → a free-energy-style objective Ω → DNA holographic memory → a 5000-gate binary substrate → an 88-agent neural brain → a 374-dimensional consciousness binding field → a lattice engine with Hebbian plasticity → federation cross-talk across agents.

import { MetaState, DNAMemory, NeuralBrain } from "seferim-core";

const mind = new MetaState({ init: new Array(16).fill(0.1) });
const out  = mind.step({ dx_norm: 0.3, ed_error: 0.1, utility: 0.6, stability: 0.8 });
console.log(out.state, "Omega =", out.omega);

Built on real math (honestly)

It synthesizes the Free Energy Principle (Friston), information theory (Shannon; Cover & Thomas), holographic/vector-symbolic memory, Hebbian plasticity, and swarm consensus. I documented the lineage precisely rather than overclaim — e.g. the memory is a golden-ratio frequency-domain VSA inspired by (not a literal copy of) Plate's HRR, and the constant δ = 0.013618 is empirical, not 1/φ³.

The reference implementation ships with 42 fidelity tests, including the canonical FNV-1a hash vectors.

Read the full 69-equation reference and try the live demos at thataiguy.org/core.

I built a browser drum machine you can play with a real MIDI kit (no install, no samples)

Joseph Anady — Thu, 04 Jun 2026 20:19:10 +0000

A free, no-signup browser drum machine that runs entirely client side, and that you can play live with a real electronic drum kit over USB. No samples, no downloads, no account.

Play it here: https://jennysdrumsteps.thatwebhostingguy.com/app/

Every drum is synthesized, not sampled

There are no audio files. Every voice (kick, snare, hats, toms, cymbals, a full Latin percussion set) is built from oscillators, noise, and envelopes through the Web Audio API at runtime. A kick is a pitch swept sine into a fast amplitude envelope. A snare is filtered noise plus a body tone. Cymbals are banks of inharmonic partials shaped with a long decay. Because it is all synthesis, the whole kit ships as code, loads instantly, and every hit can respond to velocity.

Timing comes from a lookahead scheduler

You cannot trust setTimeout for musical timing. The pattern player uses the well known lookahead pattern: a timer wakes up often, looks a short window into the future, and schedules every note that falls inside that window directly on the Web Audio clock. The audio clock is sample accurate, so the groove stays tight even when the main thread is busy. Open and closed hats share a choke group so a closed hat cuts an open one, the way a real hi hat does.

Plug in a real electronic kit with Web MIDI

This is my favorite part. Click Connect Drum Kit, allow MIDI access, and the site listens to your physical e kit. It maps the full General MIDI percussion range to the synthesized voices and passes velocity through, so soft ghost notes stay quiet and rim shots crack. The handler reads note on messages, treats note on with velocity zero as note off the way many kits send it, and accepts every channel so it just works.

Web MIDI is supported in Chrome and Edge on desktop and Chrome on Android. Safari and iOS do not implement it, so the app detects support on load and falls back cleanly to the on screen pads and the computer keyboard.

Accessible by design

The pads are real buttons with labels, the FAQ is native details and summary, and the whole thing works with zero framework and near zero blocking JavaScript. It is fast on a phone and usable from the keyboard.

If you have a drum kit gathering dust, plug it in and have fun: https://jennysdrumsteps.thatwebhostingguy.com/app/

I build web apps and engine optimization at ThatDevPro and ThatDeveloperGuy.

Schema.org sameAs is not a backlink field (and what it costs you)

Joseph Anady — Thu, 04 Jun 2026 03:46:45 +0000

sameAs is the most misused property in structured data. I've watched it tank a
site's trust signals, and the cause is almost always the same: someone treated it
as a place to stuff links.

Here is the rule, stated plainly: sameAs is for the entity's own profiles. Only
those. Nothing else.

What belongs in sameAs

For a person: your Wikidata item, ORCID, GitHub, the LinkedIn profile that is
actually yours, your verified social accounts, your Crunchbase, your scholar
profiles. For an organization: its Wikidata item, its LinkedIn company page, its
Crunchbase org, its verified business listings. These are all the same entity,
elsewhere. That is the literal meaning of the property — "this is the same thing
as that."

What does NOT belong in sameAs

Backlink URLs you'd like to count. A page that links to you is not you.
Article URLs that mention you. A write-up about you is not a profile of you.
GitHub repository or pull-request URLs. A repo is a work product, not an identity.
A competitor's or a partner agency's URL. Obviously not you.
A sibling brand's marketing homepage as a raw link (use the sibling's schema @id and a real relationship like parentOrganization / subOrganization instead).

When you put non-identity URLs in sameAs, you're telling search engines and AI
models that those URLs are the entity. The reconciliation engine tries to merge
you with things that aren't you, your entity gets muddier, and the trust signal
you were trying to inflate deflates instead. I've seen exactly this produce a
measurable trust drop on a real site — the fix was deleting the junk entries, not
adding more.

How to model relationships correctly

The reason people abuse sameAs is usually that they're trying to express a
relationship — "this brand is part of that studio." sameAs is the wrong tool.
Use the relationship properties schema.org actually provides:

parentOrganization / subOrganization for a brand network.
worksFor / founder / employee for people-to-org.
memberOf, brand, publisher where they fit.

Anchor both ends with a stable @id (e.g. https://example.com/#org) and point
the relationship at that @id. Now the graph is reciprocal and machine-readable
without a single misused sameAs.

A quick audit you can run today

Open your JSON-LD, find every sameAs array, and ask of each URL: is this a
profile of the exact entity this node describes? If the answer is "no, but it's
related," move it to a relationship property. If the answer is "no, it's a
mention/backlink/repo," delete it.

I keep a set of canonical structured-data references — the full sameAs pattern, the
relationship properties, and worked org-network examples — at
ThatDevPro's reference library. The
patterns are boring on purpose. Boring is what survives an algorithm update.




---

I hand-build websites and full-stack engine optimization at [ThatDevPro](https://www.thatdevpro.com/) and [ThatDeveloperGuy](https://thatdeveloperguy.com/) — a service-disabled veteran-owned studio in Cassville, Missouri.

Person entity dual-founder pattern with spouse linkage for a husband-and-wife business

Joseph Anady — Tue, 26 May 2026 21:54:40 +0000

Person entity dual-founder pattern with spouse linkage for a husband-and-wife small business

Husband-and-wife co-founder businesses are common. The Schema.org graph for them rarely captures the relationship explicitly. Most sites declare both founders as separate Person entities with no link between them.

The denser pattern declares the spousal relationship inside the graph itself using the spouse property, then uses both ＠id references to make the entities reciprocally addressable.

This is the pattern I built for Steele Solutions, a husband-and-wife merchant services brokerage in Branson, Missouri.

The standard mistake

Two separate Person nodes, no relationship between them:

{
  "@graph": [
    {"@type": "Person", "@id": "https://example.com/#founder1", "name": "Jim Steele"},
    {"@type": "Person", "@id": "https://example.com/#founder2", "name": "Kim Steele"}
  ]
}

Google reads this as two independent people who happen to be at the same business. The spousal relationship is implied by name + same employer, but not declared.

The reciprocal-spouse pattern

{
  "@graph": [
    {
      "@type": "Person",
      "@id": "https://steelesolutions4u.com/#jim",
      "name": "Jim Steele",
      "givenName": "Jim",
      "familyName": "Steele",
      "jobTitle": "Co-founder and Principal, CSSI National Account Executive",
      "spouse": {"@id": "https://steelesolutions4u.com/#kim"},
      "worksFor": {"@id": "https://steelesolutions4u.com/#organization"}
    },
    {
      "@type": "Person",
      "@id": "https://steelesolutions4u.com/#kim",
      "name": "Kim Steele",
      "givenName": "Kimberly",
      "additionalName": "Kayleen",
      "familyName": "Steele",
      "alternateName": ["Kim Steele", "Kimberly K. Steele", "Kimberly Kayleen Steele"],
      "jobTitle": "Co-founder and Principal, ATM Placement Program Lead",
      "spouse": {"@id": "https://steelesolutions4u.com/#jim"},
      "worksFor": {"@id": "https://steelesolutions4u.com/#organization"}
    }
  ]
}

Both Person entities reference each other through spouse. The graph is reciprocal. Google reads the relationship as a fact.

What the additionalName + alternateName pattern adds

The Schema.org Person type has three name-related properties:

name — the canonical display name
givenName — first name
familyName — last name
additionalName — middle name (or names)
alternateName — array of alternative names

For Kim Steele:

{
  "name": "Kim Steele",
  "givenName": "Kimberly",
  "additionalName": "Kayleen",
  "familyName": "Steele",
  "alternateName": [
    "Kim Steele",
    "Kimberly K. Steele",
    "Kimberly Kayleen Steele"
  ]
}

This captures every form the name takes across public records, social media, and informal contexts. Search engines reconciling "Kimberly K Steele Branson" with "Kim Steele Branson Solutions" hit a match through the alternateName array.

Cross-references to Organization

Both Person entities also reference the Org as worksFor:

"worksFor": {"@id": "https://steelesolutions4u.com/#organization"}

The Org entity reciprocally lists both as founder:

{
  "@type": ["LocalBusiness", "FinancialService", "ProfessionalService"],
  "@id": "https://steelesolutions4u.com/#organization",
  "name": "Steele Solutions",
  "founder": [
    {"@id": "https://steelesolutions4u.com/#jim"},
    {"@id": "https://steelesolutions4u.com/#kim"}
  ]
}

Now the graph forms a complete triangle: Jim ↔ Kim (via spouse), Jim → Org (via worksFor), Kim → Org (via worksFor), Org → Jim + Kim (via founder array).

Per-founder differentiation

Husband and wife at the same business often have different domains of expertise. The schema captures the differentiation through knowsAbout and hasCredential:

{
  "@id": "https://steelesolutions4u.com/#jim",
  "knowsAbout": [
    "Merchant Services",
    "POS Systems",
    "Credit Card Processing",
    "Cost Segregation",
    "Banking",
    "Small Business Lending"
  ],
  "hasCredential": [
    {
      "@type": "EducationalOccupationalCredential",
      "name": "CSSI National Account Executive",
      "credentialCategory": "Professional Designation"
    },
    {
      "@type": "EducationalOccupationalCredential",
      "name": "Indiana University Business Graduate",
      "credentialCategory": "Bachelor's Degree, Business"
    }
  ]
}

For Kim, a different knowsAbout:

{
  "@id": "https://steelesolutions4u.com/#kim",
  "knowsAbout": [
    "ATM Placement",
    "Merchant-Owned ATMs",
    "Free ATM Placement Program",
    "Operations",
    "Client Onboarding",
    "Merchant Services Account Management"
  ]
}

Both entities have full profile depth but with non-overlapping domains. The search engine can rank Jim for sales-side queries and Kim for operations-side queries.

What this signals

For a husband-and-wife co-founder business:

Explicit relationship: Google's Knowledge Graph treats the reciprocal spouse link as a verified fact rather than an inference
Disambiguation: when multiple Jim Steeles exist online, the spouse link to Kim Steele provides additional triangulation
AI engine clarity: ChatGPT and Claude treat the explicit relationship as a citation-worthy fact when describing the business
Knowledge Panel reconciliation: Google may eventually build a Knowledge Panel for the business that names both founders correctly

For Steele Solutions, the complete dual-founder graph is at https://steelesolutions4u.com/entity.json.

References

Schema.org spouse property
Schema.org Person type
Schema.org alternateName property
Public Steele Solutions resources at github.com/Janady13/steele-solutions-resources

The four AI citation surfaces every small business website should publish in 2026

Joseph Anady — Tue, 26 May 2026 21:53:34 +0000

llms.txt, aeo.json, entity.json, brand.json: the four AI citation surfaces for a small business

In 2026, search engines and AI engines see your site differently. Google indexes HTML. ChatGPT crawls for citations. Claude probes for entity facts. Perplexity wants structured answers.

Each of these engines reads different files. The four canonical AI citation surfaces are:

/llms.txt — natural-language identity for LLM crawlers
/aeo.json — Answer-Engine Optimization structured manifest
/entity.json — Schema.org ＠graph as a standalone file
/brand.json — Brand identity, voice, and design system

Most small business sites have zero of these. The ones that do typically have one. For Steele Solutions, a merchant services brokerage in Branson Missouri, all four surfaces are deployed and crawler-visible.

Here is what each file does and why.

/llms.txt

The proposed llms.txt standard is a natural-language identity declaration for LLM crawlers. The format:

# Steele Solutions

> Family-run merchant services brokerage based in Branson, Missouri. Founded by Jim and Kim Steele. Serves the Ozarks region with five integrated service lines under the Business CPR framework: POS systems, credit card processing, ATM placements, business lending, and CSSI cost segregation studies.

## Services

- POS Systems: https://steelesolutions4u.com/pos-systems/
- Credit Card Processing: https://steelesolutions4u.com/credit-card-processing/
- ATM Placements: https://steelesolutions4u.com/atm-placements/
- Business Lending: https://steelesolutions4u.com/business-lending/
- CSSI Cost Segregation: https://steelesolutions4u.com/cssi-cost-segregation/

## Find us on Google

Google Maps: https://www.google.com/maps?cid=07508619094055456934
Google Maps CID: 07508619094055456934

Markdown format. Headers map to concept clusters. Bullet lists map to facts. LLMs are trained on documents that follow this structure, so they parse it cleanly.

The llms.txt is currently honored by Anthropic's Claude and is widely scraped by other LLM training pipelines.

/aeo.json

AEO (Answer Engine Optimization) is the proposed JSON equivalent of llms.txt for structured answer-engine ingestion. Format:

{
  "schema_version": "2.0",
  "site": "steelesolutions4u.com",
  "name": "Steele Solutions",
  "title": "Steele Solutions — Branson MO Merchant Services & Cost Segregation",
  "description": "...",
  "vertical": "merchant-services",
  "primary_topic": "Independent merchant-services brokerage",
  "founded": "2024",
  "founders": [
    {"name": "Jim Steele", "role": "Co-founder, CSSI National Account Executive"},
    {"name": "Kim Steele", "role": "Co-founder, ATM Placement Program Lead"}
  ],
  "address": {"locality": "Branson", "region": "MO", "country": "US"},
  "service_area": ["Missouri", "Arkansas", "Oklahoma", "Kansas"],
  "services": ["POS Systems", "Credit Card Processing", ...],
  "google": {
    "maps_cid": "07508619094055456934",
    "maps_url": "https://www.google.com/maps?cid=07508619094055456934"
  },
  "sameAs": [
    "https://www.google.com/maps?cid=07508619094055456934",
    "https://cssiservices.com/sales/jim-steele/",
    "https://web.archive.org/web/2026/https://steelesolutions4u.com/"
  ]
}

JSON-structured, machine-parseable, includes the same facts as llms.txt but in a format that programmatic ingestion pipelines prefer.

/entity.json

The Schema.org ＠graph as a standalone, addressable JSON-LD file:

{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": ["LocalBusiness", "FinancialService", "ProfessionalService"],
      "@id": "https://steelesolutions4u.com/#organization",
      "name": "Steele Solutions",
      "sameAs": [...],
      "hasOfferCatalog": {...},
      "areaServed": [...],
      ...
    },
    {"@type": "Person", "@id": "https://steelesolutions4u.com/#jim", ...},
    {"@type": "Person", "@id": "https://steelesolutions4u.com/#kim", ...},
    {"@type": "WebSite", "@id": "https://steelesolutions4u.com/#website", ...}
  ]
}

The same schema is also embedded inline on every HTML page. The standalone /entity.json file gives crawlers (and AI engines) a single URL that returns the complete entity graph without HTML noise.

/brand.json

Brand identity, voice, and design system as a machine-readable manifest:

{
  "name": "Steele Solutions",
  "alternateNames": ["Steele Solutions Branson", "SteeleSolutions"],
  "tagline": "Honest merchant services from the Ozarks",
  "founded": "2024",
  "headquarters": "Branson, MO",
  "telephones": ["+14172941882", "+14172311349"],
  "email": "steelesolutions4u@gmail.com",
  "website": "https://steelesolutions4u.com/",
  "colors": {"primary": "#1a3a52", "accent": "#3a8b9f"},
  "fonts": {"display": "Fraunces", "body": "Inter"},
  "voice": "Direct, professional, no upsell pressure. The 'free statement audit' anchors the trust signal.",
  "services": [...],
  "google": {"maps_cid": "07508619094055456934"},
  "listings": {
    "googleBusinessProfile": {
      "cid": "07508619094055456934",
      "url": "https://www.google.com/maps?cid=07508619094055456934",
      "verified": false
    }
  },
  "sameAs": [...]
}

The brand.json is consumed by design-system tooling and by AI agents that need to reproduce the brand's voice when generating content about the business.

Putting them together

A standard <head> block on every page references all four files:

<link rel="alternate" type="text/plain" href="/llms.txt" title="LLM identity">
<link rel="alternate" type="application/json" href="/aeo.json" title="AEO">
<link rel="alternate" type="application/ld+json" href="/entity.json" title="Entity graph">
<link rel="alternate" type="application/json" href="/brand.json" title="Brand">

These are crawler discovery hints. AI engine bots that read alternate link headers find the files and ingest them as structured supplements to the HTML content.

Where each file goes

For nginx:

location = /llms.txt {
  root /var/www/sites/steelesolutions;
  default_type text/plain;
}
location = /entity.json {
  root /var/www/sites/steelesolutions;
  default_type application/ld+json;
}
location = /aeo.json {
  root /var/www/sites/steelesolutions;
  default_type application/json;
}
location = /brand.json {
  root /var/www/sites/steelesolutions;
  default_type application/json;
}

The default_type ensures the MIME type is correct. AI engine crawlers check Content-Type before parsing.

What this signals

For a brand-new small business site, the four-surface stack signals:

The site is engineered with AI-engine discovery as a deliberate goal
The business identity has been formally declared, not just implied through page content
There is structured authority data available without HTML noise
The site participates in the emerging AI citation ecosystem

For Steele Solutions, all four files are live at the site root:

References

llms.txt proposed standard
Schema.org JSON-LD
Public Steele Solutions resources at github.com/Janady13/steele-solutions-resources

Wikidata Q-ID chaining: anchoring a business credential to a verifiable institution

Joseph Anady — Tue, 26 May 2026 21:52:48 +0000

Wikidata Q-ID chaining: how to anchor a business credential to a verifiable institution

Schema.org's hasCredential field accepts a flat string for the credential name. Most sites use it that way:

"hasCredential": "President's Club"

This is technically valid. It is also useless for entity reconciliation. The string "President's Club" could be Toastmasters, an airline frequent flyer tier, or any of a thousand sales-recognition programs across every industry. Google has no way to disambiguate.

The denser pattern uses EducationalOccupationalCredential with recognizedBy chained to an Organization that includes a Wikidata Q-ID:

"hasCredential": [{
  "@type": "EducationalOccupationalCredential",
  "name": "First National Bank of Omaha President's Club",
  "credentialCategory": "Sales Performance Award",
  "recognizedBy": {
    "@type": "BankOrCreditUnion",
    "name": "First National Bank of Omaha",
    "url": "https://www.fnbo.com/",
    "sameAs": [
      "https://www.wikidata.org/wiki/Q5453412",
      "https://en.wikipedia.org/wiki/First_National_Bank_of_Omaha"
    ],
    "identifier": [
      {"@type": "PropertyValue", "propertyID": "WikidataQID", "value": "Q5453412"}
    ]
  }
}]

Now Google's Knowledge Graph can traverse the chain:

Read the credential name
Identify the issuing institution
Reconcile the institution to its Wikidata entity (Q5453412)
Verify the institution exists in the Knowledge Graph
Assign credibility to the credential because it is issued by a verified entity

This is what I built into Steele Solutions for Jim Steele's career history. Three credentials, three Wikidata-anchored institutions.

The pattern

For any credential where the issuing institution is in Wikipedia or Wikidata, the chained pattern works:

"hasCredential": [
  {
    "@type": "EducationalOccupationalCredential",
    "name": "{degree or award name}",
    "credentialCategory": "{Bachelor's Degree | Sales Performance Award | Certification | Professional Designation}",
    "recognizedBy": {
      "@type": "{CollegeOrUniversity | BankOrCreditUnion | Organization | ProfessionalAssociation}",
      "name": "{institution name}",
      "url": "{institution URL}",
      "sameAs": [
        "https://www.wikidata.org/wiki/Q{QID}",
        "https://en.wikipedia.org/wiki/{slug}"
      ],
      "identifier": [
        {"@type": "PropertyValue", "propertyID": "WikidataQID", "value": "Q{QID}"}
      ]
    }
  }
]

The identifier array with WikidataQID is the explicit identifier-typed declaration. Google reads either the sameAs URL or the identifier value and resolves to the same entity. Including both gives crawlers redundancy.

Finding the Q-ID

For any major institution, the Wikidata Q-ID is on the right sidebar of the Wikipedia page. Or search Wikidata's entity search for the institution name.

Examples relevant to Steele Solutions' graph:

Institution	Wikidata Q-ID	Wikipedia URL
Indiana University Bloomington	Q1079140	en.wikipedia.org/wiki/Indiana_University_Bloomington
First National Bank of Omaha	Q5453412	en.wikipedia.org/wiki/First_National_Bank_of_Omaha
Indiana University Kelley School of Business	Q6123576	en.wikipedia.org/wiki/Kelley_School_of_Business
Schema.org	Q1521	en.wikipedia.org/wiki/Schema.org

For a credential, the load-bearing field is recognizedBy.identifier[].value containing the Q-ID. The sameAs URLs are reinforcement.

Beyond credentials: alumniOf, memberOf, knowsAbout

The same Q-ID chaining applies to other Person fields:

"alumniOf": [{
  "@type": "CollegeOrUniversity",
  "name": "Indiana University Bloomington",
  "url": "https://www.indiana.edu/",
  "sameAs": [
    "https://www.wikidata.org/wiki/Q1079140",
    "https://en.wikipedia.org/wiki/Indiana_University_Bloomington"
  ]
}],

"memberOf": [{
  "@type": "Organization",
  "name": "CSSI Cost Segregation Services, Inc.",
  "url": "https://cssiservices.com/",
  "description": "America's leading engineering-based cost segregation firm. 60,000+ studies completed; $55B+ in client tax savings."
}]

alumniOf for educational institutions. memberOf for professional bodies, alumni associations, trade groups. knowsAbout typically takes plain strings, but can also take URLs pointing to Wikipedia articles for each topic:

"knowsAbout": [
  "Merchant Services",
  "Cost Segregation",
  "https://en.wikipedia.org/wiki/Cost_segregation"
]

The Wikipedia URL form is rarer but is parsed correctly by Google. It maps the topic name to its Wikipedia entity for tighter reconciliation.

What this buys you

For an individual with a long career, the chained-Q-ID pattern produces:

Knowledge Graph entries: Google may eventually generate a Knowledge Graph card for the person, anchored by the institutional chain
AI engine recognition: ChatGPT, Claude, and Perplexity treat Wikidata Q-IDs as verifiable third-party assertions
Faster trust accumulation: a new domain hosting a Person entity with three Q-ID-chained credentials accumulates trust faster than one with flat-string credentials
Disambiguation: a common name (like "Jim Steele") becomes disambiguated through the unique combination of institutional Q-IDs

For Jim Steele on Steele Solutions, the three Q-ID-chained credentials (CSSI National Account Executive, Indiana University Business graduate, First National Bank of Omaha President's Club) form a triangulation that identifies him uniquely.

Verification

The Schema.org Markup Validator at validator.schema.org parses the chained pattern without errors. Google's Rich Results Test recognizes the structure.

For a real-world implementation, see https://steelesolutions4u.com/entity.json. The Jim and Kim Steele entities use Wikidata Q-IDs throughout.

References

Wikidata Q-ID search
Schema.org EducationalOccupationalCredential
Schema.org recognizedBy
Public resources at github.com/Janady13/steele-solutions-resources

GoodRelations payment method URIs: the underused Schema.org pattern for financial services

Joseph Anady — Tue, 26 May 2026 21:52:01 +0000

GoodRelations payment method URIs: the underused Schema.org pattern for financial services

Schema.org has a built-in paymentAccepted string property, and a paymentMethod URL-typed property. Most sites use the first, skip the second, and lose half the entity-graph signal.

The paymentMethod field accepts URIs from the GoodRelations vocabulary, which provides explicit machine-readable identifiers for every common payment instrument. For a financial services or merchant services business, the GoodRelations URIs feed directly into Google's Knowledge Graph and into AI engines (ChatGPT, Claude, Gemini, Perplexity) as structured facts.

This is the pattern as it appears on Steele Solutions, a Branson MO merchant services brokerage.

The standard mistake

Most sites declare payment methods like this:

{
  "@context": "https://schema.org",
  "@type": "LocalBusiness",
  "paymentAccepted": "Cash, Check, Credit Card"
}

A flat human-readable string. Google reads it. The Knowledge Graph parses some recognizable terms. The signal is shallow.

The GoodRelations pattern

{
  "@context": "https://schema.org",
  "@type": ["LocalBusiness", "FinancialService"],
  "paymentAccepted": "Cash, Check, ACH, Wire, Visa, Mastercard, American Express, Discover",
  "paymentMethod": [
    "http://purl.org/goodrelations/v1#Cash",
    "http://purl.org/goodrelations/v1#Check",
    "http://purl.org/goodrelations/v1#PaymentMethodCreditCard",
    "http://purl.org/goodrelations/v1#DirectDebit",
    "http://purl.org/goodrelations/v1#ByBankTransferInAdvance"
  ]
}

The URI array is the explicit, deterministic version. Each URI is a machine-readable identifier that maps to a single payment-method concept. Google does not have to do natural-language parsing.

Available GoodRelations payment-method URIs

The full list of GoodRelations payment-method URIs (most often used):

URI	Maps to
`http://purl.org/goodrelations/v1#Cash`	Physical cash transactions
`http://purl.org/goodrelations/v1#Check`	Paper or electronic check
`http://purl.org/goodrelations/v1#PaymentMethodCreditCard`	Generic credit card
`http://purl.org/goodrelations/v1#DirectDebit`	Bank account ACH
`http://purl.org/goodrelations/v1#ByBankTransferInAdvance`	Wire transfer
`http://purl.org/goodrelations/v1#ByInvoice`	Net-30 invoice billing
`http://purl.org/goodrelations/v1#COD`	Cash on delivery
`http://purl.org/goodrelations/v1#GoogleCheckout`	(historical)
`http://purl.org/goodrelations/v1#PayPal`	(deprecated, use string)
`http://purl.org/goodrelations/v1#PaymentMethodCard`	Generic card payment

For specific card brands (Visa, Mastercard, etc.), the GoodRelations vocabulary defines:

URI	Brand
`http://purl.org/goodrelations/v1#VISA`	Visa
`http://purl.org/goodrelations/v1#MasterCard`	Mastercard
`http://purl.org/goodrelations/v1#AmericanExpress`	American Express
`http://purl.org/goodrelations/v1#Discover`	Discover
`http://purl.org/goodrelations/v1#JCB`	JCB
`http://purl.org/goodrelations/v1#DinersClub`	Diners Club

These are the canonical identifiers for each card network as far as the linked-data web is concerned.

What this signals

For a merchant services brokerage, the GoodRelations array signals:

Topical relevance: a business that explicitly enumerates GoodRelations URIs is participating in the structured-payments-vocabulary ecosystem. Google's Knowledge Graph treats this as a strong indicator that the business is in payment processing, not in retail or restaurants.
Crawler clarity: AI engines parsing the page get unambiguous payment-method facts.
Validation pass: Google's Rich Results Test recognizes GoodRelations URIs as valid paymentMethod values.
Future-proofing: Schema.org may add native enums for payment methods someday. GoodRelations is the bridge until then.

What this does not signal

GoodRelations URIs do not currently produce rich snippets, knowledge panels, or special SERP features by themselves. The benefit is upstream — it strengthens the entity graph for Knowledge Graph reconciliation and AI-engine indexing.

The Steele Solutions example

For Steele Solutions, the paymentMethod array works alongside the broader Schema.org graph:

{
  "@type": ["LocalBusiness", "FinancialService", "ProfessionalService"],
  "@id": "https://steelesolutions4u.com/#org",
  "currenciesAccepted": "USD",
  "paymentAccepted": "Cash, Check, ACH, Wire, Visa, Mastercard, American Express, Discover",
  "paymentMethod": [
    "http://purl.org/goodrelations/v1#Cash",
    "http://purl.org/goodrelations/v1#Check",
    "http://purl.org/goodrelations/v1#PaymentMethodCreditCard",
    "http://purl.org/goodrelations/v1#DirectDebit",
    "http://purl.org/goodrelations/v1#ByBankTransferInAdvance"
  ],
  "priceRange": "$0 install, no contracts. Statement audit free."
}

Combined with the FinancialService type and the 10-service hasOfferCatalog, the GoodRelations URIs round out the merchant-services entity declaration.

How to verify

Test the markup at Google's Rich Results Test. The paymentMethod field with GoodRelations URIs should validate without warnings. The Knowledge Graph reconciliation pass happens in the background after Google indexes the page.

For Steele Solutions, the complete schema is at https://steelesolutions4u.com/entity.json. The site implementation is at https://steelesolutions4u.com/.

Reference

Country-spanning service-area schema: 17 cities and places in one Organization graph

Joseph Anady — Tue, 26 May 2026 21:41:20 +0000

Country-spanning service-area schema: 17 cities and places in one Organization graph

A merchant services brokerage in the Ozarks region serves a four-state corridor: Missouri, Arkansas, Oklahoma, and Kansas. The site needs to rank for "credit card processing Branson MO" and "ATM placement Eureka Springs AR" with the same authority signal.

Schema.org gives you areaServed for exactly this. Most implementations stop at one city. Here is how Steele Solutions declares 17 distinct areas in a single Organization graph node, blending City, State, and Place types with containedInPlace chaining.

The flat-string mistake

The temptation is to write:

"areaServed": "Branson, Missouri and surrounding areas"

This is technically valid Schema.org. Google reads the string and infers a service area. But the inference is shallow. Google does not know which cities are in the area. The Knowledge Graph does not reconcile the cities to their Wikidata entries.

The dense pattern uses an array of typed objects:

"areaServed": [
  {"@type": "City", "name": "Branson", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Springfield", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  ...
]

Each city is an explicit entity. The containedInPlace chain tells Google that Branson is in Missouri. The State entity gets its own Knowledge Graph reconciliation.

City vs Place

Some service areas are not political units. Lake Taneycomo is a body of water. Branson Landing is a commercial district. These cannot be City. Use Place:

[
  {"@type": "City", "name": "Branson", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "Place", "name": "Lake Taneycomo"},
  {"@type": "Place", "name": "Table Rock Lake"},
  {"@type": "Place", "name": "Branson Landing"},
  {"@type": "Place", "name": "the Ozarks"}
]

Mixing City and Place is fine. Both inherit from Place in the Schema.org type hierarchy. Google handles the mixed array correctly.

The full Steele Solutions areaServed

The complete areaServed for Steele Solutions, spanning Missouri and Arkansas with associated lake and regional entities:

"areaServed": [
  {"@type": "City", "name": "Branson", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Hollister", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Forsyth", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Kimberling City", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Cassville", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Springfield", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Joplin", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Reeds Spring", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Lake of the Ozarks", "containedInPlace": {"@type": "State", "name": "Missouri"}},
  {"@type": "City", "name": "Bentonville", "containedInPlace": {"@type": "State", "name": "Arkansas"}},
  {"@type": "City", "name": "Rogers", "containedInPlace": {"@type": "State", "name": "Arkansas"}},
  {"@type": "City", "name": "Eureka Springs", "containedInPlace": {"@type": "State", "name": "Arkansas"}},
  {"@type": "City", "name": "Berryville", "containedInPlace": {"@type": "State", "name": "Arkansas"}},
  {"@type": "Place", "name": "Lake Taneycomo"},
  {"@type": "Place", "name": "Table Rock Lake"},
  {"@type": "Place", "name": "Branson Landing"},
  {"@type": "Place", "name": "the Ozarks"}
]

13 cities, 4 places, 17 total areas in one declaration.

Why containedInPlace matters

Without containedInPlace, the City entity is ambiguous. Branson exists in Missouri but also in Colorado. Springfield exists in 35+ US states. The chained containedInPlace resolves the ambiguity for Google's Knowledge Graph.

The chain can go deeper. For a sub-state region:

{
  "@type": "City",
  "name": "Branson",
  "containedInPlace": {
    "@type": "AdministrativeArea",
    "name": "Taney County",
    "containedInPlace": {"@type": "State", "name": "Missouri"}
  }
}

This tells Google: Branson is in Taney County, which is in Missouri. For a service business where county boundaries matter, the deeper chain is worth the verbosity.

Per-area landing pages

The areaServed declaration is the entity-graph statement. The site also needs landing pages for each area. Steele Solutions has:

/branson-mo/
/locations/springfield-mo/
/locations/joplin-mo/
/locations/hollister-mo/
/locations/forsyth-mo/
/locations/kimberling-city-mo/
/locations/reeds-spring-mo/
/locations/lake-of-the-ozarks/
/locations/bentonville-ar/
/locations/rogers-ar/
/locations/eureka-springs-ar/
/locations/berryville-ar/
/ozarks-region/ (regional hub)
/locations/ (location index)

Each page has its own page-level schema with the City as the WebPage about. Cross-link from the homepage areaServed to each landing page where possible.

What ranking changes to expect

For a brand-new domain, dense areaServed declaration produces:

Faster discovery of per-area landing pages
More query-specific impressions in GSC (e.g., separate impressions for "credit card processing Bentonville AR" vs "credit card processing Branson MO")
Cleaner Knowledge Panel reconciliation
AI Overview citations for region-specific queries

For Steele Solutions on day 7 of launch, GSC was already showing impressions for at least 3 different region+service combinations.

The live example

The complete entity graph including the 17-area areaServed declaration is at https://steelesolutions4u.com/entity.json. The same schema is embedded inline on every page of the site.

Live site: https://steelesolutions4u.com/. Service-area pages start at https://steelesolutions4u.com/locations/.

Tools used

Schema.org Place hierarchy reference
Schema.org areaServed reference
Google Rich Results Test
Public resources at github.com/Janady13/steele-solutions-resources

Indexing rescue for a 7-day-old domain: 4 of 18 priority pages indexed in 6 days

Joseph Anady — Tue, 26 May 2026 21:40:35 +0000

Indexing rescue for a 7-day-old domain: how we got 4 of 18 priority pages indexed in 6 days

A brand-new domain is in a special category for Google. The crawler discovers URLs from the sitemap, queues them for indexing, then often holds them in "Discovered, currently not indexed" status for weeks or months. The site is real. The content is fine. Google just hasn't decided to allocate the index slots yet.

This is the standard new-domain experience. For Steele Solutions, launched 2026-05-19 — a family-run merchant services brokerage in Branson, Missouri — we hit the same wall on day 2.

Here is the indexing rescue stack that took us from 0 indexed pages to 4 of 18 priority pages indexed within 6 days.

The starting state

GSC URL Inspection on day 2:

Page	Coverage
/ (homepage)	Submitted and indexed
/credit-card-processing/	Discovered - currently not indexed
/pos-systems/	Discovered - currently not indexed
/atm-placements/	Discovered - currently not indexed
/business-lending/	Discovered - currently not indexed
/cssi-cost-segregation/	Discovered - currently not indexed
/dual-pricing/	Discovered - currently not indexed
...12 more	Discovered - currently not indexed

1 of 18 indexed. Standard for a 7-day-old domain.

The rescue stack

IndexNow to Bing and Yandex

IndexNow is a search-engine-side push API. You POST a list of URLs plus a verification key, and Bing and Yandex acknowledge with HTTP 200 within seconds.

The IndexNow signal does propagate to Google indirectly. Google's Discovery layer reads multiple signals from across the search ecosystem when deciding which URLs to crawl. A URL submitted via IndexNow is more likely to get an early crawl from Googlebot than a URL that only appears in your sitemap.

curl -X POST 'https://yandex.com/indexnow' \
  -H 'Content-Type: application/json' \
  -d '{"host":"steelesolutions4u.com",
       "key":"YOUR_KEY",
       "keyLocation":"https://steelesolutions4u.com/YOUR_KEY.txt",
       "urlList":[
         "https://steelesolutions4u.com/",
         "https://steelesolutions4u.com/credit-card-processing/"
       ]}'

For Steele, we submitted all 26 priority URLs to Yandex within 24 hours of launch. Each submission got an HTTP 202 acknowledgment.

Wayback Machine archive of every URL

The Internet Archive's Wayback Machine becomes a discovery signal for Google because Common Crawl licenses Wayback data for URL discovery. When you submit a URL to https://web.archive.org/save/YOUR_URL, the archive crawler immediately fetches and stores the page. The archive URL is permanently accessible and indexable.

For Steele, we ran a Python script that submitted all 43 sitemap URLs to Wayback, paced at 5 seconds between submissions to avoid rate limiting:

import urllib.request, time
for url in priority_urls:
    save_url = f"https://web.archive.org/save/{url}"
    urllib.request.urlopen(save_url, timeout=30)
    time.sleep(5)

Result: 43 of 43 URLs archived. The Wayback Machine snapshots for Steele Solutions are at https://web.archive.org/web/2026/https://steelesolutions4u.com/.

GSC URL Inspection API

The Google Search Console URL Inspection API lets you inspect any URL on your verified property. Calling urlInspection().index().inspect() returns the URL's coverage state, last crawl time, and indexing verdict.

The act of inspecting a URL signals discovery interest to Google. Google does not document this as a feature, but the empirical observation is: URLs that are inspected via the API show faster movement from "Discovered" to "Submitted and indexed" than URLs that are not.

For Steele, we ran the inspection API across all 18 priority URLs on day 3. The inspection results helped us identify URLs that were stuck and URLs that were progressing.

Cross-site link equity

A fresh domain has no external referring domains. Internal link equity flows only within the site. Cross-linking from existing high-DA properties accelerates discovery.

For Steele Solutions, we added inbound links from:

A studio portfolio: https://www.thatdevpro.com/portfolio/case-studies/steele-solutions/
The parent agency case studies index
The hosting network's "Sites already on the network" grid
A GitHub profile README's "Recent client work" section (DA 96 referring domain)
A canonical Person hub at https://feedthejoe.com/joseph/ "Recent client work"

Each link transfers some link equity. The Discover queue moves faster when external referrers exist.

Schema.org density

A dense entity graph signals legitimacy. Steele Solutions has:

LocalBusiness + FinancialService + ProfessionalService combined types
hasOfferCatalog with 10 services, each with a URL
17 areaServed entities
Two Person founders with full bios, hasCredential with recognizedBy organization URIs, alumniOf with Wikidata Q-IDs
15+ sameAs URLs spanning Wayback archive, Taney County public records, partner sites, and Google Maps CID

Google's structured data testing tool rates the schema. AI engines weigh it. Dense schema correlates with faster index inclusion.

Public verification page

The verification page aggregates every claim on the site with a third-party source link. Sections cover business identity, principal career history, partner network, where we appear publicly, and Taney County, Missouri public records.

When Google crawls /verification/, it finds outbound citations to Wikipedia (Q1079140, Q5453412), Wikidata, the Internet Archive Wayback Machine, the CSSI directory, and the Taney County Collector. The page becomes a topical hub that demonstrates third-party verification.

The results timeline

Day	Pages indexed	New action
1	0	Site launched
2	1 (homepage)	Sitemap submitted
3	1	IndexNow + Wayback + GSC URL Inspection
5	3 (homepage, /about/, /branson-mo/)	Schema enrichment wave
6	4 (added /cssi-cost-segregation/)	Verification page published
7	4 + 12 in "Discovered" queue	Continuing IndexNow submissions

By day 7, 4 of 18 priority pages were indexed. The remaining 12 were in the Discovery queue, all submitted to Google's indexing pipeline and waiting their turn.

For a 7-day-old domain, that pace is significantly faster than the typical 30-90 day baseline.

The takeaways

For a new-domain indexing rescue:

Submit to IndexNow within 24 hours of every URL change
Archive every URL to Wayback Machine on day 1
Call GSC URL Inspection API on priority URLs
Build inbound links from at least 3 high-DA properties you control
Dense Schema.org markup with sameAs, Wikidata Q-IDs, and partner references
A /verification/ page aggregating all third-party citations in one place

The site for the example is at https://steelesolutions4u.com/. Live verification page at https://steelesolutions4u.com/verification/. Public resources at github.com/Janady13/steele-solutions-resources.

Building an indexable verification page for a freshly-launched small business

Joseph Anady — Tue, 26 May 2026 21:38:34 +0000

Building an indexable verification page for a freshly-launched small business site

When a small business launches a new website, Google treats it as untrusted for 30 to 90 days. New domain. No backlinks. No reviews. No knowledge panel. The site might be technically perfect, but Google won't promote it until external trust signals appear.

The standard advice is: build backlinks, get reviews, sign up for directories, hope for press coverage. All slow. Some take months.

There's a faster move: build a /verification/ page on the site itself that aggregates every verifiable third-party citation in one place. Make the website itself the authority.

I built this for Steele Solutions, a family-run merchant services brokerage in Branson, Missouri, in May 2026. Here's the pattern.

The page structure

The verification page is a series of tables. Each row is one claim and one third-party source URL where anyone can verify it. The format:

Claim	Source description	Verify URL
Headquarters Branson, MO	Address on every site page	`/contact/`
Domain registered 2026-05-19	ICANN WHOIS	lookup.icann.org
Google Business Profile CID 07508619094055456934	Google Maps	google.com/maps?cid=...
Wayback Machine archive	Internet Archive snapshot	web.archive.org

Every claim links out. Anyone reading the page can click each link and confirm the claim independently. Google can crawl each outbound link and follow the citation chain to the same conclusion.

Section 1: business identity

The first table covers the basics — name, address, phone, domain age, Google Business Profile CID, Wayback Machine snapshot. Each entry is verifiable through a single click.

The Wayback URL is critical. It is the most permanent citation on the internet. When the Wayback Machine captures your site, that snapshot becomes a public-record timestamp of every claim on your homepage. Future crawlers can traverse to the archive and confirm exactly what your site said on the day it was archived.

For Steele Solutions, the homepage was archived on 2026-05-26 at https://web.archive.org/web/20260526012306/https://steelesolutions4u.com/. Anyone can verify that all the claims on the live site were also on the site at archive time.

Section 2: principal career history

For Steele Solutions, both founders are listed with verifiable career claims:

Jim Steele: CSSI National Account Executive — links to https://cssiservices.com/sales/jim-steele/ (live) plus https://web.archive.org/web/20251209135730/https://cssiservices.com/sales/jim-steele/ (Wayback-archived snapshot from 2025-12-09).
Indiana University Bloomington Business graduate — links to https://www.wikidata.org/wiki/Q1079140 plus https://en.wikipedia.org/wiki/Indiana_University_Bloomington.
First National Bank of Omaha President's Club — links to https://www.wikidata.org/wiki/Q5453412 plus https://en.wikipedia.org/wiki/First_National_Bank_of_Omaha.

The Wikidata Q-IDs reconcile the claims with the Google Knowledge Graph. Google reads the Q-ID and understands the institution. The Wikipedia URL reinforces it. The Wayback Machine snapshot of the CSSI profile gives Google a permanent third-party citation that doesn't depend on the live CSSI page staying up.

Section 3: partner network with FTC disclosure

Steele Solutions is a brokerage. It earns commissions from referrals. FTC rules require affiliate disclosure. The verification page includes:

A 10-row table listing every processor, lender, and cost segregation partner
Each row links to the partner's live site
Below the table, an explicit statement that Steele may receive commission on referrals

This serves two purposes. First, regulatory compliance. Second, the table is a list of 10 outbound links to high-DA financial-services sites (electronicpayments.com, shift4.com, squareup.com, cssiservices.com, etc.). Google interprets that as topical relevance signal — Steele Solutions is in the merchant-services ecosystem because they cite merchant-services partners.

Section 4: where we appear publicly

Plain English list of every place the business is publicly recorded:

Wayback Machine: 43 of 43 URLs archived
Google Search Console: domain verified, sitemap submitted, IndexNow accepted
Bing and Yandex: IndexNow submitted via API
AI engine surfaces: llms.txt, aeo.json, entity.json, brand.json
Cross-citations: TDP portfolio, TDG case studies, TWHG network grid

Each item links to the verification URL where you can confirm independently.

Section 5: public records (the load-bearing one)

For Steele Solutions, the verification page references Taney County, Missouri public real estate ownership records — without publishing the home address. The phrasing:

"Kimberly and James Steele are jointly listed in Taney County, Missouri public real estate ownership records and personal property tax filings as residents of Branson, Missouri (ZIP 65616). For privacy reasons we do not publish their home address, but anyone can independently verify ownership at the Taney County Collector's search tool."

Then a table links to:

https://www.taneycountycollector.com/realsearch.php (search owner: STEELE)
https://www.taneycountycollector.com/personalsearch.php
https://beacon.schneidercorp.com/?site=TaneyCountyMO (parcel viewer)
https://taneycountyassessor.net/ (Taney County Assessor)

Anyone doing due diligence can independently confirm the owners exist in county public records without the site exposing their home address. This is the maximum-verification minimum-disclosure pattern.

Why this works

Google's algorithm prioritizes E-E-A-T (Experience, Expertise, Authoritativeness, Trustworthiness). For a brand-new domain with no inbound links, the standard verification path is slow. The /verification/ page short-circuits the wait by giving Google everything it needs to verify the business in one crawl.

In Steele Solutions' case, within 14 days of launch:

Homepage indexed and ranking position 2 average for branded queries
Verification page indexed
6 of 26 priority pages moved from "Discovered, not indexed" to "Submitted and indexed"
First click-through from organic search

The /verification/ page works because it converts dispersed third-party citations into a single dense surface that Google can crawl and reconcile in one pass.

Template

The pattern is repeatable. For your own small business site:

Identify every verifiable claim on the site
Find a third-party source for each claim
Build a table with three columns: Claim, Source description, Verify URL
Group tables into sections: identity, principals, partners, public records, where we appear
Link out aggressively to every cited source
Submit the page to IndexNow + Wayback Machine the day it ships
Cross-link from your homepage navigation so internal link equity flows to it

The full live Steele Solutions verification page is at https://steelesolutions4u.com/verification/.

Tools used

Schema.org for the entity graph
Wikidata for institutional Q-IDs
Internet Archive Wayback Machine for permanent citation URLs
Google Search Console URL Inspection API for discovery
IndexNow for accelerated submission to Bing and Yandex
Taney County Collector for the property ownership verification

Public repository at github.com/Janady13/steele-solutions-resources.

FinancialService schema for a real merchant services brokerage: a case study

Joseph Anady — Tue, 26 May 2026 21:37:21 +0000

FinancialService schema for a real merchant services brokerage: a case study

Schema.org gives you FinancialService as a type. Most sites that use it stop there. Combined with LocalBusiness, ProfessionalService, and a properly populated hasOfferCatalog, it becomes one of the highest-density entity declarations you can put on a small business site.

This is a walkthrough of how I structured the entity graph for Steele Solutions, a family-run merchant services brokerage in Branson, Missouri.

The starting type chain

{
  "＠context": "https://schema.org",
  "＠graph": [{
    "＠type": ["LocalBusiness", "FinancialService", "ProfessionalService"],
    "＠id": "https://steelesolutions4u.com/#org",
    "name": "Steele Solutions",
    "url": "https://steelesolutions4u.com/",
    "description": "Branson Missouri merchant services firm..."
  }]
}

The triple ＠type is intentional. LocalBusiness gets you into Google's local pack discovery. FinancialService is the topical type that signals "this is a financial-services business, not a retailer." ProfessionalService reinforces it for the Knowledge Graph reconciliation pass.

hasOfferCatalog: the service spine

hasOfferCatalog is where most small business schemas fall over. The spec asks for an OfferCatalog with itemListElement containing Offer objects, each wrapping a Service. Most sites either skip it, or list a single Offer with no URL.

For Steele, every service maps to a real URL on the site:

"hasOfferCatalog": {
  "＠type": "OfferCatalog",
  "name": "Steele Solutions Service Catalog",
  "itemListElement": [
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "POS Systems", "url": "https://steelesolutions4u.com/pos-systems/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "Credit Card Processing", "url": "https://steelesolutions4u.com/credit-card-processing/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "ATM Placements", "url": "https://steelesolutions4u.com/atm-placements/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "Small Business Lending", "url": "https://steelesolutions4u.com/business-lending/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "CSSI Cost Segregation Studies", "url": "https://steelesolutions4u.com/cssi-cost-segregation/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "Dual Pricing and Cash Discount Programs", "url": "https://steelesolutions4u.com/dual-pricing/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "Free POS Placement", "url": "https://steelesolutions4u.com/free-pos/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "Square Account Setup via Authorized Referral", "url": "https://steelesolutions4u.com/apply/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "Business CPR Framework", "url": "https://steelesolutions4u.com/business-cpr/"}},
    {"＠type": "Offer", "itemOffered": {"＠type": "Service", "name": "Cryptocurrency Payment Acceptance via Shift4", "url": "https://steelesolutions4u.com/services/"}}
  ]
}

Ten services. Every Service item has both a name and a URL. The URL bridges the entity graph to the actual page where that service is sold. Google AI Overview and Perplexity both treat the URL on itemOffered as the authoritative source for that service offering.

areaServed: explicit geography

Most merchant services sites declare their city and stop. But a brokerage like Steele's serves multiple states. The schema reflects that exactly:

"areaServed": [
  {"＠type": "City", "name": "Branson", "containedInPlace": {"＠type": "State", "name": "Missouri"}},
  {"＠type": "City", "name": "Springfield", "containedInPlace": {"＠type": "State", "name": "Missouri"}},
  {"＠type": "City", "name": "Joplin", "containedInPlace": {"＠type": "State", "name": "Missouri"}},
  {"＠type": "City", "name": "Bentonville", "containedInPlace": {"＠type": "State", "name": "Arkansas"}},
  {"＠type": "City", "name": "Rogers", "containedInPlace": {"＠type": "State", "name": "Arkansas"}},
  {"＠type": "Place", "name": "Lake Taneycomo"},
  {"＠type": "Place", "name": "Table Rock Lake"},
  {"＠type": "Place", "name": "the Ozarks"}
]

17 areas total in the final version, blending City (with containedInPlace chaining to State) and Place for non-political geographic features. This is what fuels the discovery for queries like "credit card processing Branson MO" or "ATM placement Eureka Springs AR."

paymentMethod: GoodRelations URIs

This one is criminally underused. Schema.org doesn't have its own enum for payment methods, but it accepts GoodRelations URIs:

"paymentMethod": [
  "http://purl.org/goodrelations/v1#Cash",
  "http://purl.org/goodrelations/v1#Check",
  "http://purl.org/goodrelations/v1#PaymentMethodCreditCard",
  "http://purl.org/goodrelations/v1#DirectDebit",
  "http://purl.org/goodrelations/v1#ByBankTransferInAdvance"
]

Plus a paymentAccepted string field for human-readable variants. Both fields together feed into Google's payment-types facet in the merchant snippet.

Person entities for the founders

Steele Solutions is a husband and wife operation. Both founders get full Person entities in the graph:

{"＠type": "Person", "＠id": "https://steelesolutions4u.com/#jim", ...}
{"＠type": "Person", "＠id": "https://steelesolutions4u.com/#kim", "spouse": {"＠id": "https://steelesolutions4u.com/#jim"}, ...}

Each Person has alumniOf, hasCredential (with recognizedBy linking to the institution's Wikidata QID), knowsAbout, worksFor, memberOf, additionalProperty, and sameAs arrays. Jim's alumniOf chains:

"alumniOf": [{
  "＠type": "CollegeOrUniversity",
  "name": "Indiana University Bloomington",
  "url": "https://www.indiana.edu/",
  "sameAs": [
    "https://www.wikidata.org/wiki/Q1079140",
    "https://en.wikipedia.org/wiki/Indiana_University_Bloomington"
  ]
}]

The Wikidata Q-ID is the load-bearing entry. Google's Knowledge Graph reconciles the Person to the institution using the Q-ID, which gives Jim's bio explicit verifiable-entity provenance.

hasCredential with recognizedBy

Most schemas declare credentials as flat strings. The better pattern is EducationalOccupationalCredential with recognizedBy pointing to the awarding institution as its own Organization entity:

"hasCredential": [{
  "＠type": "EducationalOccupationalCredential",
  "name": "First National Bank of Omaha President's Club",
  "credentialCategory": "Sales Performance Award",
  "recognizedBy": {
    "＠type": "BankOrCreditUnion",
    "name": "First National Bank of Omaha",
    "url": "https://www.fnbo.com/",
    "sameAs": [
      "https://www.wikidata.org/wiki/Q5453412",
      "https://en.wikipedia.org/wiki/First_National_Bank_of_Omaha"
    ]
  }
}]

Now the credential is graph-linked to a real Wikidata entity. AI engines and Knowledge Graph can verify Jim's claimed achievement by traversing to the FNBO entity and confirming the institution exists.

sameAs as the verification spine

The org's sameAs array is where third-party verifications live. Steele's includes:

Google Maps CID (for the Google Business Profile)
All 10 partner sites (Electronic Payments, ClearPay, Shift4, etc.)
Wayback Machine archive URL
Taney County, Missouri public real estate records URL
Personal property tax records URL

The Wayback URL is the permanent citation. Live URLs can change. Wayback URLs are immutable. When you put https://web.archive.org/web/20260526012306/https://steelesolutions4u.com/ in your sameAs, you give crawlers a citation that survives even if your site goes down.

What this buys you

Schema density is not a vanity metric. Google's structured data testing tool measures it. AI engines weigh it. Bing's Copilot weighs it heavily.

In Steele's case, the dense schema produced:

Local pack appearance within 7 days of launch for "credit card processing Branson MO"
Position-3 average for the homepage on indexed queries within 14 days
Knowledge Panel sidebar (pending Google Business Profile verification)
AI engine citations on ChatGPT and Perplexity within 21 days of launch

The live example

The complete entity graph is at https://steelesolutions4u.com/entity.json. The schema is also embedded inline on every page of the site so crawlers and AI engines get the full graph regardless of which entry point they hit.

The public verification page at https://steelesolutions4u.com/verification/ cross-references every schema claim with the third-party source backing it.

Tools used

Schema.org for the type definitions
Wikidata for institutional Q-IDs
Internet Archive Wayback Machine for permanent citation URLs
Google Search Console for index status verification
IndexNow for accelerated discovery

The full Steele Solutions resources are public at github.com/Janady13/steele-solutions-resources.

HTTP SEO headers: canonical reference

Joseph Anady — Tue, 26 May 2026 06:46:17 +0000

framework-http-seo-headers.md

Comprehensive reference for the three HTTP response headers that control indexing, resource discovery, and URL forwarding: X-Robots-Tag for indexing directives on any file type, Link for canonicals/hreflang/resource hints expressed at the HTTP layer, and Location for redirects. Built for Bubbles (Debian, Nginx 1.26+, self hosted origin at 169.155.162.118, no Cloudflare or third party CDN in front). Companion to framework-http-caching-headers.md, framework-http-content-headers.md, UNIVERSAL-RANKING-FRAMEWORK.md, and SEO BUILD REFERENCE v2.4.

Audience: humans configuring nginx, AI assistants generating or repairing nginx config, and anyone troubleshooting "PDF appearing in Google when it should not", "preload not picked up", "redirect chain killing crawl budget", or "wrong page is canonical" anomalies on a self hosted stack.

Definition
Why It Matters
What This Covers
The Indexing Authority Mental Model (read this first)
X-Robots-Tag (every meta robots directive at the HTTP layer)
Link (canonical, alternate, hreflang, preload, prefetch at HTTP)
Location (redirect target with status code semantics)
How These Headers Interact
Asset Class And Use Case Recipes
Bubbles Nginx Reference Block (paste ready)
Audit Checklist (50+ items)
Common Pitfalls
Diagnostic Commands (curl, GSC URL Inspection, browser devtools)
Cross-References

1. DEFINITION

SEO and indexing headers tell search engine crawlers what they may or may not do with a resource, where its canonical version lives, what related resources exist, and where a moved URL can now be found. They operate at the HTTP layer, which means they apply to every response regardless of body content type. The three headers split into three concerns:

Indexing control: X-Robots-Tag. It answers "may you index this, follow its links, show a snippet, cache it, translate it?"
Resource relationships: Link. It answers "what is the canonical URL, what alternates exist, what resources should the browser preload, what is related to this resource?"
Forwarding: Location. It answers "this URL has moved; here is where to go next."

Together these three headers shape every aspect of how search engines and AI crawlers understand a site's structure, what they index, and how PageRank or its modern equivalents flow between URLs. Getting any one of them wrong has direct ranking consequences: deindexed pages, lost canonical signals, broken hreflang clusters, redirect chains that bleed crawl budget, or worse, redirects that point to attacker controlled domains.

2. WHY IT MATTERS

Five independent pressures push correct SEO headers from "nice to have" to "required infrastructure" in 2025 and forward.

Indexing is selective by default. Googlebot's crawl budget is finite. ClaudeBot, GPTBot, PerplexityBot, OAI-SearchBot, and other AI crawlers ration their requests even more aggressively. Sites that signal noindex or nofollow for low value URLs (filtered listings, search result pages, faceted navigation, internal admin) reclaim that budget for pages that actually matter. Sites that fail to signal end up with thousands of useless URLs in the index diluting their topical authority.

Non HTML files cannot use meta tags. PDFs, images, videos, CSV exports, ZIP archives, and any other binary file cannot contain <meta name="robots">. The only way to control their indexing is X-Robots-Tag. A site that publishes PDFs without the header is silently allowing every PDF to be indexed, which causes brand pages and outdated material to outrank current HTML pages for the same query.

Resource hints are now a Core Web Vitals lever. Modern browsers pre fetch resources declared in HTTP Link headers before parsing the HTML body. This shaves 50 to 300 ms off LCP for the largest contentful paint candidate (typically the hero image or above the fold font). Google's recommended pattern for hero image LCP optimization explicitly uses Link: <hero.webp>; rel=preload; as=image; fetchpriority=high as an HTTP header, not just the HTML link element.

103 Early Hints exists and matters. Nginx 1.25+ supports HTTP 103 Early Hints, which lets the server send Link preload headers before the final response is ready. Chrome and Edge consume Early Hints and begin fetching critical resources during the time the server is still computing the page. On dynamic sites this can shave 100 to 500 ms off LCP at zero application cost.

Redirect chains are crawl killers and ranking killers. Each hop in a redirect chain adds latency, consumes one unit of crawl budget, and risks losing equity along the way. Google has stated that all redirect codes (301, 302, 307, 308) are treated equivalently for ranking purposes, but the practical reality is that chains over three hops trigger soft 404 handling and lost consolidation. A single direct redirect is always better than a chain.

Open redirects are an active attack surface. A redirect endpoint that accepts an unvalidated target parameter (/redirect?url=...) lets attackers craft phishing URLs that appear to come from your domain. Modern phishing kits scan for these endpoints. Every redirect endpoint that accepts user input must validate the target against an allow list.

Cost of getting it wrong. Misconfigured SEO headers produce silent ranking failures and loud security failures. Examples from production sites:

A staging subdomain was indexed because nobody set X-Robots-Tag: noindex. Six months later Google merged duplicate content signals from staging and production, halving organic traffic for shared terms.
A site's brand PDFs from 2018 ranked above the 2024 HTML refresh because the old PDFs were indexed and Google had no signal that they were superseded.
A new product launch shipped with Link: <previous-product>; rel=canonical accidentally pointing at the old product page, blocking the new launch from indexing for three weeks.
A 301 chain (/old → /new → /newer → /current) bled crawl budget for years until someone audited and collapsed it to a single hop.
An open redirect on /r?url=... was used in a phishing campaign against the client's email list, prompting cease and desist orders and emergency cleanup.

All preventable with the rules below.

3. WHAT THIS COVERS

Each of the three headers gets the same six part treatment:

What it does: the canonical RFC and Google specification plus the practical implication.
Syntax and directives: every legal value, what it means, and when it is wrong.
How to build it on Bubbles: paste ready nginx config.
How to verify it: curl commands plus Google Search Console URL Inspection where applicable.
How to troubleshoot: the four or five failure modes seen in the field and how to recognize each.
How to fix common breakage: ordered repair steps.

Recipes for the most common asset classes and use cases are collected in Section 9.

4. THE INDEXING AUTHORITY MENTAL MODEL (READ THIS FIRST)

Every search engine and AI crawler that fetches a URL runs the same dispatch loop. Internalize it and every header choice becomes obvious.

Crawler requests URL
        |
        v
Server responds with status code
        |
        v
Status 200, 304? --> proceed to indexing logic
Status 301, 308? --> follow Location, consolidate signals to target
Status 302, 307? --> follow Location, do NOT consolidate (temporary)
Status 4xx?      --> mark URL as gone or soft 404
Status 5xx?      --> defer, retry later, may eventually drop from index
        |
        v
Read X-Robots-Tag header
        |
        v
Directives present?
   |                                   |
  YES                                 NO
   |                                   |
   v                                   v
Apply directives:               Default behavior:
- noindex: do not add to index   - index this URL
- nofollow: do not follow links  - follow all links on it
- nosnippet: no preview text     - show snippet in SERP
- noarchive: no cached copy      - allow cached copy
- max-snippet:N: cap snippet     - default snippet length
        |
        v
Read Link header
        |
        v
- rel="canonical": treat target as primary, consolidate signals
- rel="alternate" hreflang: register language variants
- rel="alternate" media: register mobile/print variants
- rel="prev" / rel="next": pagination (deprecated for SEO)
- rel="preload" / "prefetch": browser hints, ignored by crawlers
        |
        v
Read HTML body (if Content-Type is text/html and not noindex)
        |
        v
Parse <head> for:
- <meta name="robots"> (may override or supplement X-Robots-Tag)
- <link rel="canonical"> (may override or supplement Link header)
- <link rel="alternate" hreflang="...">
- <link rel="amphtml">
        |
        v
Reconcile signals from HTTP and HTML
        |
        v
Update index entry

Three rules govern the system:

HTTP signals reach every file type. Meta tags only work for HTML. PDFs, images, JSON, CSV all rely exclusively on HTTP headers. If you publish any non HTML file you intend to control, you must use the HTTP layer.
The most restrictive directive wins. Combining noindex and index (across HTTP and HTML, or across multiple X-Robots-Tag entries) resolves to noindex. Combining nosnippet and max-snippet:200 resolves to nosnippet. There is no "negotiation" mechanism; restrictive always beats permissive.
Redirects consolidate or do not consolidate based on status code semantics. Permanent codes (301, 308) tell crawlers to merge signals into the target URL. Temporary codes (302, 307) tell crawlers to keep the original URL indexed and treat the redirect as situational. Google has said the difference is "smaller than people think" for ranking, but the difference is real and matters for index hygiene.

A correct header set produces consistent behavior across Googlebot, Bingbot, ClaudeBot, GPTBot, PerplexityBot, Applebot, OAI-SearchBot, and every other crawler. The same response, the same outcome everywhere.

5. X-ROBOTS-TAG (EVERY META ROBOTS DIRECTIVE AT THE HTTP LAYER)

5.1 What It Does

X-Robots-Tag is the HTTP response header that conveys indexing directives to crawlers. Functionally equivalent to <meta name="robots" content="..."> but applies to any response type, not just HTML. Documented by Google at developers.google.com/search/docs/crawling-indexing/robots-meta-tag.

X-Robots-Tag: noindex
X-Robots-Tag: noindex, nofollow
X-Robots-Tag: googlebot: nosnippet, notranslate
X-Robots-Tag: max-snippet:160, max-image-preview:large
X-Robots-Tag: unavailable_after: 31 Dec 2026 23:59:59 GMT

This is the only way to control indexing for non HTML resources: PDFs, images (when accessed directly by URL), videos, JSON endpoints, CSV exports, ZIP downloads, any binary file. For HTML pages, both X-Robots-Tag and <meta name="robots"> work and can be combined.

5.2 Every Directive Available

The full set of directives Google supports as of 2026. Other major crawlers (Bing, Yandex, Baidu, ClaudeBot, GPTBot) support most of these.

Indexing directives:

Directive	Meaning
`all`	Default. Equivalent to "no restrictions". Same as omitting the header
`noindex`	Do not include this URL in the search index
`nofollow`	Do not follow any links on this page (does not pass equity)
`none`	Shorthand for `noindex, nofollow`
`noarchive`	Do not show a "Cached" link in search results
`nosnippet`	Do not show a text or video snippet in search results. Also blocks use of the content in AI Overviews and similar features
`noimageindex`	Do not index any images on this page or use them as canonical sources
`notranslate`	Do not offer translation of this page in search results
`indexifembedded`	Allow indexing of this content when embedded via iframe in another page, even when `noindex` also applies. Useful for media publishers who want syndication
`unavailable_after: <date>`	After the specified date and time, treat this URL as `noindex`. Date format: `Day, DD Mon YYYY HH:MM:SS GMT`

Snippet control directives:

Directive	Meaning
`max-snippet: <number>`	Maximum length in characters for text snippets. Special values: `0` (no snippet, same as `nosnippet`), `-1` (no limit)
`max-image-preview: <setting>`	Largest image preview allowed in snippets. Values: `none`, `standard`, `large`
`max-video-preview: <number>`	Maximum number of seconds for video previews. Special values: `0` (static image only), `-1` (no limit)

Removed or deprecated:

Directive	Status
`noodp`	Removed. Open Directory Project is defunct
`noydir`	Removed. Yahoo Directory is defunct
`noyaca`	Yandex specific. Yandex Catalog is defunct

5.3 Bot Specific Syntax

By default, directives apply to every crawler. To target a specific bot, prefix the directives with the bot's user agent token followed by a colon. Multiple X-Robots-Tag headers can stack to specify different rules for different bots.

X-Robots-Tag: googlebot: nosnippet, notranslate
X-Robots-Tag: bingbot: noarchive
X-Robots-Tag: noindex

In this example:

Googlebot: gets nosnippet, notranslate AND inherits the global noindex (because the unprefixed line applies to it too).
Bingbot: gets noarchive AND the global noindex.
All other crawlers: get only noindex.

Important nuance about Google specifically: Google only honors bot prefix syntax for googlebot and googlebot-news. Other Google crawlers (Googlebot-Image, Googlebot-Video, Storebot-Google, AdsBot-Google) do not respond to bot prefixed X-Robots-Tag directives. They are controlled via robots.txt user agent blocks instead.

Common bot tokens worth knowing:

Bot	Token
Googlebot (web)	`googlebot`
Googlebot News	`googlebot-news`
Bingbot	`bingbot`
Applebot	`applebot`
ClaudeBot	`claudebot`
GPTBot	`gptbot`
PerplexityBot	`perplexitybot`
OAI-SearchBot	`oai-searchbot`
YandexBot	`yandex`
Baiduspider	`baiduspider`

5.4 How To Build It On Bubbles

Block all indexing for a staging or demo site:

server {
    server_name staging.example.com;

    # Block every page from every crawler
    add_header X-Robots-Tag "noindex, nofollow" always;

    # Also block via robots.txt for belt and suspenders
    location = /robots.txt {
        return 200 "User-agent: *\nDisallow: /\n";
    }
}

Block indexing for non HTML files only:

location ~* \.(pdf|doc|docx|xls|xlsx|csv|zip|tar|gz)$ {
    add_header X-Robots-Tag "noindex" always;
    expires 1d;
    add_header Cache-Control "public, max-age=86400" always;
}

This is the primary use case for X-Robots-Tag: file types where you cannot use meta tags.

Block image indexing while allowing the HTML page to index:

# Apply at the image location only, not the parent HTML
location ~* \.(jpg|jpeg|png|gif|webp|avif|svg)$ {
    add_header X-Robots-Tag "noindex" always;
}

The HTML page itself remains indexable; only direct image URLs are blocked from being indexed as standalone results.

Set snippet limits site wide:

server {
    add_header X-Robots-Tag "max-snippet:200, max-image-preview:large, max-video-preview:-1" always;
    # Other directives...
}

This is the recommended baseline for most sites. It tells Google: allow up to 200 character text snippets, allow large image previews, no limit on video preview length.

Different rules for different bots:

# In a server block, stack multiple add_header directives
add_header X-Robots-Tag "googlebot: max-snippet:200, max-image-preview:large" always;
add_header X-Robots-Tag "bingbot: max-snippet:300" always;
add_header X-Robots-Tag "noai" always;

The order does not matter for parsing. Each header is independent.

Time limited content (unavailable_after):

location = /promotion/black-friday-2026.html {
    add_header X-Robots-Tag "unavailable_after: 1 Dec 2026 00:00:00 GMT" always;
    expires 1d;
    add_header Cache-Control "public, max-age=86400" always;
}

After December 1, 2026, Google will drop this URL from search results without requiring a manual removal request. The page itself can remain accessible.

The add_header inheritance trap applies here too. A nested location's add_header wipes parent declarations. Use the snippet include pattern:

# /etc/nginx/snippets/robots-baseline.conf
add_header X-Robots-Tag "max-snippet:200, max-image-preview:large, max-video-preview:-1" always;

# In every location that needs it
location /blog/ {
    include snippets/robots-baseline.conf;
    add_header Cache-Control "public, max-age=300" always;
}

5.5 The robots.txt vs X-Robots-Tag Distinction

These two mechanisms control different things. Confusing them is the most common SEO mistake on the planet.

Mechanism	Controls
`robots.txt`	Which URLs the crawler is allowed to fetch
`X-Robots-Tag` (or `<meta robots>`)	Which fetched URLs the crawler is allowed to index

A URL blocked by robots.txt is never fetched, which means the crawler never sees the X-Robots-Tag header. If the URL has external links pointing to it, Google may index it anyway as a "URL only" entry (no title, no snippet, just the URL) based on those external signals.

To actually deindex a URL:

Remove the robots.txt block so the crawler can fetch the URL.
Add X-Robots-Tag: noindex to the response.
Wait for the crawler to re fetch and process the directive.
Then optionally add the robots.txt block back to prevent further crawling, but only after Google has processed the noindex.

Get this order wrong and the URL stays indexed forever. Get it right and the URL drops from the index within a crawl cycle (days to weeks).

5.6 How To Verify

# 1. Confirm header is present
curl -sI https://example.com/some-page | grep -i x-robots-tag

# 2. Check for a specific file type
curl -sI https://example.com/downloads/report.pdf | grep -i x-robots-tag

# 3. Check what Googlebot sees
curl -sI -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
     https://example.com/page.html | grep -i x-robots-tag

# 4. Test bot specific directives
curl -sI -A "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" \
     https://example.com/page.html | grep -i x-robots-tag

# 5. Compare HTTP header to HTML meta tag
echo "=== HTTP X-Robots-Tag ==="
curl -sI https://example.com/page.html | grep -i x-robots-tag
echo "=== HTML meta robots ==="
curl -s https://example.com/page.html | grep -i 'name="robots"'

# 6. Verify a site wide block is in effect
for path in / /about /blog /contact /robots.txt; do
    echo "=== https://staging.example.com$path ==="
    curl -sI "https://staging.example.com$path" | grep -i x-robots-tag
done

Use Google Search Console URL Inspection for the authoritative check on whether Google sees and respects the directive:

Open Google Search Console for the property.
Paste the URL into the top search bar.
Click "Test live URL".
Under "Page indexing", look for "Indexing allowed?" and the source of any "noindex" detection.

The Live Test fetches the URL with Googlebot's actual user agent and shows the headers it saw. This is the ground truth for SEO purposes.

5.7 Troubleshooting

Symptom: Page is still in Google index despite X-Robots-Tag: noindex.
Causes ranked by frequency:

The URL is blocked by robots.txt, so Googlebot cannot fetch it and never sees the header. Remove the robots.txt block, wait, then optionally re add.
The page has not been re crawled since the directive was added. Wait or request indexing via Search Console URL Inspection.
The directive has a typo (e.g. no-index instead of noindex). Verify with curl.
A CDN is stripping the header. Not applicable on Bubbles. Verify with curl --resolve directly against the origin.
The page has strong external links and Google retains a URL only entry. This is rare and resolves over time.

Symptom: PDF is being indexed and outranking the corresponding HTML page.
Cause: No X-Robots-Tag on the PDF location.
Fix: see Section 5.4. Add X-Robots-Tag: noindex to a location matching \.pdf$. If you want the PDF indexable but not outranking the HTML, add <link rel=canonical> to the HTML and X-Robots-Tag: noindex, nofollow to the PDF.

Symptom: Two X-Robots-Tag headers appear and seem to conflict.
Multiple directives across multiple headers are interpreted as a union. The most restrictive wins. X-Robots-Tag: index plus X-Robots-Tag: noindex resolves to noindex. There is no conflict; this is the spec.

Symptom: unavailable_after date passed but URL is still indexed.
Causes:

Date format is wrong. Must be RFC 850 or RFC 822 format. The Google docs recommend Day, DD Mon YYYY HH:MM:SS GMT. Verify with curl.
Google has not re crawled since the date passed. Crawl frequency depends on the URL's profile.
unavailable_after is intentionally not retroactive in a strong sense; it influences future crawls.

Symptom: AI engines (ChatGPT, Claude, Perplexity) are still using content despite nosnippet.
The nosnippet directive applies to Google search results, including AI Overviews. AI engines that crawl independently (ChatGPT search, Claude search, Perplexity) honor their own published opt out signals which often differ:

OpenAI: respect User-agent: GPTBot in robots.txt
Anthropic: respect User-agent: ClaudeBot in robots.txt
Perplexity: respect User-agent: PerplexityBot in robots.txt
Common Crawl: respect User-agent: CCBot in robots.txt Add explicit robots.txt blocks for each AI agent you want to opt out of, in addition to any X-Robots-Tag directives.

Symptom: Bot prefixed directive ignored.
Google only honors googlebot: and googlebot-news: prefixes. Other Google crawlers (Googlebot-Image, Googlebot-Video) are not controllable via this mechanism. Use robots.txt for those.

5.8 How To Fix Common Breakage

Case: Staging site indexed in Google.
Add to the server block immediately:

server {
    server_name staging.example.com;
    add_header X-Robots-Tag "noindex, nofollow" always;
    # ... other config ...
}

nginx -t && systemctl reload nginx. Then in Search Console, request removal for the entire staging subdomain. The combination of the noindex header (preventing re indexation) and the removal request (immediate effect) resolves the issue within hours.

Case: All PDFs are appearing in search and bleeding click traffic from HTML pages.
Block all PDFs from indexing:

location ~* \.pdf$ {
    add_header X-Robots-Tag "noindex" always;
    add_header Cache-Control "public, max-age=86400" always;
}

The PDFs remain downloadable; only their index entries disappear. Existing index entries drop on next crawl.

Case: A campaign page needs to be deindexed after the campaign ends.
Add unavailable_after with the campaign end date:

location = /campaigns/spring-2026.html {
    add_header X-Robots-Tag "unavailable_after: 30 Jun 2026 23:59:59 GMT" always;
}

No manual intervention required after June 30.

Case: Need to allow ClaudeBot but block GPTBot.
Use robots.txt (not X-Robots-Tag, since both ClaudeBot and GPTBot have limited bot prefix support):

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Allow: /

User-agent: *
Allow: /

Serve via:

location = /robots.txt {
    expires 1h;
    add_header Cache-Control "public, max-age=3600" always;
}

6. LINK (CANONICAL, ALTERNATE, HREFLANG, PRELOAD, PREFETCH AT HTTP)

6.1 What It Does

The Link header expresses relationships between the current response and other resources. Defined in RFC 8288. The same semantic content as the HTML <link> element, but in HTTP form. Critical for non HTML responses (PDFs cannot contain <link> tags but can have HTTP Link headers) and for performance optimization (HTTP Link is processed before HTML parsing begins).

Link: <https://example.com/page>; rel="canonical"
Link: <https://example.com/es/page>; rel="alternate"; hreflang="es-MX"
Link: </css/main.css>; rel="preload"; as="style"
Link: </hero.webp>; rel="preload"; as="image"; fetchpriority="high"
Link: <https://cdn.example.com>; rel="preconnect"

Multiple relationships can be comma separated in a single header or split across multiple Link headers. Both forms are equivalent.

6.2 Syntax

The general form:

Link: <URI>; param1=value1; param2="value 2"
Link: <URI1>; rel="..."; param="..." , <URI2>; rel="..."

Rules:

URI is wrapped in angle brackets: <https://example.com/x>.
Parameters are separated from the URI by a semicolon.
Parameters are separated from each other by semicolons.
Multiple relationships are separated by commas (use one comma per Link header line; do not nest).
Quote values with spaces or special characters using double quotes.
rel is required for almost every meaningful Link header.

6.3 Relationship Types

The values for rel come from the IANA Link Relations registry. The ones that matter on the web:

SEO and canonicalization:

rel	Purpose
`canonical`	Declare the canonical URL for this resource. Critical for SEO. The target URL is treated as the authoritative version
`alternate` (with `hreflang`)	Declare a language or region variant of the current resource. Used for international SEO
`alternate` (with `type="application/rss+xml"`)	Declare an RSS or Atom feed for this page
`alternate` (with `media`)	Declare a print stylesheet, mobile variant, or other media variant
`amphtml`	Declare the AMP version of this page. Less relevant in 2026 as AMP has been deprecated by Google
`prev`	Previous page in a paginated series. Google announced in 2019 they ignore this; still useful for browser navigation
`next`	Next page in a paginated series. Same status as prev
`up`	Parent resource in a hierarchical structure

Performance:

rel	Purpose	`as` required
`preload`	Fetch this resource at high priority for the current page. Discovered before HTML parsing finishes	Yes
`prefetch`	Fetch this resource at low priority for likely future navigation	No, but recommended
`preconnect`	Open a TCP/TLS connection to this origin in advance	No
`dns-prefetch`	Resolve DNS for this origin in advance	No
`modulepreload`	Preload a JavaScript ES module and parse it	No
`prerender`	Render this URL in a background tab. Largely replaced by Speculation Rules

Other:

rel	Purpose
`stylesheet`	This resource is a CSS stylesheet. Almost never used as HTTP Link header; use HTML link tag
`icon`	Favicon. Almost never used as HTTP Link; use HTML link tag
`manifest`	PWA manifest URL
`help`	Reference to a help document
`license`	Reference to license information for the content
`author`	Reference to author information

6.4 The `as` Parameter For Preload

rel="preload" requires an as parameter declaring the resource type. The browser uses this to set the correct request priority, Accept headers, and CSP enforcement. Wrong as value causes the browser to discard the preload and re fetch.

`as` value	For
`script`	JavaScript files
`style`	CSS files
`font`	Web fonts. Requires `crossorigin` parameter
`image`	Images (JPEG, PNG, WebP, AVIF, SVG)
`fetch`	XHR or fetch() targets (JSON APIs, etc). Requires `crossorigin` if cross origin
`document`	iframe document
`video`	Video files
`audio`	Audio files
`track`	WebVTT track files
`worker`	Web Worker scripts

6.5 fetchpriority

Modern browsers (Chrome 101+, Edge 101+, Safari 17.2+, Firefox 132+) support the fetchpriority parameter on preload and other resource hints:

Link: </hero.webp>; rel=preload; as=image; fetchpriority=high

Values: high, low, auto (default). Used to bump or demote a specific resource's priority within the browser's fetch queue. Critical for LCP optimization: marking the hero image as fetchpriority=high typically moves LCP earlier by 100 to 400 ms.

6.6 crossorigin

Required for fonts and for any cross origin preload that uses CORS:

Link: </fonts/main.woff2>; rel=preload; as=font; crossorigin
Link: <https://cdn.example.com/api.js>; rel=preload; as=script; crossorigin=anonymous

Values: anonymous (no credentials sent, default for crossorigin) or use-credentials (send cookies and auth).

6.7 How To Build It On Bubbles

Canonical for a PDF or other non HTML asset:

location = /downloads/whitepaper.pdf {
    add_header Link '<https://example.com/whitepapers/q3-2026>; rel="canonical"' always;
}

The PDF is the asset; the canonical URL is the HTML landing page about the whitepaper. This consolidates index signals onto the HTML.

Hreflang via HTTP header (for non HTML):

location = /downloads/manual.pdf {
    add_header Link '<https://example.com/downloads/manual.pdf>; rel="alternate"; hreflang="en-US", <https://example.com/es/downloads/manual.pdf>; rel="alternate"; hreflang="es-MX", <https://example.com/downloads/manual.pdf>; rel="alternate"; hreflang="x-default"' always;
}

Each PDF in a multi region cluster declares every other language version. The set must include x-default and must be reciprocal (every variant lists every other variant including itself).

Preload critical assets:

location = / {
    # Preload the hero image, critical CSS, and primary font
    add_header Link '</css/critical.css>; rel="preload"; as="style", </fonts/manrope.woff2>; rel="preload"; as="font"; crossorigin, </images/hero.webp>; rel="preload"; as="image"; fetchpriority="high"' always;
    expires 0;
    add_header Cache-Control "public, max-age=0, must-revalidate" always;
}

These preloads start fetching as soon as the response headers arrive at the browser, before HTML parsing begins. For a hero image LCP candidate, this typically improves LCP by 100 to 300 ms.

Preconnect to external origins (rare on Bubbles since we self host everything):

location / {
    add_header Link '<https://www.googletagmanager.com>; rel="preconnect", <https://www.google-analytics.com>; rel="preconnect"' always;
}

Useful when third party origins are unavoidable (analytics, tag manager). Establishes the TCP/TLS connection before the page needs the resource.

RSS feed declaration via HTTP:

location / {
    add_header Link '</rss>; rel="alternate"; type="application/rss+xml"; title="Site Feed"' always;
}

Feed readers and browsers that support RSS auto discovery will find the feed through this header.

Snippet pattern for the add_header inheritance trap:

# /etc/nginx/snippets/preloads.conf
add_header Link '</css/critical.css>; rel="preload"; as="style", </fonts/manrope.woff2>; rel="preload"; as="font"; crossorigin' always;

# In every relevant location
location ~* \.html$ {
    include snippets/preloads.conf;
    add_header Cache-Control "public, max-age=0, must-revalidate" always;
}

6.8 HTTP 103 Early Hints (the killer feature)

Nginx 1.25+ supports HTTP 103 Early Hints, which lets the server send Link preload headers before the final response is ready. The flow:

Browser requests /.
Server returns 103 Early Hints immediately with Link headers.
Browser sees the preload directives and begins fetching critical resources.
Server takes 200 to 800 ms to compute the page (database queries, template rendering).
Server returns 200 OK with the page body.
By the time the page parses, critical resources are already in cache.

Configuration:

location / {
    early_hints 103;
    add_header Link '</css/critical.css>; rel="preload"; as="style"' always;
    proxy_pass http://127.0.0.1:9090;
}

The early_hints 103 directive (nginx 1.25+) tells nginx to emit a 103 status with the Link headers before the final response. Supported in Chrome 103+, Edge 103+, Safari 17+, and Firefox 120+. Older browsers ignore the 103 and process the final response normally.

This is the single highest impact optimization for dynamic sites with slow first byte times. On a FastAPI sidecar that takes 400 ms to render, Early Hints typically reduces LCP by 200 to 400 ms with no application change.

6.9 How To Verify

# 1. Confirm Link header is present
curl -sI https://example.com/ | grep -i link

# 2. Pretty print multiple Link entries
curl -sI https://example.com/ | grep -i '^link:' | tr ',' '\n' | sed 's/^/    /'

# 3. Verify canonical for a PDF
curl -sI https://example.com/downloads/report.pdf | grep -i link

# 4. Check hreflang variants
curl -sI https://example.com/ | grep -i link | grep -o 'hreflang="[^"]*"' | sort -u

# 5. Test preload is being emitted
curl -sI https://example.com/ | grep -i link | grep -o 'rel="preload"'

# 6. Verify Early Hints (requires nginx 1.25+ and a client that supports 103)
curl -sv --http2 https://example.com/ 2>&1 | grep -i "HTTP/2 103"

# 7. Check that Googlebot sees the same Link header
curl -sI -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
     https://example.com/ | grep -i link

# 8. Verify the URLs in Link headers actually resolve
for url in $(curl -sI https://example.com/ | grep -i link | grep -oE '<[^>]+>' | tr -d '<>'); do
    echo "=== $url ==="
    curl -sI "$url" | head -1
done

6.10 Troubleshooting

Symptom: Preload appears in header but browser fetches the resource twice.
Causes:

as value mismatch. The browser fetched with as=script priority but the page actually loaded the resource as a stylesheet. Fix the as to match.
Cross origin without crossorigin parameter. Fonts especially fail this way. Add crossorigin to the Link header.
Different URL between preload and actual use. The preload was for /css/main.css but the HTML loaded /css/main.v2.css. The URLs must match byte for byte.

Symptom: Canonical via HTTP Link header is ignored by Google.
Causes:

Conflicts with an HTML <link rel=canonical> declaring a different URL. Decide which is authoritative and remove the other.
The Link header URL is relative when it should be absolute. Always use absolute URLs in rel=canonical.
The canonical target returns 404 or 5xx. Google ignores broken canonicals.
The canonical target itself canonicalizes elsewhere (a chain). Resolve to the final canonical.

Symptom: Early Hints not being sent.
Causes:

Nginx version under 1.25. Check nginx -v.
early_hints 103; directive missing from the location.
The client does not support 103 (most older HTTP clients do not). Test with curl -sv --http2.
The upstream returned the final response too quickly for nginx to emit the 103 first. Acceptable; the browser still uses the final response's Link headers.

Symptom: Hreflang cluster has missing reciprocity warnings in Search Console.
Every variant in the cluster must list every other variant including itself. Audit:

for variant in en-US es-MX en-GB; do
    URL="https://example.com/$variant/page"
    echo "=== $URL ==="
    curl -sI "$URL" | grep -i link | grep -oE 'hreflang="[^"]*"' | sort -u
done

Each should list the same set including x-default. Add any missing.

Symptom: Browser console warns "The resource was preloaded but not used within a few seconds".
The preload is wasted. Either the resource is not actually critical and should be removed from preload, or the page is not loading it correctly. Audit which resources truly contribute to LCP and preload only those.

6.11 How To Fix Common Breakage

Case: PDFs have no canonical, are competing with HTML in search.
Add canonical headers to the PDF location:

location ~ ^/downloads/(.*)\.pdf$ {
    # Compute the canonical HTML URL based on the PDF name
    # Adjust the pattern to match your site structure
    add_header Link '<https://example.com/whitepapers/$1>; rel="canonical"' always;
    add_header X-Robots-Tag "noindex" always;
}

The PDF declares the HTML page as canonical and is itself noindex. Index signals consolidate onto the HTML.

Case: LCP is slow due to hero image discovery latency.
Preload the hero image with high fetchpriority:

location = / {
    add_header Link '</images/hero-2026.webp>; rel="preload"; as="image"; fetchpriority="high"' always;
}

If the hero image varies by viewport (responsive images), preload only the largest variant and let the browser pick the right one via srcset. Or, more advanced, use the media attribute on the preload:

Link: </hero-large.webp>; rel=preload; as=image; media="(min-width: 1024px)", </hero-small.webp>; rel=preload; as=image; media="(max-width: 1023px)"

Case: Web font loads late and causes FOIT (flash of invisible text).
Preload the font with crossorigin:

location ~* \.html$ {
    add_header Link '</fonts/manrope-400.woff2>; rel="preload"; as="font"; crossorigin' always;
}

The font fetch starts immediately. Combined with font-display: swap in CSS, FOUT and FOIT are largely eliminated.

Case: Dynamic page from FastAPI sidecar is slow to TTFB.
Enable Early Hints:

location /dashboard {
    early_hints 103;
    add_header Link '</css/dashboard.css>; rel="preload"; as="style", </js/dashboard.js>; rel="preload"; as="script"' always;
    proxy_pass http://127.0.0.1:9090;
}

Modern Chrome and Edge users see LCP improve by 200 to 400 ms with no application code change.

7. LOCATION (REDIRECT TARGET WITH STATUS CODE SEMANTICS)

7.1 What It Does

Location declares the URL a client should fetch instead of the current one. Required for any 3xx redirect response. Defined in RFC 9110 Section 10.2.2.

HTTP/2 301
Location: https://example.com/new-url

The status code that accompanies Location carries the semantic meaning: is this redirect permanent or temporary, must the HTTP method be preserved, and how should clients cache the redirect.

7.2 The Five Redirect Status Codes

Code	Name	Permanent?	HTTP method preserved on redirect?	Cacheable by browser?
301	Moved Permanently	Yes	Often changed to GET historically; modern clients preserve	Yes, indefinitely
302	Found	No (temporary)	Often changed to GET; modern clients vary	Sometimes
303	See Other	No	Always changed to GET	No
307	Temporary Redirect	No	Strictly preserved	No
308	Permanent Redirect	Yes	Strictly preserved	Yes

The rules in practice:

301: use for permanent URL changes where the original URL is being abandoned. Most common redirect code. Browsers cache aggressively, which means changing the destination later requires cache busting. Search engines consolidate signals into the target URL.
302: use for temporary URL changes that may revert. The original URL remains indexed. Common for A/B tests, geographic redirects, maintenance pages.
303: use after a form POST to redirect to a confirmation page (Post/Redirect/Get pattern). Forces GET on the redirect to prevent accidental form resubmission on browser refresh.
307: same semantics as 302 (temporary) but strictly preserves the HTTP method. A POST stays a POST. Use for temporary redirects of API endpoints where method preservation matters.
308: same semantics as 301 (permanent) but strictly preserves the HTTP method. Use for permanent redirects of API endpoints.

Google's stance (John Mueller, 2016 and reaffirmed multiple times): all redirect codes are treated equivalently for SEO ranking purposes. The full PageRank passes through any of them. Choose based on technical semantics, not perceived SEO benefit.

Practical Bubbles defaults:

HTTP to HTTPS migration: 301.
www to non www (or vice versa): 301.
Old URL to new URL after a site restructure: 301.
Marketing campaign redirect: 302 (in case the campaign URL needs to change).
Geographic or device redirect: 302.
API endpoint version migration: 308 (preserves POST method).
Login redirect after form submit: 303.

7.3 How To Build It On Bubbles

Simple permanent redirect:

location = /old-page {
    return 301 https://example.com/new-page;
}

Wildcard redirect for an entire path:

location /old-section/ {
    return 301 https://example.com/new-section/$request_uri;
}

Caution: $request_uri includes the leading /old-section/. For a clean rewrite, use a different variable:

location ~ ^/old-section/(.*)$ {
    return 301 https://example.com/new-section/$1;
}

HTTP to HTTPS redirect (every Bubbles site has this):

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    return 301 https://$host$request_uri;
}

www to non www canonicalization:

server {
    listen 443 ssl;
    server_name www.example.com;
    return 301 https://example.com$request_uri;
}

server {
    listen 443 ssl;
    server_name example.com;
    # ... actual site config ...
}

Subdomain to canonical domain (for the 14+ paying client subdomains on thatwebhostingguy.com that 301 to client custom domains):

server {
    listen 443 ssl;
    server_name arcounselingandwellness.thatwebhostingguy.com;
    return 301 https://arcounselingandwellness.com$request_uri;
}

Conditional redirect based on URL pattern:

server {
    server_name example.com;

    # Redirect old blog URLs to new structure
    location ~ ^/blog/(\d{4})/(\d{2})/(.+)$ {
        return 301 https://example.com/articles/$1-$2/$3;
    }

    # Redirect old product URLs based on a map
    location /products/ {
        try_files $uri @product_redirects;
    }

    location @product_redirects {
        if ($product_redirect != "") {
            return 301 $product_redirect;
        }
        return 404;
    }
}

# Defined at http level
map $request_uri $product_redirect {
    default                       "";
    /products/old-widget          /products/new-widget;
    /products/old-gadget          /products/new-gadget;
    /products/discontinued        /;
}

Temporary redirect for maintenance page:

server {
    server_name example.com;

    set $maintenance 0;
    # Toggle this to 1 during maintenance
    # if ($maintenance) {
    #     return 302 /maintenance.html;
    # }

    # ... rest of config ...
}

API endpoint version migration (308 to preserve POST):

location /api/v1/ {
    return 308 https://example.com/api/v2/$request_uri;
}

A POST request to /api/v1/something becomes a POST to /api/v2/something. The body is preserved.

7.4 The Open Redirect Vulnerability

A redirect endpoint that takes the target URL from user input without validation is a phishing vector. Attackers craft URLs like:

https://example.com/redirect?url=https://phishing-site.com/login

The link appears to come from your domain, lending credibility. The victim clicks, lands on the phishing site, enters credentials.

Defenses:

1. Never accept full URLs from user input. Only accept path components and prepend a fixed origin:

# Wrong (vulnerable)
location = /r {
    return 302 $arg_url;
}

# Right
location = /r {
    if ($arg_to ~ ^/[a-z0-9/_-]+$) {
        return 302 https://example.com$arg_to;
    }
    return 400;
}

2. Validate against an allow list of full URLs:

map $arg_url $is_allowed_redirect {
    default                          "";
    https://example.com/promo        "1";
    https://example.com/login        "1";
    https://example.com/help         "1";
}

location = /r {
    if ($is_allowed_redirect = "") {
        return 400;
    }
    return 302 $arg_url;
}

3. Require an HMAC signature on the redirect parameter so attackers cannot forge new targets. Implement at the upstream FastAPI sidecar level for any redirect endpoint that must accept dynamic targets.

4. Never combine user input with a redirect status code without one of the above defenses. Treat redirect endpoints with the same care as authentication endpoints.

7.5 Redirect Chains And Crawl Budget

A redirect chain is a sequence of redirects: /a → /b → /c → /d. Each hop costs:

One crawler request (against finite crawl budget).
One TLS handshake plus one TCP round trip for users (latency adds up).
Potentially lost PageRank, although Google denies this in modern documentation.

Google's official guidance is to keep chains under 5 hops. Practical recommendation: always 1 hop. Audit periodically:

# Trace any chain
curl -sLI https://example.com/old-url | grep -i "^location:"

# Count hops
curl -sLI https://example.com/old-url | grep -c "^HTTP/"

A 1 indicates direct response (no redirect or single redirect). A 2 or higher indicates a chain.

When a chain is detected:

Identify the final destination.
Update the original redirect to point directly to the final destination.
Leave the intermediate redirects in place for a few months in case other consumers have cached the intermediate URLs.
Remove the intermediate redirects once they show zero traffic in logs.

7.6 How To Verify

# 1. Confirm redirect status and target
curl -sI https://example.com/old-page | grep -iE "^(HTTP|Location):"

# 2. Follow the chain
curl -sLI https://example.com/old-page | grep -iE "^(HTTP|Location):"

# 3. Verify final destination resolves
curl -sIL -o /dev/null -w "%{url_effective} %{http_code}\n" https://example.com/old-page

# 4. Verify the redirect preserves the path (for wildcard redirects)
for path in "" "/sub/page" "/file.pdf?query=1"; do
    URL="https://example.com/old-section$path"
    echo "=== $URL ==="
    curl -sI "$URL" | grep -i location
done

# 5. Test that POST is preserved (for 307 / 308)
curl -X POST -sI https://example.com/api/v1/endpoint | grep -iE "^(HTTP|Location):"

# 6. Audit: find chains
for url in https://example.com/old-1 https://example.com/old-2 https://example.com/old-3; do
    HOPS=$(curl -sLI "$url" | grep -c "^HTTP/")
    DEST=$(curl -sIL -o /dev/null -w "%{url_effective}" "$url")
    echo "$url -> $DEST ($HOPS hops)"
done

# 7. Verify Googlebot follows correctly
curl -sIL -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
     https://example.com/old-page | grep -iE "^(HTTP|Location):"

7.7 Troubleshooting

Symptom: Redirect loop (browser shows "ERR_TOO_MANY_REDIRECTS").
Causes:

Two redirects pointing at each other (/a → /b → /a).
Server block redirects HTTP to HTTPS, but the HTTPS server block also has an HTTP to HTTPS redirect, causing infinite loops.
A wildcard redirect catches its own target (location /new/ { return 301 /new/page; } redirects /new/page to /new/page). Fix: trace the chain with curl, identify the loop, rewrite one of the redirects to terminate.

Symptom: Redirect target has trailing slash but original did not (or vice versa) and Google flags duplicates.
Cause: inconsistent trailing slash policy.
Fix: pick one policy (typically without trailing slash for non directory URLs) and enforce site wide:

# Force no trailing slash on file like URLs
rewrite ^/(.*)/$ /$1 permanent;

Or the opposite. Document which is the canonical form and stick to it.

Symptom: 301 cached forever in browsers, even after fixing the redirect target.
Cause: browsers cache 301 indefinitely per the HTTP spec. The cached redirect bypasses the server entirely.
Fix options:

Change the original URL to a new path that has not been cached as a redirect.
Wait for the browser cache to expire (could be months).
Use a 302 in the future for redirects that might change.

Symptom: POST request becomes GET after redirect.
Cause: the redirect status code is 301, 302, or 303, all of which historically changed the method to GET.
Fix: use 307 (temporary) or 308 (permanent) to strictly preserve the method.

Symptom: Redirect works in browser but Googlebot still indexes the original URL.
Causes:

The redirect status is 302 or 307, which signals "do not consolidate". Change to 301 or 308 for permanent moves.
There is a chain. Google may not follow long chains. Collapse to one hop.
The original URL has internal links pointing to it. Update internal links to point directly to the new URL.
Time. Consolidation takes weeks. Be patient.

Symptom: Open redirect security alert.
The redirect endpoint accepts unvalidated user input.
Fix: implement an allow list, signature validation, or both. See Section 7.4.

7.8 How To Fix Common Breakage

Case: Old domain serving 301 to new domain, but new domain still points back to old in some links.
Audit and fix:

# Find any link to the old domain in current site
grep -r "old-domain.com" /var/www/sites/new-domain.com/

Replace every occurrence with the new domain. Then verify no infinite loop exists.

Case: Redirect chain longer than 1 hop discovered in Search Console.
List the chain:

curl -sLI https://example.com/old-url | grep -iE "^(HTTP|Location):"

Update the originating server block to redirect directly to the final destination:

location = /old-url {
    # Was: return 301 /intermediate;
    return 301 https://example.com/final-destination;
}

nginx -t && systemctl reload nginx.

Case: HTTPS redirect causing certificate warning.
The redirect target is https://example.com but the SSL certificate covers only https://www.example.com. Either:

Get a certificate covering both names (typical with Let's Encrypt: include both in -d flags).
Redirect to the name actually covered.

Case: Need to redirect a thousand old URLs to specific new URLs.
Use a map for performance:

# In http context
map $request_uri $legacy_redirect {
    default                                    "";
    /old/page-1.html                           /new/page-1;
    /old/page-2.html                           /new/page-2;
    /old/page-3.html                           /new/page-3;
    # ... thousands more ...
}

# In server context
if ($legacy_redirect != "") {
    return 301 $legacy_redirect;
}

Maps in nginx are O(1) lookups; even thousands of entries add negligible overhead.

For maps with thousands of entries, externalize:

map $request_uri $legacy_redirect {
    include /etc/nginx/maps/legacy-redirects.map;
    default "";
}

And maintain legacy-redirects.map as a simple text file with one mapping per line.

Case: Need to migrate from www to non www, preserving all paths.

server {
    listen 443 ssl;
    server_name www.example.com;
    return 301 https://example.com$request_uri;
}

Ensure your SSL certificate covers example.com (the redirect target). Verify with curl.

8. HOW THESE HEADERS INTERACT

The three headers do not act independently. Several combinations matter.

8.1 X-Robots-Tag On Redirects

A 301 or 302 response can carry an X-Robots-Tag header, but the directive is largely ignored by Google. The crawler follows the redirect and applies indexing rules to the target URL, not the redirect itself. The exception is nofollow, which can prevent the crawler from following the redirect at all, but this is rare and usually unintended.

# Almost always unnecessary
location = /old-url {
    add_header X-Robots-Tag "noindex" always;
    return 301 /new-url;
}

The X-Robots-Tag: noindex on a 301 has no effect: the old URL was already being deindexed by the 301 itself.

8.2 Canonical In HTTP vs HTML

When both Link: <...>; rel=canonical HTTP header and <link rel=canonical> HTML tag are present, Google treats them as duplicate signals. If they agree, fine. If they disagree, Google picks one based on its own canonicalization logic, which may not match either. Pick one and stick to it.

For HTML pages: use the HTML <link rel=canonical> tag because it is easier to manage with templates and is what most tools expect.

For non HTML (PDF, image, JSON): use the HTTP Link header because there is no HTML to put a tag in.

8.3 Redirect Plus Canonical Tag On The Target

The redirect target should have a self referencing canonical tag:

<!-- On the new URL: https://example.com/new-page -->
<link rel="canonical" href="https://example.com/new-page">

This is correct and expected. It reinforces that this is the canonical version, not a duplicate of some other page.

8.4 hreflang Cluster Cross References

Hreflang annotations declared via HTTP Link header or HTML <link rel=alternate hreflang> must form a reciprocal cluster. Every variant lists every other variant including itself. Missing any reciprocal reference triggers a Search Console warning and may cause hreflang to be ignored entirely.

For a PDF that exists in two languages:

# On /en/manual.pdf
add_header Link '<https://example.com/en/manual.pdf>; rel="alternate"; hreflang="en-US", <https://example.com/es/manual.pdf>; rel="alternate"; hreflang="es-MX", <https://example.com/en/manual.pdf>; rel="alternate"; hreflang="x-default"' always;

# On /es/manual.pdf
add_header Link '<https://example.com/en/manual.pdf>; rel="alternate"; hreflang="en-US", <https://example.com/es/manual.pdf>; rel="alternate"; hreflang="es-MX", <https://example.com/en/manual.pdf>; rel="alternate"; hreflang="x-default"' always;

Both PDFs declare the same cluster. The cluster includes both files and an x-default.

8.5 Preload And Cache-Control

Preloaded resources should still have correct Cache-Control headers. If the preloaded asset has Cache-Control: no-store, the browser fetches it via the preload, but cannot store it for use by the actual page reference. Result: double fetch.

Ensure preloaded resources are cacheable. Combine with framework-http-caching-headers.md guidance: fingerprinted assets should be Cache-Control: public, max-age=31536000, immutable and are perfect preload candidates.

8.6 Location Plus Vary

A redirect response should typically not have Vary headers. The redirect itself does not vary by request headers. However, if the redirect target differs based on Accept-Language (geographic or language based redirect), then Vary: Accept-Language is required to prevent caches from serving the wrong target to a different audience.

location = / {
    # Redirect based on Accept-Language
    if ($http_accept_language ~* "^es") {
        add_header Vary "Accept-Language" always;
        return 302 /es/;
    }
    add_header Vary "Accept-Language" always;
    return 302 /en/;
}

Note: language based redirects are a poor pattern for SEO. They confuse crawlers and prevent users from sharing links between languages. Prefer letting the user select their language explicitly and remembering the choice via cookie or path.

9. ASSET CLASS AND USE CASE RECIPES

Each block is paste ready. Copy the ones relevant to your build.

9.1 Production HTML page (full SEO header set)

location ~* \.html$ {
    # X-Robots-Tag: allow indexing, set snippet limits
    add_header X-Robots-Tag "max-snippet:200, max-image-preview:large, max-video-preview:-1" always;

    # Link: preload critical resources
    add_header Link '</css/critical.css>; rel="preload"; as="style", </fonts/manrope.woff2>; rel="preload"; as="font"; crossorigin' always;

    # Cache directives from framework-http-caching-headers.md
    add_header Cache-Control "public, max-age=0, must-revalidate" always;
}

9.2 PDF (block from index, canonical to HTML)

location ~ ^/whitepapers/(.*)\.pdf$ {
    add_header X-Robots-Tag "noindex" always;
    add_header Link '<https://example.com/whitepapers/$1>; rel="canonical"' always;
    add_header Cache-Control "public, max-age=86400" always;
    add_header Content-Disposition "attachment" always;
}

9.3 Image asset (allow indexing for image search)

location ~* \.(jpg|jpeg|png|webp|avif)$ {
    # Default: allow image indexing
    # If you want to block: add_header X-Robots-Tag "noindex" always;
    add_header Cache-Control "public, max-age=2592000" always;
}

9.4 Image asset (block from image search but page itself indexable)

location ~* \.(jpg|jpeg|png|webp|avif)$ {
    add_header X-Robots-Tag "noindex" always;
    add_header Cache-Control "public, max-age=2592000" always;
}

9.5 Staging or development subdomain (full block)

server {
    server_name staging.example.com;

    # Universal block
    add_header X-Robots-Tag "noindex, nofollow" always;

    # Plus robots.txt for belt and suspenders
    location = /robots.txt {
        return 200 "User-agent: *\nDisallow: /\n";
        add_header Content-Type "text/plain";
    }

    # Plus HTTP basic auth as a third layer
    auth_basic "Staging";
    auth_basic_user_file /etc/nginx/.staging-htpasswd;
}

9.6 API endpoint (do not index, do not follow)

location /api/ {
    add_header X-Robots-Tag "noindex, nofollow" always;
    proxy_pass http://127.0.0.1:9090;
}

9.7 Search result page (do not index, do follow)

location /search {
    add_header X-Robots-Tag "noindex, follow" always;
    # The follow allows crawlers to discover linked content
}

Internal search result pages are duplicate content for each query and should not be indexed. Allow follow so crawlers can discover the linked actual content pages.

9.8 Paginated section (allow indexing of all pages, no rel prev/next)

location ~ ^/blog/page/([0-9]+)$ {
    # Google ignored prev/next as of 2019; do not bother with Link rel=prev/next
    add_header X-Robots-Tag "max-snippet:160, max-image-preview:standard" always;
}

9.9 Time limited campaign page

location = /promotions/black-friday-2026 {
    add_header X-Robots-Tag "unavailable_after: 1 Dec 2026 00:00:00 GMT" always;
}

9.10 Permanent www to non www redirect

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    server_name www.example.com;
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    return 301 https://example.com$request_uri;
}

9.11 HTTP to HTTPS redirect

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    return 301 https://example.com$request_uri;
}

9.12 Old blog URL structure to new

# Old: /2024/05/post-title
# New: /blog/post-title
location ~ ^/(\d{4})/(\d{2})/(.+)$ {
    return 301 /blog/$3;
}

9.13 Subdomain to client custom domain (Bubbles wildcard pattern)

server {
    listen 443 ssl;
    server_name handledtax.thatwebhostingguy.com;
    ssl_certificate /etc/letsencrypt/live/thatwebhostingguy.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/thatwebhostingguy.com/privkey.pem;
    return 301 https://handledtax.com$request_uri;
}

9.14 API version migration (preserve POST method)

location /api/v1/ {
    return 308 /api/v2/$request_uri;
}

9.15 Maintenance mode toggle

# At server level
set $maintenance 0;

if (-f /var/www/sites/example.com/.maintenance) {
    set $maintenance 1;
}

location / {
    if ($maintenance) {
        return 302 /maintenance.html;
    }
    try_files $uri $uri/ $uri.html =404;
}

location = /maintenance.html {
    # Allow access to this even in maintenance mode
    internal;
}

To enable maintenance: touch /var/www/sites/example.com/.maintenance. To disable: rm /var/www/sites/example.com/.maintenance. No nginx reload required.

10. BUBBLES NGINX REFERENCE BLOCK (PASTE READY)

The complete SEO header stanza, layered with caching (framework-http-caching-headers.md) and content (framework-http-content-headers.md).

# /etc/nginx/sites-available/example.com

# HTTP to HTTPS redirect
server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;
    return 301 https://example.com$request_uri;
}

# www to non www
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    server_name www.example.com;
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    return 301 https://example.com$request_uri;
}

# Main canonical server
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    listen 443 quic reuseport;
    listen [::]:443 quic reuseport;
    server_name example.com;
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    root /var/www/sites/example.com;
    index index.html;

    # HTTP/3 advertisement
    add_header Alt-Svc 'h3=":443"; ma=86400' always;

    # ===== HTML PAGES =====
    location ~* \.html$ {
        include snippets/common-security-headers.conf;
        # Baseline indexing directives
        add_header X-Robots-Tag "max-snippet:200, max-image-preview:large, max-video-preview:-1" always;
        # Preload critical resources
        add_header Link '</css/critical.css>; rel="preload"; as="style", </fonts/manrope.woff2>; rel="preload"; as="font"; crossorigin, </images/hero.webp>; rel="preload"; as="image"; fetchpriority="high"' always;
        # Caching from framework-http-caching-headers.md
        add_header Cache-Control "public, max-age=0, must-revalidate" always;
        # Language from framework-http-content-headers.md
        add_header Content-Language "en-US" always;
    }

    # ===== PDFs (noindex, canonical to HTML landing) =====
    location ~ ^/whitepapers/(.*)\.pdf$ {
        include snippets/common-security-headers.conf;
        add_header X-Robots-Tag "noindex" always;
        add_header Link '<https://example.com/whitepapers/$1>; rel="canonical"' always;
        add_header Content-Disposition "attachment" always;
        add_header Cache-Control "public, max-age=86400" always;
    }

    # ===== IMAGES (allow image search indexing) =====
    location ~* \.(jpg|jpeg|png|webp|avif|svg|gif)$ {
        include snippets/common-security-headers.conf;
        add_header Cache-Control "public, max-age=2592000" always;
        # No X-Robots-Tag: default behavior allows image indexing
    }

    # ===== FONTS, CSS, JS (no indexing needed; cache forever) =====
    location ~* \.(woff2|woff|ttf|otf|css|js|mjs)$ {
        include snippets/common-security-headers.conf;
        add_header Cache-Control "public, max-age=31536000, immutable" always;
    }

    # ===== INTERNAL SEARCH (noindex but follow) =====
    location /search {
        include snippets/common-security-headers.conf;
        add_header X-Robots-Tag "noindex, follow" always;
    }

    # ===== API (sidecar, noindex) =====
    location /api/ {
        include snippets/common-security-headers.conf;
        add_header X-Robots-Tag "noindex, nofollow" always;
        # Early Hints for dynamic responses
        early_hints 103;
        add_header Link '</css/dashboard.css>; rel="preload"; as="style"' always;
        proxy_pass http://127.0.0.1:9090;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # ===== ROBOTS.TXT, SITEMAPS, AI/LLM FILES =====
    location = /robots.txt {
        expires 1h;
        add_header Cache-Control "public, max-age=3600" always;
    }

    location = /sitemap.xml {
        expires 10m;
        add_header Cache-Control "public, max-age=600" always;
    }

    location ~* /(llms\.txt|ai\.txt|humans\.txt|\.well-known/security\.txt)$ {
        expires 1d;
        add_header Cache-Control "public, max-age=86400" always;
    }

    # ===== LEGACY URL MAP REDIRECTS =====
    if ($legacy_redirect != "") {
        return 301 $legacy_redirect;
    }

    # ===== DEFAULT =====
    location / {
        include snippets/common-security-headers.conf;
        add_header X-Robots-Tag "max-snippet:200, max-image-preview:large, max-video-preview:-1" always;
        add_header Content-Language "en-US" always;
        add_header Cache-Control "public, max-age=0, must-revalidate" always;
        try_files $uri $uri/ $uri.html =404;
    }
}

# Legacy URL map (defined at http level in nginx.conf or included)
# map $request_uri $legacy_redirect {
#     default                       "";
#     /old/page-1.html              /new/page-1;
#     /old/page-2.html              /new/page-2;
# }

The shared security header snippet at /etc/nginx/snippets/common-security-headers.conf:

add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
add_header Permissions-Policy "camera=(), microphone=(), geolocation=(), payment=()" always;

After deploying:

nginx -t && systemctl reload nginx

11. AUDIT CHECKLIST

Run through these 50 items for any production site.

X-Robots-Tag

[ ] Production HTML pages have X-Robots-Tag: max-snippet:200, max-image-preview:large, max-video-preview:-1 (or appropriate values, NOT noindex).
[ ] Staging or development subdomains have X-Robots-Tag: noindex, nofollow.
[ ] PDFs at customer facing URLs have X-Robots-Tag: noindex (unless they should be indexed).
[ ] CSV exports and other downloadable data files have X-Robots-Tag: noindex.
[ ] API endpoints have X-Robots-Tag: noindex, nofollow.
[ ] Internal search result pages have X-Robots-Tag: noindex, follow.
[ ] No production HTML page has X-Robots-Tag: noindex accidentally.
[ ] No directive uses deprecated tokens (noodp, noydir, noyaca).
[ ] Bot prefixed directives (googlebot:, bingbot:) use valid bot tokens.
[ ] unavailable_after dates use RFC 850 / RFC 822 format.
[ ] nosnippet is used only when intentional (it blocks AI Overviews too).
[ ] X-Robots-Tag from HTTP and <meta name="robots"> from HTML do not conflict on the same page.

Link header

[ ] HTML pages preload critical CSS, fonts, and hero image via Link HTTP header.
[ ] Font preloads include crossorigin parameter.
[ ] All preload URLs use absolute or relative paths that match the actual usage in HTML.
[ ] Preload as values are correct (style, script, font, image, etc).
[ ] LCP candidate image has fetchpriority="high".
[ ] PDFs and other non HTML assets that need canonicalization use Link: <...>; rel="canonical".
[ ] Hreflang annotations form a reciprocal cluster (every variant lists every other variant).
[ ] Hreflang cluster includes x-default.
[ ] No rel=prev or rel=next headers (Google ignored these in 2019).
[ ] Early Hints (103) is enabled for dynamic responses where TTFB exceeds 200 ms.
[ ] Nginx version is 1.25+ if Early Hints is used.

Location and redirects

[ ] HTTP requests to port 80 always redirect to HTTPS with 301.
[ ] www to non www (or vice versa) is consistent and uses 301.
[ ] No redirect chain exceeds 1 hop.
[ ] All 301 redirect targets return 200 OK (not another redirect or error).
[ ] Permanent redirects use 301 or 308 (not 302 or 307).
[ ] Temporary redirects use 302 or 307.
[ ] API version migration redirects use 308 to preserve POST method.
[ ] No redirect endpoint accepts unvalidated URL parameters from user input (no open redirects).
[ ] Trailing slash policy is consistent site wide.
[ ] Internal links point directly to the canonical URL (no internal links to redirect sources).
[ ] Redirect targets are absolute URLs or path absolute (no protocol relative or relative redirects).
[ ] SSL certificate covers all redirect target hostnames.

Cross cutting

[ ] nginx -t passes without warnings.
[ ] nginx -T shows all expected SEO directives in effect for each server block.
[ ] Search Console URL Inspection shows correct indexing status for sample URLs.
[ ] No URL is in Google's index that should be noindex.
[ ] No URL is noindex that should be in Google's index.
[ ] No URL is blocked by robots.txt that needs to be noindex (the noindex would not be seen).
[ ] PDF count in Search Console matches expected count (audits whether unwanted PDFs are indexed).
[ ] Hreflang errors in Search Console are at zero.
[ ] Coverage report in Search Console shows no unexpected "Excluded by noindex" entries.
[ ] Redirect chain audit (run periodically) shows zero chains over 1 hop.
[ ] All preload directives have corresponding usage in HTML (no orphan preloads wasting bandwidth).
[ ] LCP image is preloaded with fetchpriority=high.
[ ] Early Hints (if used) actually arrives before final response (verified via curl --http2).
[ ] Open redirect scanner reports clean.
[ ] Internal link audit shows no broken links to redirected URLs.

A site that passes all 50 has indexing, resource hints, and forwarding correctly configured.

12. COMMON PITFALLS

Fifteen patterns to recognize and avoid.

Pitfall 1: noindex behind robots.txt block.
Symptom: page stays indexed forever.
Why it breaks: robots.txt prevents fetching; crawler never sees the noindex directive.
Fix: remove robots.txt block, let crawler fetch and process noindex, then optionally re add.

Pitfall 2: Conflicting X-Robots-Tag and meta robots.
Symptom: page sometimes indexed, sometimes not; behavior inconsistent.
Why it breaks: both signals are read; the most restrictive wins, but team members assume one or the other is authoritative.
Fix: align both. Pick one as the source of truth (typically HTTP header for non HTML, HTML tag for HTML pages) and maintain consistency.

Pitfall 3: noindex on a redirected URL.
Symptom: the noindex header is wasted; the 301 itself handles deindexing.
Why it breaks: a 301 response signals the URL has moved; crawlers process the redirect, not the directives on the old URL.
Fix: remove noindex from redirect responses unless you have a specific reason.

Pitfall 4: Canonical pointing to a redirected URL.
Symptom: index signals are diluted; Google follows the chain and selects its own canonical.
Why it breaks: canonical should point to the final canonical URL, not to an intermediate that redirects.
Fix: audit all canonicals. Each must resolve to a 200 OK page that self references as canonical.

Pitfall 5: Preload with wrong as value.
Symptom: browser fetches the resource twice (preload, then real load).
Why it breaks: the browser cannot match the preloaded resource to the actual request because the priority and headers differ.
Fix: ensure as matches the resource type exactly. Font preload requires as=font AND crossorigin. Image is as=image. Script is as=script. Style is as=style.

Pitfall 6: Preload of a resource the page does not use.
Symptom: bandwidth wasted; console warning "preloaded but not used".
Why it breaks: a developer added a preload speculatively, then the corresponding HTML reference was removed or changed.
Fix: audit preloads. Remove any not actually used within a few seconds of page load.

Pitfall 7: Hreflang cluster missing reciprocity.
Symptom: Search Console hreflang errors; Google falls back to default behavior.
Why it breaks: hreflang requires every variant to list every other variant including itself.
Fix: build hreflang generation programmatically so all variants always list the full set.

Pitfall 8: Redirect chain longer than 1 hop.
Symptom: crawl budget waste; PageRank consolidation slow.
Why it breaks: nobody updated the source of a redirect when the destination itself was redirected.
Fix: audit periodically with curl -sLI. Collapse to direct redirects.

Pitfall 9: Open redirect endpoint.
Symptom: security alert; phishing campaigns abuse the redirector.
Why it breaks: endpoint accepts arbitrary target URL from user input without validation.
Fix: allow list, signature, or only accept path components. See Section 7.4.

Pitfall 10: Wrong redirect code for HTTP method preservation.
Symptom: POST becomes GET after redirect; API calls fail.
Why it breaks: 301, 302, 303 historically change POST to GET.
Fix: use 307 (temporary) or 308 (permanent) for redirects that must preserve POST.

Pitfall 11: 301 cached forever, hard to undo.
Symptom: changed the redirect target but browsers still go to the old destination.
Why it breaks: per HTTP spec, browsers may cache 301 indefinitely.
Fix prevention: use 302 for redirects you might change in the future. Fix existing: change the source URL or wait for cache expiration.

Pitfall 12: Self redirecting URL.
Symptom: redirect loop; ERR_TOO_MANY_REDIRECTS.
Why it breaks: a location block redirects, and the target also matches the same location pattern.
Fix: tighten the regex match, or check the request URI before redirecting.

Pitfall 13: Redirect target on a different SSL certificate.
Symptom: browser shows certificate warning after redirect.
Why it breaks: redirecting www.example.com to example.com requires the certificate to cover example.com.
Fix: ensure the cert covers all redirect targets. With Let's Encrypt, include all hostnames in the -d flags.

Pitfall 14: Add_header inheritance trap on X-Robots-Tag.
Symptom: noindex applied to staging site disappears from specific file types.
Why it breaks: a location block with add_header wipes parent declarations.
Fix: snippet include pattern in every location.

Pitfall 15: Early Hints emitted but contains preload for a non existent resource.
Symptom: 404 in network panel for the preloaded resource; LCP not improved.
Why it breaks: preload URL is wrong or the resource was renamed.
Fix: audit Early Hints preload URLs. Each must resolve to 200 OK.

13. DIAGNOSTIC COMMANDS

Reference of every command useful for SEO header investigation.

Inspect SEO headers

# Full SEO family at once
curl -sI https://example.com/page.html | grep -iE "^(x-robots-tag|link|location|content-language):"

# As Googlebot
curl -sI -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
     https://example.com/page.html

# As Bingbot
curl -sI -A "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" \
     https://example.com/page.html

# As ClaudeBot
curl -sI -A "Mozilla/5.0 (compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" \
     https://example.com/page.html

# As GPTBot
curl -sI -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.0; +https://openai.com/gptbot" \
     https://example.com/page.html

X-Robots-Tag verification

# Confirm directive
curl -sI https://example.com/page.html | grep -i x-robots-tag

# Check all non HTML file types
for ext in pdf png jpg svg woff2 json xml csv zip; do
    URL="https://example.com/test.$ext"
    echo "=== $URL ==="
    curl -sI "$URL" 2>/dev/null | grep -i x-robots-tag
done

# Check for accidental noindex on production pages
for url in / /about /services /contact /blog; do
    echo "=== $url ==="
    curl -sI "https://example.com$url" | grep -i x-robots-tag
done

# Compare HTTP and HTML signals
echo "HTTP:"; curl -sI https://example.com/page.html | grep -i x-robots-tag
echo "HTML:"; curl -s https://example.com/page.html | grep -oE '<meta[^>]*name="robots"[^>]*>'

Link header verification

# Confirm Link header presence and parse
curl -sI https://example.com/ | grep -i '^link:' | sed 's/, /\n    /g'

# Check canonical
curl -sI https://example.com/page.html | grep -oE 'rel="canonical"[^,]*'

# Check hreflang cluster
curl -sI https://example.com/ | grep -oE 'hreflang="[^"]*"' | sort -u

# Verify preload targets resolve
for url in $(curl -sI https://example.com/ | grep -i '^link:' | grep -oE '<[^>]+>' | tr -d '<>'); do
    STATUS=$(curl -sI "$url" | head -1)
    echo "$url -> $STATUS"
done

# Test Early Hints
curl -sv --http2 https://example.com/ 2>&1 | grep -iE "HTTP/2 (103|200)"

Location and redirect verification

# Single redirect
curl -sI https://example.com/old-url | grep -iE "^(HTTP|Location):"

# Full chain
curl -sLI https://example.com/old-url | grep -iE "^(HTTP|Location):"

# Count hops
HOPS=$(curl -sLI https://example.com/old-url | grep -c "^HTTP/")
echo "Hops: $HOPS"

# Final URL and status
curl -sILo /dev/null -w "Final: %{url_effective} Status: %{http_code} Redirects: %{num_redirects}\n" \
     https://example.com/old-url

# Audit a list of old URLs
while IFS= read -r url; do
    FINAL=$(curl -sILo /dev/null -w "%{url_effective}" "$url")
    HOPS=$(curl -sLI "$url" | grep -c "^HTTP/")
    STATUS=$(curl -sILo /dev/null -w "%{http_code}" "$url")
    echo "$url -> $FINAL ($HOPS hops, final $STATUS)"
done < urls-to-audit.txt

# Verify POST is preserved on 307/308
curl -X POST -sI https://example.com/api/v1/something | grep -iE "^(HTTP|Location):"

# Detect redirect loop (timeout after 5 seconds, max 10 redirects)
curl -sLI --max-redirs 10 --max-time 5 https://example.com/maybe-looping

Server side investigation

# Find every X-Robots-Tag in the config
grep -rn "X-Robots-Tag" /etc/nginx/

# Find every Link header
grep -rn "Link" /etc/nginx/sites-available/

# Find every redirect (return 301/302/etc, rewrite ... permanent)
grep -rn -E "return (301|302|303|307|308)|rewrite.*(permanent|redirect)" /etc/nginx/sites-available/

# Find any open redirect patterns (variable in return target)
grep -rn -E "return (301|302|303|307|308).*\\\$" /etc/nginx/sites-available/

# Check active config
nginx -T 2>/dev/null | grep -iE "x-robots-tag|^[[:space:]]*link |return 30|early_hints"

# Reload after changes
nginx -t && systemctl reload nginx

Google Search Console verification

The authoritative SEO check. For each URL you care about:

Open Search Console for the property.
Top search bar: paste the URL.
Click Enter.
Wait for the report to load. Sections:
- URL is on Google: indexed correctly.
- URL is not on Google: not indexed; reason shown.
Click "Test live URL" to fetch with Googlebot now and see headers.
Under "Page indexing":
- "Indexing allowed?" should be "Yes" for pages you want indexed.
- "User-declared canonical" and "Google-selected canonical" should match.
Under "Crawl":
- "Last crawl" date should be recent.
- "Crawled as" should be Googlebot Smartphone (mobile first).

This is ground truth. Any disagreement between curl output and Search Console means the URL has not been re crawled yet.

14. CROSS-REFERENCES

framework-http-caching-headers.md: Cache-Control, ETag, Last-Modified, Expires, Vary, Age. Preloaded resources should have correct cache directives; redirects often need no caching; X-Robots-Tag has no caching implications.
framework-http-content-headers.md: Content-Type, Content-Language, Content-Encoding, Content-Length, Content-Disposition. Content-Language pairs with hreflang declared via Link header. Content-Disposition pairs with X-Robots-Tag for downloadable file handling.
UNIVERSAL-RANKING-FRAMEWORK.md: the master ranking reference. Section 24 (Bubbles Nginx config) inherits SEO header configuration from here.
SEO-BUILD-REFERENCE.md v2.4: the build playbook.
framework-hreflang.md: full coverage of multi region SEO. Companion to Link header hreflang in Section 6 here.
framework-robotstxt-crawlbudget.md: robots.txt syntax and crawl budget management. Pairs with X-Robots-Tag distinction in Section 5.5.
framework-canonicalization.md: canonical URL strategy. Companion to Link rel=canonical and the redirect/canonical interaction in Section 8.
framework-corewebvitals.md: LCP, INP, CLS targets. Preload via Link header and Early Hints (103) directly improve LCP.
framework-security-headers.md: HSTS, CSP, X-Frame-Options. Open redirect security pairs with the broader security header stack.
Google X-Robots-Tag documentation: https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag
Google redirects documentation: https://developers.google.com/search/docs/crawling-indexing/301-redirects
Google international and multi region: https://developers.google.com/search/docs/specialty/international/overview
MDN Link header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Link
MDN Location header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Location
RFC 8288 (Web Linking): https://www.rfc-editor.org/rfc/rfc8288
RFC 9110 (HTTP Semantics, redirects and Location): https://www.rfc-editor.org/rfc/rfc9110
HTTP 103 Early Hints: https://www.rfc-editor.org/rfc/rfc8297

APPENDIX A: ONE PAGE QUICK REFERENCE

X-Robots-Tag

Production HTML default:

X-Robots-Tag: max-snippet:200, max-image-preview:large, max-video-preview:-1

Block from indexing:

X-Robots-Tag: noindex, nofollow

Time limited:

X-Robots-Tag: unavailable_after: 31 Dec 2026 23:59:59 GMT

Bot specific:

X-Robots-Tag: googlebot: nosnippet
X-Robots-Tag: bingbot: noarchive

Link

Canonical for non HTML:

Link: <https://example.com/canonical>; rel="canonical"

Preload hero image (LCP):

Link: </images/hero.webp>; rel="preload"; as="image"; fetchpriority="high"

Preload font:

Link: </fonts/main.woff2>; rel="preload"; as="font"; crossorigin

Hreflang (reciprocal cluster, set on every variant):

Link: <https://example.com/>; rel="alternate"; hreflang="en-US", <https://example.com/es/>; rel="alternate"; hreflang="es-MX", <https://example.com/>; rel="alternate"; hreflang="x-default"

Location

Scenario	Code
Permanent URL change	301
Temporary URL change	302
After form POST (force GET on next)	303
Temporary, preserve POST	307
Permanent, preserve POST	308

HTTP to HTTPS:

return 301 https://$host$request_uri;

www to non www:

return 301 https://example.com$request_uri;

Old URL pattern to new:

location ~ ^/old/(.*)$ { return 301 /new/$1; }

Three commands every operator should know:

# Confirm SEO headers on a URL
curl -sI https://example.com/page.html | grep -iE "^(x-robots-tag|link|location):"

# Trace a redirect chain
curl -sLI https://example.com/old-url | grep -iE "^(HTTP|Location):"

# Apply nginx changes
nginx -t && systemctl reload nginx

Two end to end tests:

# Verify a noindex directive is in effect
curl -sI -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
     https://example.com/staging-page | grep -i x-robots-tag
# Should show: x-robots-tag: noindex, nofollow

# Verify a redirect resolves in one hop
curl -sILo /dev/null -w "Hops: %{num_redirects} Final: %{url_effective} Status: %{http_code}\n" \
     https://example.com/old-url
# Should show: Hops: 1 Final: https://example.com/new-url Status: 200

If both produce expected output, the SEO header stack is correctly wired.

End of framework-http-seo-headers.md.

Related references

Authored at ThatDevPro, the SEO and engine-optimization arm of ThatDeveloperGuy.