DEV Community

Manish Kumar
Manish Kumar

Posted on

Bringing the Semantic Web to GitLab Pages: Implementing RFC 9110 Content Negotiation in Go

The Semantic Web and Linked Open Data (LOD) initiatives rely on a foundational principle: a single persistent URI (e.g., https://example.com/alice or https://example.com/ontology) should be accessible to both humans in their web browsers and machines consuming semantic data models.

Through HTTP Content Negotiation, a browser requesting Accept: text/html receives a styled HTML page, while an RDF crawler or knowledge-graph ingestion pipeline requesting Accept: text/turtle or Accept: application/ld+json receives machine-readable triples from the exact same URI.

In my recent contribution to GitLab Pages (MR !1384, closing Issue #1135), I designed and implemented RFC 9110-compliant content negotiation for static sites hosted on GitLab Pages.

Here is an engineering deep-dive into how we implemented media-type negotiation, specificity weighting, and cache safety in Go.


1. The Problem: Extensionless URLs in Static Hosting

Historically, GitLab Pages mapped extensionless HTTP requests (GET /research-paper or GET /) exclusively to .html (or index.html).

For scientific organizations, academic publishers, and open-data projects hosting ontologies on GitLab Pages:

  • A browser requesting /schema received schema.html.
  • A machine or graph database sending Accept: application/rdf+xml or Accept: text/turtle still received HTML or a 404 error.

Because static sites couldn't resolve semantic file variants without forcing users to type explicit extensions like schema.ttl, GitLab Pages could not serve Linked Data in compliance with W3C best practices.


2. Designing an RFC 9110 Compliant Accept Parser in Go

HTTP content negotiation is defined in RFC 9110 (Section 12.5.1). A robust content negotiation parser must handle:

  1. Quality Values (q): Entries with higher weights (q=1.0 vs q=0.5) take priority. Zero quality (q=0) rejects the type.
  2. Specificity Weighting:
    • Exact type/subtype: application/ld+json (highest precedence)
    • Subtype wildcards: text/* (medium precedence)
    • Global wildcards: */* (lowest precedence)

We built a parser that parses candidate media types and orders them by quality and specificity:

type MediaRange struct {
    Type       string
    Subtype    string
    Quality    float64
    Specificity int
}
Enter fullscreen mode Exit fullscreen mode

When a client sends:

Accept: text/html;q=0.8, text/turtle;q=0.9, */*;q=0.1
Enter fullscreen mode Exit fullscreen mode

The parser evaluates text/turtle ($q=0.9$) first, followed by text/html ($q=0.8$), and finally the wildcard fallback.


3. Mapping Media Types to Semantic File Extensions

GitLab Pages now recognizes common Linked Data and Semantic Web formats:

MIME Media Type File Extension Format Description
text/turtle .ttl Terse RDF Triple Language
application/ld+json .jsonld JSON for Linked Data
application/rdf+xml .rdf Resource Description Framework XML
application/n-triples .nt N-Triples
text/n3 .n3 Notation3
application/owl+xml .owl Web Ontology Language
text/html, application/xhtml+xml .html HyperText Markup Language

Resolution Algorithm:

  1. When receiving a request for /dataset:
  2. Inspect the client's Accept header.
  3. For each candidate media type in descending preference order, test if the corresponding file exists on disk (e.g., dataset.ttl, dataset.jsonld, dataset.rdf).
  4. If a match is found, serve that file with the matching Content-Type header.
  5. If no semantic file matches or no Accept header is provided, safely fall back to the default dataset.html.

4. Cache Correctness: Why Vary: Accept is Crucial

In static web hosting and CDNs, caching responses correctly is just as important as serving them.

If a CDN caches dataset.html for GET /dataset, a subsequent request from an RDF crawler with Accept: text/turtle would mistakenly receive the cached HTML from the CDN edge server!

To prevent cache poisoning:

  • Whenever content negotiation takes place, GitLab Pages appends:
  Vary: Accept
Enter fullscreen mode Exit fullscreen mode
  • This instructs CDNs, reverse proxies, and browsers that the cached representation depends strictly on the incoming Accept header.
  • Direct extension requests (e.g., GET /dataset.ttl) bypass negotiation and do not emit unnecessary Vary headers.

5. Testing & Verification

The feature is covered by unit tests and end-to-end integration tests in TestReader_ContentNegotiation:

  • Directory Indexes: GET / with Accept: application/rdf+xml serves index.rdf.
  • Quality Fallbacks: Verifying that q=0.9 wins over q=0.7.
  • Graceful HTML Fallback: If an unsupported media type is requested, standard HTML is delivered without breaking normal user navigation.
go test -v ./internal/vfs/zip -run '^TestReader_ContentNegotiation$'
Enter fullscreen mode Exit fullscreen mode

Conclusion

With this contribution, thousands of static websites hosted on GitLab Pages can now serve both human-friendly documentation and Linked Open Data from identical canonical URLs.

Special thanks to the GitLab Pages maintainers for their thorough reviews and guidance throughout this implementation!


Check out the merged implementation on GitLab:

👉 GitLab Pages MR !1384


Author: Manish Kumar

GitLab: @manishpatel00

Top comments (0)