The Quiet Files That Make a Website Trustworthy

I used to think a website was only about visible things: pages, animations, speed, and "wow" UI.

Then one day, after deploying my portfolio, a scanner flagged something tiny:

"security.txt not configured."

At first, I ignored it.

But that one warning sent me down a rabbit hole, and I learned something important:

Some of the most important website files are the ones users never see.

They are quiet, but they build trust, improve security, and help search engines understand your site.

In this post, I will explain these files in a simple way:

• .well-known
• security.txt
• pgp-key.txt (your GPG public key)
• robots.txt
• sitemap.xml

If you are a beginner, this is for you.

TL;DR

If your website is public, these files should exist:

• /.well-known/security.txt: how to report vulnerabilities.
• /.well-known/pgp-key.txt: your public encryption key for sensitive reports.
• /robots.txt: crawler guidance.
• /sitemap.xml: map of your public URLs.

Together, they make your site easier to trust, crawl, and maintain.

Imagine Your Website as a City

Your homepage is the city center.

Your projects page is the museum.

Your contact page is city hall.

Now imagine visitors are not only humans.

Some are bots, crawlers, and security researchers.

How do they know:

• Where to report a vulnerability?
• Whether encrypted contact is available?
• What pages they should crawl?
• Where your important pages are listed?

That is where these files help:

• .well-known/ (a standard place for machine-readable files).
• security.txt (how to report vulnerabilities).
• pgp-key.txt (how to send encrypted security reports).
• robots.txt (crawler guidance).
• sitemap.xml (map of your public pages).

1. .well-known: The Official Notice Board

Think of .well-known as your website's official notice board for machines.

It is a standards-based location where tools expect to find important metadata.

Example paths:

• https://yourdomain.com/.well-known/security.txt
• https://yourdomain.com/.well-known/pgp-key.txt

Why It Matters

• Security scanners check this location first.
• Automation tools trust standard paths.
• It reduces confusion and makes your site look professional.

Quick Tip

• Keep these files simple text.
• Keep the URL stable.
• Avoid moving them around once published.

2. security.txt: "If You Find a Bug, Contact Me Here"

security.txt is like a public help desk for security researchers.

If someone finds a vulnerability, they should not have to guess where to report it.

A practical security.txt can look like this:

Contact: mailto:[e-mail]
Contact: https://yourdomain.com/contact
Encryption: https://yourdomain.com/.well-known/pgp-key.txt
Acknowledgments: https://yourdomain.com/security/thanks
Policy: https://yourdomain.com/contact
Canonical: https://yourdomain.com/.well-known/security.txt
Expires: 2027-04-20T00:00:00.000Z
Preferred-Languages: en

What Each Line Means

• Contact: where someone can report a vulnerability
• Encryption: where your public key is published
• Acknowledgments: page where you thank valid reporters
• Policy: your disclosure policy or contact workflow
• Canonical: the official URL of this file
• Expires: validity date (update this before expiry)
• Preferred-Languages: language preference for reports

Why It Matters

• Encourages responsible disclosure.
• Helps researchers report safely and quickly.
• Improves trust signals for audits and security checks.

Common Mistakes

• Expired Expires date
• Broken Encryption URL
• Contact pointing to a dead inbox
• Publishing this file but never checking reports

3. pgp-key.txt (Many People Call It a GPG Key): Your Locked Mailbox

When a vulnerability is sensitive, researchers may want encrypted communication.

That is where your public PGP key comes in.

You publish your public key (usually ASCII-armored) at:

• https://yourdomain.com/.well-known/pgp-key.txt

It looks like this:

-----BEGIN PGP PUBLIC KEY BLOCK-----
...
-----END PGP PUBLIC KEY BLOCK-----

How to Generate and Publish Quickly

gpg --full-generate-key
gpg --armor --export [e-mail] > public/.well-known/pgp-key.txt

Important Notes

• This file should contain only your public key block.
• Keep your private key secret.
• Reference this file from security.txt using the Encryption: field.
• If you rotate keys, update this file and security.txt immediately.

Why It Matters

• Enables safe disclosure for high-impact findings.
• Shows your project takes security seriously.

4. robots.txt: Traffic Signs for Crawlers

robots.txt is the rule board for bots.

It does not secure private data, but it tells compliant crawlers what they should and should not crawl.

Simple example:

User-agent: *
Allow: /

Sitemap: https://yourdomain.com/sitemap.xml

You can also block internal routes if needed:

User-agent: *
Disallow: /admin/
Disallow: /private/

Why It Matters

• Helps crawl efficiency.
• Prevents unnecessary crawling of irrelevant areas.
• Connects crawlers to your sitemap.

Important Reminder

• robots.txt is not access control.
• Do not rely on it to protect private data.
• Use real auth and server-side protection for sensitive routes.

5. sitemap.xml: Your SEO Map

If robots.txt is traffic signs, sitemap.xml is the city map.

It tells search engines what pages exist and how often they may change.

Example entry:

<url>
  <loc>https://yourdomain.com/projects</loc>
  <changefreq>monthly</changefreq>
  <priority>0.8</priority>
</url>

Why It Matters

• Faster discovery of pages.
• Better indexing consistency.
• Cleaner SEO foundation.

What to Keep in Mind

• Include only canonical, public URLs.
• Keep it updated after route changes.
• Add new pages like /security/thanks when relevant.

How these files work together

• .well-known gives a standard location.
• security.txt gives a reporting workflow.
• pgp-key.txt gives confidential communication.
• robots.txt gives crawler guidance.
• sitemap.xml gives URL discovery.

Together, they quietly improve trust, security, and discoverability.

A short real-world flow

Imagine someone finds an XSS issue on your site.

Here is what happens when your setup is good:

1. They check /.well-known/security.txt.
2. They see your contact and encryption info.
3. They fetch your /.well-known/pgp-key.txt.
4. They send an encrypted report.
5. You reproduce, fix, and thank them on /security/thanks.

No confusion. No random DMs. No lost report.

That is why these files matter.

Story from my own setup

When I first added these files, I thought:

"No normal visitor will ever read this."

But scanners stopped complaining.

My deployment checks became cleaner.

And most importantly, I knew that if someone found a serious issue, they had a safe and clear way to reach me.

That feeling is worth a lot.

Quick Next.js Structure (Copy-Friendly)

public/
  .well-known/
    security.txt
    pgp-key.txt
  robots.txt
  sitemap.xml

app/
  security/
    thanks/
      page.tsx

Verify in 30 seconds

curl -fsSL https://yourdomain.com/.well-known/security.txt
curl -fsSL https://yourdomain.com/.well-known/pgp-key.txt | head -n 5
curl -fsSL https://yourdomain.com/robots.txt
curl -fsSL https://yourdomain.com/sitemap.xml | head -n 20

If all four commands return valid content, your baseline is strong.

Beginner Checklist You Can Copy

• [ ] Add /.well-known/security.txt
• [ ] Add /.well-known/pgp-key.txt
• [ ] Add Encryption: and Acknowledgments: in security.txt
• [ ] Add a /security/thanks page
• [ ] Keep robots.txt and sitemap.xml updated
• [ ] Set a reminder to update Expires before it lapses

Final Thought

Beautiful UI gets attention.

These tiny files earn trust.

If your site is public, these are not "extra" files anymore. They are part of good web citizenship.

If you are building in public, these files are one of the easiest professional upgrades you can make in a single afternoon.