Salih Guler for AWS

Posted on Jul 22

Building an Agent Friendly Personal Website with Specs

#ai #agents #webdev #programming

If you visit salih.dev right now, you'll notice it redirects somewhere else. That's been bugging me for months. I wanted a proper site where I could list my talks, link my blog posts, and make everything discoverable for both search engines and AI agents. But between conferences and content creation, I never had a full day to sit down and build one from scratch.

So I changed the question. Instead of "when will I have time for this?" I asked: "what can I get done in one iteration?"

A few days ago, after dropping my son off at school, I walked into a nearby cafe with my laptop and started planning. One hour later, I had a working personal website with 19 imported blog posts, conference listings, agent readiness, and a responsive design.

Here's how.

📺 This post accompanies the YouTube video where I walk through the whole process live. The source code is on GitHub.

Step 1: Get a design reference

I wanted a minimalistic black and white design, text-heavy, light mode first. I opened my AI assistant and described what I needed:

A map on the left showing my current city (so people can see where I'm based)
Blog posts with header images on the right
Upcoming and past conference sections
Social links, an About page, a Blog subpage

The point is to produce a visual reference you can hand to the coding assistant later.

The first result looked close enough. I downloaded it and moved on.

Step 2: Write the specification

This is the part that made everything else productive. Before touching any code, I wrote a spec.md that defined the project's purpose, rules, goals, and non-goals.

Here's the actual file from the repository:

# salih.dev Personal Website Building

The purpose of this document is to build a text-heavy personal website
that will eventually live at the salih.dev address.

# Rules
- You MUST use Astro to build this website.
- You MUST use TypeScript as the language.
- You MUST create the content using Astro's markdown support.
- You SHOULD NOT write tests for it.
- You MUST commit your iterations and changes with logical commit messages.
- You MUST use images/reference-image-for-design.png as the design reference.
  For changes, always ask for approval.
- For each task, you MUST come up with a plan describing what you are going to
  do and ask for approval with clear acceptance criteria.

# Goals
- The website has a responsive design.
- The website is SEO optimized.
- The website is AEO optimized.
- The website considers the following for agent readiness:
  - Discoverability — robots.txt, Sitemap, Link response headers, DNS-AID
  - Content Accessibility — Markdown content negotiation
  - Bot Access Control — AI bot rules in robots.txt, Content Signals, Web Bot Auth
  - Protocol Discovery — MCP Server Card, Agent Skills, WebMCP, API Catalog,
    OAuth discovery
  - Commerce — x402, MPP, UCP, ACP
- The website's markdown support handles:
  - All text types
  - Links
  - Code (with syntax highlighting)
  - Image embedding
  - Video embedding from YouTube
  - And anything else that blog posts typically require
- The website is fully functional locally.

# Non-Goals
- Deployment to a hosting service.

A few things worth noting about this spec:

Strong language matters. When you talk to an AI coding assistant, you need to be specific. "You MUST use Astro" is different from "maybe we could try Astro?" Soft language gives the AI room to substitute its own preferences. Capital letters, bold text, and strong framing ("MUST", "SHOULD NOT") keep the assistant on track.

Opinionated tech choices up front. Astro with TypeScript. No debate, no "evaluate alternatives" phase. I chose Astro because it has excellent markdown support for text-heavy sites and ships zero JavaScript by default. That decision was made before writing the spec.

No tests. One-hour budget. Tests come later when the site is live and I'm iterating on it.

I stay in the driver's seat. The spec requires the assistant to plan before building, with clear acceptance criteria, and ask for my approval. Even in an hour of AI-assisted coding, I'm responsible for what gets built. We're software engineers. Whatever we build is our responsibility.

Agent Readiness

Agent readiness is a first-class goal. Not an afterthought. The spec calls out specific categories — discoverability, content accessibility, protocol discovery — that resulted in endpoints most personal websites don't have.

The implementation breaks into five categories. Here's what each one produced:

1. Discoverability

The basics: a robots.txt, a sitemap.xml, an RSS feed, and a llms.txt file.

The llms.txt file is the interesting one. It's a machine-readable summary of the site: who I am, what pages exist, what each blog post is about (with one-line descriptions), and what content-use policies apply. It's generated dynamically from the site's actual content, so adding a blog post automatically adds it to llms.txt. No manual syncing.

# Salih Güler

> Salih Güler is a Senior Developer Advocate at AWS focused on frontend
> and mobile app development, developer experience, and serverless architecture.

Use canonical HTML URLs for citations. Request any content page with
`Accept: text/markdown`, or follow its advertised `.md` alternate.

## Core pages
- [Home](https://salih.dev/): Profile, current location, conferences, and latest posts
- [About](https://salih.dev/about/): Biography and areas of work
- [Blog](https://salih.dev/blog/): Complete post index
- [Full corpus](https://salih.dev/llms-full.txt): Combined public Markdown content

## Content use
- Search indexing: allowed
- AI-assisted answers and user-directed retrieval: allowed
- Model training: declined

2. Content accessibility (markdown negotiation)

Every HTML page on the site has a markdown alternate. The middleware checks the Accept header on incoming requests. If a client sends Accept: text/markdown and prefers it over text/html, the server responds with the markdown version directly. No scraping, no HTML parsing.

Each HTML response also includes Link headers advertising the markdown alternate, the canonical HTML URL, and the API catalog:

Link: <https://salih.dev/about/>; rel="canonical"; type="text/html"
Link: <https://salih.dev/about.md>; rel="alternate"; type="text/markdown"
Link: <https://salih.dev/api/catalog.json>; rel="service-desc"; type="application/json"

An agent that sees those headers knows exactly where to get structured content without guessing.

3. Bot access control

The robots.txt distinguishes between search/retrieval bots and training crawlers:

# User-directed retrieval bots: allowed
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: PerplexityBot
User-agent: Claude-User
Allow: /

# Training crawlers: declined
User-agent: GPTBot
User-agent: CCBot
User-agent: Google-Extended
User-agent: anthropic-ai
User-agent: ClaudeBot
Disallow: /

Every response also includes a Content-Signal header: search=yes, ai-input=yes, ai-train=no. This tells bots the policy per-page, not just at the robots.txt level.

4. Protocol discovery

The site serves several .well-known endpoints that agents can check:

/.well-known/agent-readiness.json — A manifest listing the site's policy (search: yes, AI input: yes, training: no), available representations (HTML, markdown, llms.txt, llms-full.txt), and a full capability inventory with status for each.
/.well-known/skills/index.json — Declares a read-only skill that tells agents how to access markdown content on the site.
/.well-known/webmcp.json — Responds honestly that no browser tools or transactional actions are available (important: agents check this and knowing the answer is "nothing" prevents unnecessary probing).
/api/catalog.json — A catalog of all available documents and protocol endpoints.
/api/openapi.json — An OpenAPI spec describing the site's API surface.

For capabilities that aren't implemented (OAuth, DNS-AID, commerce protocols), the site still serves the endpoints with a structured "unavailable" response and a reason. An agent checking /.well-known/mcp/server-card.json gets back a clear "no MCP transport is operated by this site" instead of a 404 that leaves it guessing.

5. Commerce (intentionally empty)

The spec listed commerce protocols (x402, MPP, UCP, ACP). For a personal blog, none of them apply. But the capability registry explicitly marks them as unavailable with a note. This is the right approach: declare what you don't support so agents stop looking for it.

Why this matters

Most personal websites serve HTML and hope for the best. If an AI agent wants to cite your blog post, it has to scrape the page, parse the DOM, guess at the main content vs. navigation, and hope the result is coherent.

With agent readiness:

An agent can request Accept: text/markdown and get clean content directly
It can read llms.txt to understand the full site in seconds
It can check agent-readiness.json to understand what protocols the site supports
It respects your content-use preferences (search yes, training no) because you stated them in a machine-readable way

All of this came from putting "agent readiness" in the spec with the specific categories spelled out. The AI assistant knew what endpoints to build because the spec told it exactly what was needed.

Before feeding this to the coding assistant, I ran it through an AI tool to fix grammar issues. My English isn't perfect (not a native speaker), and that's fine. The spec doesn't need to be literary. It needs to be clear.

Step 3: Plan and implement

With the design image and spec.md ready, I opened VS Code with GPT 5.6 Sol with Amazon Bedrock as the AI coding assistant.

The assistant entered planning mode first. It read the spec, asked clarifying questions: Do you want editable placeholders? What aesthetic direction? What information should be pre-filled? As I answered, the plan took shape. If I didn't know an answer to something, I could click the info button to understand what the option meant.

Once planning finished, I got a markdown file with implementation details, a summary, and technical decisions. It looked like the spec but with more depth — the assistant had pulled in information about agent-readiness standards, Astro best practices, and the specific endpoints needed.

I clicked "Implement the plan" and the assistant started building.

This doesn't mean I walked away. I watched the output, refreshed the local dev server, and course-corrected when needed. At one point, the assistant was building things in a way that made sense for it but not for me. Taking a close look at each iteration is part of the process.

After the implementation pass, I refreshed the page and saw: the correct city on the map, my social icons (including Bluesky), the blog section ready for content, the About page with my bio. No email shown because I chose not to share it. Everything pulling from the configuration.

Step 4: Import existing blog posts

My articles were scattered across dev.to and Medium. Copying them manually was out of the question with 19 posts. I told the assistant: "bring in my blog posts from dev.to."

The assistant wrote an import script (scripts/import-dev.mjs) that:

Hits the dev.to API with my username
Fetches each article's full content
Transforms the markdown (fixes broken links, converts {% youtube %} liquid tags to a custom remark directive format)
Writes each post as an Astro content collection entry with proper frontmatter (title, description, date, category, tags, hero image, AI summary)

Each article requires reviewed metadata (a category and an AI summary) that I provide in a map inside the script. If an article doesn't have its metadata, the script throws. This forces me to review each import rather than blindly dumping content.

npm run import:dev

For Medium, it wrote a second script (scripts/import-medium.mjs) that fetches my RSS feed, parses the HTML body, strips Medium's tracking pixels and footer boilerplate, converts HTML to markdown with Turndown, extracts hero images, and handles duplicate detection. If an article already exists (because I published it on both platforms), it merges the source link rather than creating a duplicate.

npm run import:medium

19 posts imported in seconds.

After the import, I checked the blog listing — header images rendered, detail pages showed code highlighting with syntax colors, and YouTube videos embedded properly using the custom remark plugin. That YouTube support I called out in the spec? It produced a directive syntax:

::youtube{id="dQw4w9WgXcQ" title="My video"}

This renders as a responsive iframe embed. It worked because the requirement was explicit in the spec.

Step 5: Polish

With everything imported, I asked "what's left?" Two things:

A mobile layout fix — some elements weren't sizing correctly on smaller screens
A CSS patch for the map component so it doesn't overflow when the viewport is too large or too small

Both were quick fixes. The assistant handled them in one pass.

What we built

Everything you see was done in under an hour with one file (the spec), one image (the design reference), and one AI coding assistant (GPT 5.6 Sol in VS Code). We took an idea and turned it into a product.

The site includes:

Blog posts rendering from markdown (syntax highlighting, YouTube embeds, GFM tables, images)
Conference listings (upcoming and past) from a config file
Agent readiness endpoints: llms.txt, llms-full.txt, .well-known/agent-readiness.json, WebMCP manifest, MCP server card, agent skills, OpenAPI catalog, content negotiation middleware
RSS feed, sitemap, SEO meta tags, Open Graph
Categories and tags with their own listing pages
Responsive design, mobile-friendly
Import scripts for dev.to and Medium with duplicate detection

Open source and what's next

The full source is on GitHub: github.com/salihgueler/salih-dev

Clone it, run npm install && npm run dev, and you'll have a local version in seconds. Swap out my content in src/config/site.ts and src/content/blog/ for yours and you're done. You can use any hosting service to put it live.

I'll be publishing a follow-up on deploying with AWS. Stay tuned for that.

If you have questions, drop them in the video comments or find me on social media with @salihgueler handle.