DEV Community: Andrew Gnasevych

Serve Markdown to AI crawlers from static HTML pages with a few .htaccess lines

Andrew Gnasevych — Thu, 12 Nov 2026 09:49:07 +0000

AI assistants read web pages, and HTML is a noisy format for them: navigation, scripts, styles and cookie banners around a few paragraphs of content. A Markdown version of the same page is cheaper to read and easier to quote. I added one to the static pages of my site and it took a handful of Apache rules. Here is the setup, with the mistakes I made.

Disclosure: the pages I used this on belong to my app, Teleprompter Offline & Private, and I also build a WordPress plugin that does the same job for WordPress.

The idea

For every page /some-page/index.html, put a file /some-page/index.md next to it with the same content. Then serve the Markdown file when the request comes from a known AI crawler or when the client asks for it with an Accept: text/markdown header. Everyone else, browsers and Googlebot included, keeps getting the HTML.

The rules

AddType "text/markdown; charset=utf-8" .md

RewriteEngine On
RewriteCond %{REQUEST_URI} /$
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI}index.md -f
RewriteCond %{HTTP_ACCEPT} text/markdown [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-User|PerplexityBot|CCBot|Bytespider) [NC]
RewriteRule ^(.*)$ $1index.md [E=MDPAGE:1,L]

Header append Vary "Accept, User-Agent" env=MDPAGE

Three details matter:

The file-exists condition. Only pages that have an index.md are touched. Pages without one are served as usual.
The Vary header. The same URL now returns two different bodies, so caches must know it depends on Accept and User-Agent. Without Vary, a shared cache can hand the Markdown to a browser or the HTML to a bot.
The OR flag. The Accept check and the User-Agent check are alternatives, so the first has [NC,OR].

I also added a discovery link in the page head, so crawlers that read HTML can find the Markdown:

<link rel="alternate" type="text/markdown" href="/some-page/index.md" title="Markdown version">

Test it with curl

# browser: HTML
curl -s -A "Mozilla/5.0" https://example.com/some-page/ | head -3
# bot user agent: Markdown
curl -s -A "GPTBot" https://example.com/some-page/ | head -3
# content negotiation: Markdown
curl -s -H "Accept: text/markdown" https://example.com/some-page/ | head -3
# check the headers
curl -sI -A "GPTBot" https://example.com/some-page/ | grep -i "content-type\|vary"

Mistakes I made

A cache in front of the server. After deploying, my bot test still returned HTML. A reverse-proxy cache had the old page for that exact URL and served it before Apache saw the request. The same URL with ?x=1 returned Markdown. After clearing the cache everything worked. Test with a cache-busting query string, and purge after changes.
Treating user agents as identity. A User-Agent is a claim anyone can make. That is fine for serving public content, but it means this is not an access control. A WordPress plugin can go further and verify the bot's IP; plain Apache rules cannot.
Different content. The Markdown must say what the page says. Serving different content to bots than to people is cloaking, and search engines treat it as spam. I generate both files from the same source so they cannot drift.

Does it work?

I cannot tell yet how much AI assistants use it. What I can verify is the delivery: browsers get HTML, bots get Markdown, caches vary correctly. I also list the pages in the sitemap and keep the usual structured data in the HTML, because normal search still matters.

Where I use it: the pages of my teleprompter app. The landing page is https://www.a-devgroup.com/teleprompter/ and its Markdown version is at https://www.a-devgroup.com/teleprompter/index.md. The app itself, a free offline teleprompter for iPhone with voice-follow scrolling, is on the App Store: https://apps.apple.com/ua/app/teleprompter-offline-private/id6808964093?pt=128162591&ct=devto_mdbots&mt=8

If you run WordPress instead of static files, the plugin I build does the negotiation for posts and pages: https://www.a-devgroup.com/adg-ai-markdown/

Question for you: do you serve anything special to AI crawlers, or do you leave them the plain HTML?

Building a voice-follow teleprompter on iPhone without sending audio anywhere

Andrew Gnasevych — Wed, 11 Nov 2026 16:14:04 +0000

A teleprompter that scrolls as you speak sounds simple: recognize speech, find the words in the script, move the text. I built that for iOS and the recognizing part was the easy part. The hard parts were making the scroll feel calm and not trusting the recognizer too much. Here is what I learned. Disclosure: this is my app, Teleprompter Offline & Private.

1. Keep recognition on the device

I use Apple's speech recognition and require on-device mode. If the language has no on-device model, the app says so instead of silently falling back to a server. That choice makes the app work in Airplane Mode and means your voice never leaves the phone. The cost: some languages need the offline speech pack, downloaded once in iOS settings.

2. Match words, not text

The recognizer returns a stream of partial transcripts. I align them against the script with a fuzzy matcher that searches a lookahead window of 40 to 70 words past the current position. Fuzzy, because people skip words, say "gonna" for "going to", and the recognizer mishears.

3. The bug that looked like a smoothing problem

The scroll jerked. I spent days tuning easing curves. Then I found the cause was the matcher, not the animation. One stray match on a breath or a noise could jump the cursor across the whole lookahead window, 40 words at once, and the text lurched.

The fix was boring: cap how far the cursor can advance per update (12 words). If the speaker really did jump ahead, the next updates catch up. A jump that fast is almost always noise.

4. Start every animation from zero velocity

My first smoothing carried velocity from one animation segment to the next. It looked elegant and sometimes overshot the target wildly after a noisy update. Now every segment starts at zero velocity and eases toward the target with a critically damped curve. From zero velocity a critically damped ease approaches the target monotonically and never crosses it, so the text cannot overshoot.

5. Plan for the recognizer going quiet

During video recording the recognizer can lose the audio stream. If the stream stops for more than two seconds, the app falls back to a steady words-per-minute scroll, keeps recording, and tells the user. A teleprompter that stops dead on camera is worse than one that scrolls at the wrong speed.

6. Rewind is rarer than you think

Going back needs a stricter rule: the cursor must have been standing still for a couple of updates before I allow a relocation, and a forward jump must match at least four words. Otherwise repeated phrases in a script pull the cursor to the wrong place.

What I would tell myself at the start

Put the limits in the matcher, not in the animation.
Prefer a rule you can prove (monotonic approach to target) over a clever one.
Test with a script that repeats phrases. Real scripts do.

The app is free to download, works offline and needs no account: https://apps.apple.com/ua/app/teleprompter-offline-private/id6808964093?pt=128162591&ct=devto_voicefollow&mt=8

Comparison with other teleprompter apps: https://www.a-devgroup.com/teleprompter/best-teleprompter-apps/

45% of my app's first downloads came from ChatGPT, not from my website

Andrew Gnasevych — Mon, 09 Nov 2026 15:18:24 +0000

I build Teleprompter Offline & Private, an iPhone teleprompter with voice-follow scrolling that works offline, with no account and no tracking. I'm a solo developer and it earns about $100+ a month right now. Not life-changing, but real, and it is growing slowly.

What surprised me in App Store Connect: the App Referrer report shows ChatGPT as the source of 45% of the app's first downloads. Not Google. Not my landing page. ChatGPT seems to quote App Store listings, Reddit threads and comparison pages, rather than my own site.

What I changed because of it

Rewrote the App Store title, subtitle and description to match how people phrase the question ("offline teleprompter", "teleprompter with voice follow", "no account").
Localized the listing into 12 languages.
Listed the app on comparison sites (AlternativeTo, SaaSHub).
Published plain comparison pages with the facts from public App Store listings, and made sure AI crawlers can read them: each page has a Markdown version they get by default.

How to check your own app

App Store Connect, Analytics, Sources, App Referrer. Look at the share of web referrals that are AI assistants. The numbers are small, so do not over-read them, but the direction was clear for me.

What I take from it

The listing is the page that gets quoted, so it deserves more work than the landing page. A question like "which teleprompter works offline without an account" gets answered from whatever text states that plainly.

Question for you: where do your first users actually come from? Have you checked the referrer data lately?

App: https://apps.apple.com/ua/app/teleprompter-offline-private/id6808964093?pt=128162591&ct=devto_chatgpt45&mt=8

Side-by-side comparison of six iPhone teleprompter apps (facts from public App Store listings): https://www.a-devgroup.com/teleprompter/best-teleprompter-apps/