Introduction
A few months ago, actually, few weeks, the days are getting slower, I digress... A few WEEKS ago, I built Better Markdown, my answer and solution to my personal problem with existing markdown editors and viewers, which also came from a Rise Academy's frontend track week's assignment, it is an all-round and minimalistic editor and viewer for markdown files with theme customization.
The application was complete as-is, but I fell victim to the add-AI-to-anything-and-everything virus going around and decided to take things one step further and add some AI features to the application. Just kidding though, the features I added as we'll find are not born from an unneeded compulsion at all but are pretty handy and help close UX gaps to improve the application; A sleek chat interface running a Gemini model under the hood that you can converse with to ask any questions related to your current open md file, or simply summarize your entire document for a quick TLDR.
Why file-related questions and a summarizer?
To be honest, right after completing the first round of features to launch the application and long before this week's task, I already had big plans and AI features were at the top of my list on improvements to make.
A chat interface that allows you ask questions about parts of your md file and generate summaries were obvious and great ideas for an AI MVP for the app, and as more justification, they covered actual gaps I had with my experience using the application, I did build it for my use primarily , after all, so I double as the top customer with useful user feedback. Long backend documentations were mostly skimmed through and fed back to agents for consumption and usage. An instantly generated summary of these docs would eliminate any human-error omissions I could make while skimming and help me save time, and if I ever needed specific context on a part of the file, being able to ask about that would help too, instead of having to search for keywords in the code editor, which the editor library I used didn't give the best user experience for anyway.
Technical Decisions
AI service provider
For AI provider selections, the assignment pointed me at the free, no-card options— Google AI Studio, Groq, and OpenRouter — and Google won fairly quickly for one decisive reason: context window.
(Please go listen to Claude's plan if you haven't already.)
The entire feature hinges on stuffing your whole md file into the prompt as context, and markdown files can get long; Gemini's roughly 1M-token window swallows even a big document without me having to chunk anything, the free Groq model caps out far lower and would choke on a large file, and OpenRouter's free roster is a bit unpredictable where the context you get depends entirely on which model you land on — and I'd rather have one predictable, generous window than something I have to keep re-checking. @ai-sdk/google is also a first-class provider in the Vercel AI SDK, so it was the path of least resistance on top of being the best fit. Here's a look at the numbers behind my decision;
| Provider (free tier) | The model I'd reach for | Context window |
|---|---|---|
| Google AI Studio | Gemini 2.5 / 3.5 Flash | ~1M (1,048,576 tokens) |
| Groq | Llama 3.3 70B | ~128K tokens |
| OpenRouter | assorted free models | varies widely (~8K–1M) |
Sources: Gemini API models · Groq — Llama 3.3 70B · OpenRouter free models. Figures as of mid-2026; free tiers shift often.
To Stream or Not to Stream (That is the question! - Shakespeare, probably)
On streaming — yes, obviously. It's a chat, and chat should stream as I learnt in our classes earlier this week; watching a wall of text drop in all at once after a three-second stare is just a worse experience than tokens trickling in as they're generated. So the whole surface streams, useChat on the front end talking to streamText on the back, and even the one-shot Summarize button rides that same streaming path since under the hood it's really just a canned message into the same chat.
Message array structure
The file isn't part of the conversation history, it's context and I attach the current document to every single request, read fresh from the editor's store at send time, then rebuild a system prompt on the server each turn that tells the model to answer only from that document and to admit when something isn't in it. The obvious instinct is to send the file once and let it ride along in the message history, but Better Markdown is a live editor; the second you seed it once and then keep typing, the model's answers go stale and start confidently describing a document that no longer exists. Re-sending every turn keeps it synced to what's actually on your screen, and Gemini's context budget makes that cheap enough that I don't have to think about it.
How I approached the integration
Under-the-hood logic
I went backend-first, we were required to build an /api/chat route, but without a backend for my app, this would have been an issue, but Vercel functions turned out to be the tidy answer — drop a function in /api, run vercel dev locally, and you get a real streaming endpoint without turning the app into something it isn't. I built that route in isolation and tested it out with curl before writing a single line of UI, which I'd recommend to anyone; it saves you from debugging the model, the network, and React all at the same time.
User-facing UI
This part came together fast: a slide-over Sheet driven by useChat, message bubbles that render assistant replies through the app's existing markdown-preview pipeline so summaries come out looking native, and proper error handling — the AI SDK masks stream errors behind an un-user-friendly "An error occurred", so I unmasked them while building, then mapped them to friendly messages with a Retry button.
The last mile was the fun and personal favorite stuff, a funny irony with LLM's is how for all the poor and generic-looking UI they tend to generate, the major AI chat applications have excellent UI and UX, so I wanted to capture that and made sure to cover these:
- Auto-scroll that follows the response
- An input that grows as you type
- Copy-as-plain-text on the answers
- And a loading indicator I built out of the app's own logo mark animating itself, with a little typewriter of "thinking" verbs blinking away beside it - inspired straight from Claude's spinner and loading verbs. Look at the cute thing go!
What I'd do differently?
Re-sending the entire document on every turn is clean and correct, but it isn't free. On a hobby project with a free-tier key this isn't immediately an issue, but if this ever saw real traffic I'd reach for Gemini's context caching, or only re-send when the document has actually changed, rather than blindly shipping the whole file each time.
Also, there's no real backend or accounts system so this is just wide open for now. The key lives server-side so it isn't exposed, but nothing stops one enthusiastic user from burning through the quota. I actually scoped out a proper backend with accounts and per-user caps in my "big plans" but deliberately left it out to keep this within the scope and time constraints of the assignment, but that's the obvious next step before I'd let more than a handful of coworkers loose on it, haha.
Top comments (0)