Aryan Choudhary

Posted on Jan 24

The Hard Part of AI Isn’t the Model. It’s Everything Else.

#ai #tooling #promptengineering #ux

When I first wrote about building Commentto, the focus was on why I built it.
This time, the story is different.

What changed over the past few weeks wasn’t the model, the API, or some magical prompt trick. What changed was how I started thinking about the system around the AI and how small, seemingly boring decisions affected whether the output actually felt usable.

I didn’t make the AI smarter.
I made the questions better, and the context more honest.

That turned out to matter a lot more than I expected.

Prompt tightness vs expressiveness

Early versions of Commentto were technically fine.

The comments stayed on topic, followed instructions, avoided hallucinations, and respected every rule I gave them. On paper, everything looked correct. In practice, the output still didn’t feel right. It sounded like what an AI should say, not like what a person would naturally write when reacting to something they just read.

The mistake wasn’t bad prompting.
It was over-constraint.

I was treating every part of the pipeline the same, even though the goals were very different.

So I split responsibility.

Summarization stayed strict, predictable, and conservative
Comment generation was allowed to be more expressive and flexible

To explain this without jargon:
temperature is simply how predictable the model is.

A lower temperature keeps the output stable and repeatable. A higher temperature allows variation in phrasing, tone, and rhythm. Neither is “better” by default. It just depends on what you’re asking the model to do.

So the setup became intentional:

// summarization (accuracy matters more than personality)
temperature: 0.2

// comment generation (personality matters more than precision)
temperature: 0.6

Summaries should be boring and reliable.
Comments should sound like a human reacting.

I also stopped writing long lists of rules and shifted toward behavioral guidance. Instead of telling the model what not to do, I focused on describing how this voice behaves when it speaks. That single change did more for output quality than adding another paragraph of constraints ever did.

Stability didn’t come from more rules.
It came from placing the right constraints in the right place.

And yes, at some point ChatGPT helped me rewrite the prompts, Groq’s API used those prompts, and the AI wrote comments with them. It’s AI all the way down.

Context isn’t universal. It has shape.

The next issue showed up once I stopped testing Commentto on just blogs.

Long-form articles are forgiving. Feeds are not.

Trying to summarize a short post or a tweet using the same logic as a blog almost always produces awkward results. You get things like “This post discusses…” which immediately gives away that something artificial is happening.

The fix wasn’t smarter summarization.
It was understanding what kind of content the user is actually reading.

So I moved to a progressive extraction strategy:

First, try to read what’s visibly taking up space in the viewport
If that fails, fall back to semantic containers like article or main
As a last resort, read the page body

Here’s the core logic:

function extractPageText(): string {
  const viewportText = extractViewportText();

  if (viewportText.trim().length > 300) {
    return viewportText.slice(0, 15000);
  }

  const articleText = extractArticleText();

  if (articleText.trim().length > 300) {
    return articleText.slice(0, 15000);
  }

  return document.body.innerText.slice(0, 15000);
}

Viewport extraction isn’t clever.
It’s just honest.

If something is visible, text-heavy, and occupying real screen space, that’s probably what the user is reacting to. The system shouldn’t pretend otherwise.

The bigger lesson here was simple:
AI pipelines should adapt to the shape of the content instead of forcing everything through the same mental model.

When generation became editing

One feature quietly changed how I thought about Commentto as a product.

Draft enhancement.

Instead of always generating comments from scratch, users can paste something rough and let the AI improve clarity and flow without changing what they meant. The constraint here was critical.

The AI edits.
It does not invent.

Task:
- Improve clarity, flow, and tone
- Preserve the original intent
- Do NOT add new ideas
- Do NOT turn this into a summary

This shifted Commentto from being just a generator into something closer to a writing assistant. Sometimes you don’t want new thoughts. You already know what you want to say. You just want it to come out cleaner.

That distinction mattered more than any new feature.

What this project changed for me

Iterating on Commentto reinforced a few things I’ll probably carry forward into other projects:

AI quality often improves when you remove features instead of adding them
Progressive enhancement beats clever one-pass logic
Small, low-noise affordances outperform heavy configuration
Real-world content is messy, systems should degrade gracefully

Most importantly, I learned to stop fighting the tool.

Understanding comes first. Code comes second.

Closing

Commentto didn’t improve because the AI got smarter.
It improved because the system around it became more intentional and more honest.

No accounts.
No persistence beyond the browser.
No pretending the AI knows more than it does.

I’m planning to release this as a full-fledged Chrome extension next month, and I’m curious what would actually make it more useful for you.

If you were using something like this regularly, what features would you want next?

DEV Community