Most AI chat apps are built for conversations.
I wanted one built for experimentation.
Every time I wanted to compare models, my workflow looked something like this:
- Open ChatGPT.
- Open Gemini.
- Open another tab for DeepSeek.
- Maybe another for OpenRouter.
- Copy the same prompt into each.
- Wait.
- Compare responses manually.
- Forget which model used fewer tokens or responded faster.
After doing this enough times, it became obvious that the problem wasn't the models—it was the workflow.
So I built AutarkChat.
The Problem
The AI ecosystem has exploded over the past year.
We now have hundreds of models available through dozens of providers.
OpenAI.
Anthropic.
DeepSeek.
Gemini.
Mistral.
Groq.
OpenRouter.
Together AI.
Tencent.
Each has different strengths, different costs, different latency, and different output quality.
Yet most chat applications still assume you'll only talk to one model at a time.
As a developer, that's rarely what I want.
Sometimes I want to know:
- Which model follows my instructions best?
- Which one is fastest?
- Which one is cheapest?
- Which one produces cleaner code?
- Which one wastes the fewest tokens?
Answering those questions shouldn't require six browser tabs.
Building AutarkChat
AutarkChat isn't another AI wrapper.
It's a developer workspace focused on comparing, testing, and evaluating language models.
The idea is simple:
Instead of asking "What did this model say?", ask "How did these models perform against each other?"
That small shift completely changes how the interface is designed.
Side-by-Side Comparison
The core feature is the comparison workspace.
Instead of switching between conversations, you select multiple models and send a single prompt.
Each model begins streaming independently.
No model waits for another.
If one provider is slower, the others continue streaming normally.
This turned out to be far more useful than I expected.
You immediately start noticing differences:
- reasoning style
- verbosity
- formatting
- hallucinations
- instruction following
- response speed
without constantly switching tabs.
Streaming Shouldn't Be Fragile
One thing I noticed while experimenting with different providers is that streaming failures happen more often than you'd expect.
Sometimes:
- the provider times out
- your network disconnects
- a request is cancelled
- one provider fails while another succeeds
Many chat applications treat these situations as "everything failed."
I didn't want that.
Instead, AutarkChat treats every model independently.
If two models finish successfully and one crashes,
the successful responses are already saved.
Nothing is lost.
If a request fails,
your prompt is restored automatically,
so you don't have to rewrite it.
It sounds like a small detail,
but after using it for weeks,
I don't think I could go back.
Supporting Multiple Providers
Another goal was flexibility.
I didn't want the application tied to a single vendor.
AutarkChat supports configuring custom OpenAI-compatible endpoints, allowing you to connect services like:
- OpenAI
- OpenRouter
- DeepSeek
- Mistral
- Together AI
- Groq
- Tencent
- and other compatible APIs
Switching providers shouldn't require switching applications.
Measuring More Than Just Responses
When experimenting with prompts, quality is only part of the picture.
Cost matters.
Latency matters.
Token usage matters.
That's why every response includes token information directly inside the interface.
Instead of opening dashboards later, you can immediately compare:
- Prompt tokens
- Completion tokens
- Total tokens
This makes it much easier to understand the trade-offs between models.
Personalization
Most people repeatedly type the same context into new conversations.
Things like:
- who they are
- what they work on
- preferred programming languages
- writing preferences
AutarkChat lets you create reusable profiles and global instructions.
Every conversation automatically includes that context, eliminating repetitive prompt setup.
Tech Stack
AutarkChat is built using:
- Next.js 15
- TypeScript
- MongoDB
- Tailwind CSS
- Framer Motion
The goal was to keep the stack modern while keeping the interface responsive during multiple concurrent streams.
Challenges Along the Way
Building the UI wasn't actually the hardest part.
The interesting engineering problems were things like:
- synchronizing multiple streaming responses
- safely persisting completed generations
- preventing one failed provider from affecting others
- restoring interrupted requests
- managing different provider capabilities behind one consistent interface
Those details aren't immediately visible, but they're what make the application feel reliable.
What's Next
This is just the beginning.
Some ideas I'm exploring include:
- conversation branching
- better prompt history
- benchmark workflows
- reusable prompt collections
- model capability detection
- richer analytics
- additional provider integrations
The goal isn't to become another chatbot.
The goal is to build a workspace developers actually enjoy using when working with multiple LLMs.
Try It Out
If you're someone who regularly compares AI models, I'd love to hear what you think.
⭐ GitHub:
https://github.com/unreadlogs/AutarkChat
Feedback, issues, feature requests, and contributions are always welcome.
Thanks for reading!
Top comments (1)
Finally someone built this. I've been juggling tabs for months and it's such a pain. The parallel streaming alone makes this worth trying. Gonna spin it up this weekend—curious if local models work too. Nice work!