How I Cut My Voice Agent’s Groq API Usage by ~70% Without Changing the Model

Asad Ibrahim — Mon, 10 Aug 2026 09:41:11 +0000

A few weeks ago, I added a small AI voice assistant called Jarvis to my portfolio website.

The setup was intentionally simple: Groq handled text generation, while the browser’s Web Speech API handled speech-to-text and text-to-speech. Visitors could ask Jarvis about my work, projects, experience, or services, and it could also help them book a call.

It worked.

There was just one problem.

After about five minutes of fairly light testing, I had already burned through the Groq free-tier limit.

My first thought was probably the obvious one:

Do I need another provider or a paid API plan?

Before changing anything, though, I decided to figure out where the tokens were actually going.

That turned out to be the right decision.

3,113 Prompt Tokens Just to Say “Hello”

The first thing I wanted was an actual measurement.

Instead of relying only on a token-counting estimate, I made a real Groq request with a minimal output limit and inspected the token usage returned by the API.

Then I sent:

hello

The result surprised me:

3,113 prompt tokens.

Not for a complicated question.

Not for a request involving several projects.

Just “hello.”

At that point, the problem became pretty obvious.

My model wasn't necessarily expensive.

My request architecture was.

I found two major problems.

I was sending my entire portfolio every time

My system prompt contained almost everything about me:

services
project descriptions
testimonials
FAQs
statistics
tech stack
experience
behavioral instructions

And all of it was being sent on every request, regardless of what the visitor asked.

Someone saying “Hi” was effectively sending thousands of tokens of portfolio context to the model.

Someone asking about pricing received the same context.

Someone asking about one project also received information about every other project.

It worked, but it was incredibly wasteful.

I had duplicated part of my knowledge base

While auditing the knowledge object, I found something even simpler.

My tech stack existed twice under two different fields.

Two parts of the system had been built separately, and both included the same information without me noticing.

So I wasn't just sending too much context.

I was literally paying in tokens to send some of it twice.

The Bigger Realization: Not Every Message Needs AI

This ended up being the most important change.

I had originally treated Jarvis like this:

User message → Groq → response

Every message went through the model.

But why should an LLM generate an answer to:

Hello

Or:

Who are you?

Or:

Do you sign NDAs?

Those answers are already known.

So I added a lightweight local intent layer before Groq.

Now the flow looks more like:

User message → local intent check → Groq only when necessary

Simple regex and keyword matching handles predictable requests locally.

For example:

Greetings → predefined greeting
“Who are you?” → fixed identity response
Pricing questions → predefined pricing-policy response
Known FAQs → existing FAQ answer
Requests for private information → fixed refusal
Obvious prompt-injection attempts → fixed security response

If the intent matches one of these cases, Groq is never called.

API usage for that interaction: zero.

This also made the assistant feel faster because there is no reason to wait for model inference when the application already knows the answer.

Then I Shrunk the System Prompt

The next target was the prompt itself.

Instead of shipping my entire portfolio with every request, I separated the prompt into two layers.

Layer 1: A small base prompt

The base prompt contains only information Jarvis always needs:

identity
behavior
privacy rules
pricing policy
important boundaries
response style

After cleaning it up, the base context dropped to roughly 652 tokens, compared with the 3,113-token request I measured earlier.

Layer 2: Retrieve Only Relevant Portfolio Knowledge

The rest of my portfolio became selectively retrieved context.

I divided the knowledge into sections such as:

services
projects
testimonials
faqs
contact
stats

Before calling Groq, the application checks the user's message and determines which sections are actually relevant.

If someone asks:

“What AI automation work have you done?”

Jarvis might receive relevant services and AI project information.

It doesn't need every testimonial, contact detail, FAQ, and unrelated project.

Usually only 2–4 relevant sections are added to the prompt.

It isn't a complicated vector database or a full RAG pipeline.

For a portfolio this size, simple retrieval works perfectly well.

And more importantly, it's cheap.

I Also Found a Retry Problem

There was another source of unnecessary requests that wasn't immediately obvious: retries.

SDKs often retry certain failed requests automatically, including rate-limit errors.

That's normally helpful.

But when you're already hitting a quota limit, automatic retries can make the situation worse.

A request fails with a 429.

The client retries.

It fails again.

Another retry happens.

From the application's point of view, the visitor sent one message.

From the API's point of view, multiple attempts may have occurred.

So I tightened the retry behavior and added request-level protection around the voice agent.

The final flow includes:

request deduplication
cooldown protection
aborting superseded requests
controlled retries
request tracking

The goal was simple:

One visitor message should result in at most one intentional Groq generation request.

The Result

After the changes, the difference was significant.

Before

Measured “hello” request:

3,113 prompt tokens

After

Compact base context:

~652 tokens

For requests that still need Groq, selective knowledge retrieval reduced prompt-token usage by roughly:

70–72% in my testing.

And several common interactions now use zero Groq requests:

greetings
identity questions
pricing-policy questions
known FAQs
security refusals
obvious prompt-injection attempts
duplicate submissions

The interesting part is that I didn't switch models.

I didn't move to another AI provider.

And I didn't solve the problem by simply paying for a larger quota.

I changed how my application used the model.

What I Learned

When an AI application starts burning through API quota, it's easy to assume the model or provider is the problem.

Sometimes it is.

But before switching providers, I think it's worth looking at the request path itself.

Ask:

Does this request actually need an LLM?

Am I sending context the model doesn't need?

Am I sending the same information repeatedly?

Can part of this response be deterministic?

Can I retrieve only the knowledge relevant to this question?

Can one user action accidentally trigger multiple API requests?

In my case, those questions mattered far more than changing the model.

The biggest optimization wasn't finding a cheaper LLM.

It was calling the LLM less often and sending it less unnecessary information when I did.

I'm Asad Ibrahim, a full-stack developer and AI integration engineer. I build AI-powered web applications, automation systems, voice assistants, and production integrations for businesses.

I also document experiments like this from projects I'm actually building.

You can see more of my work, AI projects, and engineering case studies at asadibrahim.com.

DEV Community: Asad Ibrahim

How I Cut My Voice Agent’s Groq API Usage by ~70% Without Changing the Model