DEV Community: Ecohash

Build a voice agent with Whisper, Kokoro, and an OpenAI-compatible API

Ecohash — Sat, 10 Oct 2026 07:03:48 +0000

A voice agent turns speech into speech: it transcribes what the user says, sends the text to a language model, and speaks the reply back. On EcoHash you build all three stages through one OpenAI-compatible API and one key, so there are no three vendors and no three billing accounts to stitch together. Whisper (whisper-large-v3-turbo) does speech to text, a chat model such as llama-3.1-8b-instruct writes the reply, and Kokoro (kokoro-82m) turns it into audio. None of it needs a GPU of your own, since the models are served for you. This post walks through the pipeline, the three API calls, and when an RTX Pro 6000 workspace is worth it for heavier voice work.

Three API calls, one key. Whisper hears, a chat model thinks, Kokoro speaks. About $0.006 per minute of generated speech.

Who this is for

Developers building a voice assistant, an IVR replacement, or a spoken interface for an app.
Teams that want speech to text, an LLM, and text to speech from one provider rather than three.
Anyone prototyping a voice feature before deciding on dedicated capacity.

What you can build

A back-and-forth voice assistant that listens, thinks, and speaks.
A transcription-plus-reply feature inside an existing app.
A spoken front end for a chatbot you already run on EcoHash.

How a voice agent works

The loop has three stages, and each maps to one EcoHash endpoint:

Speech to text. The user's audio goes to Whisper, which returns text. This is the transcription endpoint.
LLM. The text goes to a chat model, which returns a reply. This is the chat completions endpoint.
Text to speech. The reply goes to Kokoro, which returns audio you play back. This is the speech endpoint.

Then you repeat for the next turn. The whole loop uses one key and one base URL, so the three stages share auth, billing, and SDK.

The models

Speech to text · Model: Whisper Large V3 Turbo · Model ID: whisper-large-v3-turbo
Speech to text (alt) · Model: Qwen3 ASR · Model ID: qwen3-asr-1-7b
LLM · Model: Llama 3.1 8B Instruct · Model ID: llama-3.1-8b-instruct
Text to speech · Model: Kokoro 82M · Model ID: kokoro-82m
Text to speech (alt) · Model: Qwen3 TTS · Model ID: qwen3-tts

Use a small or mid-size chat model for the LLM stage so replies come back fast. A coding-specialized model is the wrong pick for general conversation.

Here is what the speech models measure on a single RTX Pro 6000, end-to-end, in July 2026. Speech to text:

qwen3-asr-1-7b · WER %: 3.28 · Peak RTFx: 360 · Price: input $0.05/1M
whisper-large-v3 · WER %: 3.64 · Peak RTFx: 45 · Price: $0.006/min
whisper-large-v3-turbo · WER %: 4.37 · Peak RTFx: 59 · Price: $0.006/min

Text to speech:

kokoro-82m · TTFA: 120 ms · Price: $1/1M
qwen3-tts · TTFA: 2621 ms · Price: $2/1M

On the HF Open ASR Leaderboard, the models EcoHash serves land among the accurate ones (purple = served on EcoHash):

Full data and method: ecohash-benchmarks.

The three API calls

The API is OpenAI-compatible, so the OpenAI SDK works once you set the base URL and key. One client covers all three stages.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ecohash.com/v1",
    api_key="eco_...",  # create a key at console.ecohash.com
)

# 1. Speech to text
with open("input.wav", "rb") as f:
    transcript = client.audio.transcriptions.create(
        model="whisper-large-v3-turbo",
        file=f,
    ).text

# 2. LLM reply
reply = client.chat.completions.create(
    model="llama-3.1-8b-instruct",
    messages=[
        {"role": "system", "content": "You are a concise voice assistant. Keep replies short."},
        {"role": "user", "content": transcript},
    ],
).choices[0].message.content

# 3. Text to speech
with client.audio.speech.with_streaming_response.create(
    model="kokoro-82m",
    voice="af_heart",
    input=reply,
    response_format="wav",
) as speech:
    speech.stream_to_file("reply.wav")

print("You said:", transcript)
print("Assistant:", reply)

That is a full single-turn voice agent. For multi-turn, keep the message history and run the loop again for each new audio input.

Running it without a GPU

You do not need one. All three models are served through the API, so you can build and run a voice agent with no hardware to provision, and Kokoro in particular is small enough that it never needs an RTX Pro 6000.

A workspace helps later, not at the start: when you want to co-locate a heavier speech-to-text, LLM, and text-to-speech pipeline on one card, push higher throughput, or run your own serving stack. That is a scaling choice. See /gpu-compute.

Limitations

This post does not promise real-time or a specific latency. End-to-end voice latency depends on network, the models you pick, whether you stream, and your own pipeline. Measure it on your setup and cite the method and date for any number you publish.
Whisper and Kokoro do not require an RTX Pro 6000. They run through the API and move onto GPU capacity only when throughput demands it.
The example is request-response. A production voice agent adds streaming audio, voice activity detection, turn-taking, and barge-in, which are your application's job.
Model IDs and voices change. Confirm them on /models, /models/whisper-large-v3-turbo, and /models/kokoro-82m before you build.

FAQ

What is a voice agent? A system that listens, thinks, and speaks: speech to text, then an LLM, then text to speech. The user talks, and the agent replies in audio.

Which models do I need? Three: a speech-to-text model like whisper-large-v3-turbo, a chat model like llama-3.1-8b-instruct, and a text-to-speech model like kokoro-82m.

Can I do all of it with one API key? Yes. On EcoHash the transcription, chat, and speech endpoints share one base URL and one key, so there is no three-vendor integration.

Do I need an RTX Pro 6000 to run Kokoro or Whisper? No. Both run through the API. A workspace is optional, for co-locating a heavier pipeline or higher throughput.

Is it fast enough for real-time conversation? That depends on your network, model choices, streaming, and pipeline. This post does not publish latency numbers; measure your own setup before relying on a target.

Which voices does Kokoro support? Several; the example uses af_heart. See /models/kokoro-82m.

How much does a voice agent cost? The LLM stage is billed per token, and the audio stages by what they process. See /pricing for current rates.

Which chat model should I use for the LLM stage? A small or mid-size general model keeps replies quick. See Best models to run on RTX Pro 6000 96GB.

Next steps

See voice and speech on EcoHash: /use-cases/voice-speech
Speech-to-text model: /models/whisper-large-v3-turbo
Text-to-speech model: /models/kokoro-82m
Browse all models: /models

Qwen3 Coder 30B on RTX Pro 6000: API, dedicated endpoint, or GPU workspace?

Ecohash — Tue, 29 Sep 2026 03:48:15 +0000

Three ways to run Qwen3 Coder 30B (qwen3-coder-30b-a3b-instruct), and the right one comes down to your traffic and how much control you want. The shared OpenAI-compatible API gets you a first call in minutes and bills per token, which fits prototypes and bursty traffic. A dedicated endpoint reserves capacity for steadier throughput and predictable latency in production, still with no GPU to manage. An hourly RTX Pro 6000 workspace gives you the whole card for fine-tuning, batch jobs, or your own serving stack. The model and the API contract are the same across all three, so moving up later does not mean a rewrite.

Same model, three ways to run it. Start on the per-token API, reserve a dedicated endpoint for production, rent the card itself for fine-tuning.

Facts


Model	Qwen3 Coder 30B (`qwen3-coder-30b-a3b-instruct`)
Type	Mixture-of-Experts, about 30B total parameters
Context served	32,768 tokens
API price	input $0.10 / output $0.30 per 1M tokens (pricing)
GPU workspace	RTX Pro 6000 at $1.89/GPU-hour
API base URL	`https://api.ecohash.com/v1`
Hugging Face	`Qwen/Qwen3-Coder-30B-A3B-Instruct`

On a single RTX Pro 6000, end-to-end, we measure it at 30 ms to first token and 9 ms per output token, about 110 tokens per second on a single stream (July 2026). A responsive coding assistant lives or dies on time to first token, and in a same-model provider comparison (llama-3.1-8b-instruct across providers, EcoHash in purple) it comes in lowest in the field:

Full data, method, and charts: ecohash-benchmarks.

Who this is for

Developers adding Qwen3 Coder to a coding assistant, an agent, or a code-review tool.
Teams deciding whether to call a shared API, reserve capacity, or run their own GPU.
Anyone moving a Qwen3 Coder prototype toward steady production traffic.

What you can decide

Which of the three paths fits your current stage.
What changes and what stays the same when you move between them.
When it is worth reserving a dedicated endpoint or renting a workspace.

The shared API

The shared API serves Qwen3 Coder through an OpenAI-compatible endpoint at https://api.ecohash.com/v1. You send a request, get a completion, and pay per token. There is no GPU to provision and no capacity to plan, and because traffic runs on shared infrastructure, latency moves with overall load.

This is where most projects should start, and it suits bursty traffic well since you pay for what you use rather than for reserved capacity. See inference.

The dedicated endpoint

A dedicated endpoint reserves capacity for Qwen3 Coder so your traffic is not competing with anyone else's. You still call the same API and still manage no GPU. What you gain is steadier throughput and more predictable latency, which starts to matter once a coding feature is in production and people expect consistent response times.

Reach for it when your traffic is steady and high enough that predictability is worth reserving capacity for. See dedicated inference.

The GPU workspace

A GPU workspace is an RTX Pro 6000 you rent by the hour at $1.89/GPU-hour. You get the whole card and control of the environment, so you can run a custom serving stack such as vLLM or SGLang, fine-tune with LoRA or QLoRA, or run batch jobs. Qwen3 Coder 30B fits on one 96 GB card with room for context and batching.

Reach for it when the API and dedicated endpoints do not give you enough control, or when the work is not request-response serving at all, like fine-tuning or offline batch generation. See GPU compute and fine-tuning.

How the three compare

	Shared API	Dedicated endpoint	GPU workspace
What it is	Call the model over the API	Reserved capacity for the model	Rent the RTX Pro 6000 by the hour
You manage	Nothing	Nothing	The environment and serving stack
How you pay	Per token	Reserved capacity	Per GPU-hour ($1.89)
Latency	Varies with shared load	Steadier, more predictable	The full card is yours
Scaling	Elastic	Reserved throughput	One card, or more on request
Best for	Prototypes, spiky traffic	Steady production traffic	Fine-tuning, batch, custom serving
Where	inference	dedicated inference	GPU compute

The model ID and the API contract hold across the first two, and the same model runs in the workspace, so moving between them does not force a rewrite.

Calling Qwen3 Coder

The API is OpenAI-compatible, so existing OpenAI SDK code works once you change the base URL and key. The same call works on the shared API or a dedicated endpoint.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ecohash.com/v1",
    api_key="eco_...",  # create a key at console.ecohash.com
)

resp = client.chat.completions.create(
    model="qwen3-coder-30b-a3b-instruct",
    messages=[
        {"role": "user", "content": "Write a Python function that parses an ISO 8601 timestamp into a datetime."}
    ],
)
print(resp.choices[0].message.content)

curl https://api.ecohash.com/v1/chat/completions \
  -H "Authorization: Bearer $ECOHASH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-coder-30b-a3b-instruct",
    "messages": [{"role": "user", "content": "Write a Python function that parses an ISO 8601 timestamp into a datetime."}]
  }'

When to move up

Prototype and early traffic: shared API. You are still changing prompts and shipping features, and per-token billing keeps cost tied to use.
Consistent production traffic: dedicated endpoint. Once users notice latency and volume is steady, reserved capacity buys predictability without infrastructure work.
Full control or non-serving work: GPU workspace. Fine-tuning, batch generation, and custom serving need the card itself.

You do not choose once. Plenty of teams run the API in development, a dedicated endpoint in production, and spin up a workspace only when they fine-tune.

Limitations

The context served here is 32,768 tokens. Do not assume a larger window unless it is confirmed for this deployment.
Performance depends on quantization, batch size, and context length. Reproduce any figure with its method and date.
Dedicated endpoint and workspace pricing depend on configuration. Check pricing and dedicated inference.
Model IDs change as the catalog changes. Confirm qwen3-coder-30b-a3b-instruct on the model catalog before you build.

FAQ

Can I run Qwen3 Coder 30B on an RTX Pro 6000? Yes. It is a Mixture-of-Experts model with about 30B total parameters and fits on one 96 GB RTX Pro 6000 with room for context and batching.

What is the difference between the API and a dedicated endpoint? Both use the same OpenAI-compatible API and neither asks you to manage a GPU. The shared API bills per token on shared infrastructure; a dedicated endpoint reserves capacity for steadier throughput and more predictable latency.

When should I rent a GPU workspace instead? When you need the whole card: fine-tuning with LoRA or QLoRA, batch generation, or a custom serving stack such as vLLM or SGLang.

What context length does Qwen3 Coder have here? 32,768 tokens as served.

How much does Qwen3 Coder cost? Through the API, $0.10 per 1M input tokens and $0.30 per 1M output tokens. A GPU workspace is $1.89/GPU-hour. See pricing for current rates.

Do I have to change my code to switch paths? No. The model ID and API contract stay the same across the shared API and a dedicated endpoint, and the same model runs in a workspace.

Which models pair well with Qwen3 Coder for a coding product? See Best models to run on RTX Pro 6000 96GB for embeddings, rerankers, and chat models that share the same key.

Next steps

See the model page: Qwen3 Coder 30B
Start on the shared API: ecohash.com/inference
Reserve a dedicated endpoint: ecohash.com/dedicated-inference
Rent an RTX Pro 6000: ecohash.com/gpu-compute

What makes RTX Pro 6000 96GB a good GPU for AI inference?

Ecohash — Wed, 23 Sep 2026 01:47:33 +0000

The RTX Pro 6000 Blackwell Server Edition gives you 96 GB of GDDR7 on one card, and for inference that capacity is the whole point. 96 GB holds a 20B to 35B open model with room left for KV cache and larger batches, so you serve it on a single GPU instead of splitting it across several. In the size range most teams actually deploy, that keeps both latency and operations simpler. On EcoHash the same card backs three ways to use it, a shared OpenAI-compatible API, a dedicated endpoint, and an hourly GPU workspace, and you move between them without changing the model or your code. It is a strong fit for mid-sized open models, and it is not the card for the very largest ones.

96 GB on one card. A 20B to 35B open model, its KV cache, and real batch sizes fit on a single GPU at $1.89 per hour.

Who this is for

Developers checking whether a single 96 GB card can host the open model they want to serve.
Teams weighing a shared model API against dedicated or self-managed GPU capacity.
Anyone sizing an RTX Pro 6000 for 20B to 35B models before spending on it.

What you can decide

Whether your model fits on one card, or needs quantization or more GPUs.
Which path fits your stage: shared API, dedicated endpoint, or workspace.
When a model is large enough that a different GPU is the better call.

Facts


Hardware	NVIDIA RTX Pro 6000 Blackwell Server Edition
VRAM	96 GB GDDR7
Architecture	Blackwell generation
Interconnect	PCIe (single-GPU and multi-instance inference)
Best-fit models	20B to 35B open models, plus selected quantized larger models
EcoHash GPU price	$1.89 per GPU-hour (pricing)
Access on EcoHash	Shared API, dedicated endpoint, GPU workspace
API base URL	`https://api.ecohash.com/v1`

Why 96 GB matters

VRAM sets the ceiling on what you can serve. The weights have to fit, and so does the KV cache, which grows with batch size and context length. Once the weights eat most of the card, you are left with small batches, short context, or a model split across GPUs.

96 GB changes that math for mid-sized models. A 30B-class model in a memory-efficient format still leaves room for concurrent requests and long prompts on one card, so you skip tensor-parallel serving and the latency and operational overhead that come with it. A 7B or 13B model has even more headroom, which you spend on bigger batches and longer context rather than leaving idle.

Blackwell, on a single card

The RTX Pro 6000 is a Blackwell-generation card on PCIe. EcoHash runs it as single-GPU and multi-instance capacity rather than an NVLink cluster, which suits inference: one card serves one model, or hosts a few smaller ones side by side.

Blackwell also handles the low-precision formats modern inference leans on, such as FP8, which lets a larger model fit in the same memory and can lift throughput. How much depends on the model and the serving stack, so treat the 96 GB as a fixed fact and treat quantization gains as something to measure on your own workload.

How it compares to other inference GPUs

These are published hardware specs, not our own benchmarks, and they are here for context.

GPU	VRAM	Interconnect	Typical best fit
RTX Pro 6000 (Blackwell)	96 GB GDDR7	PCIe	Single-GPU serving of 20B to 35B models; headroom on smaller ones
RTX 6000 Ada (previous gen)	48 GB GDDR6	PCIe	Smaller models, tighter memory budgets
L40S	48 GB GDDR6	PCIe	Cost-focused serving of smaller models
H100	80 GB HBM3	NVLink	High-throughput and multi-GPU serving of large models
H200	141 GB HBM3e	NVLink	Very large models and long-context workloads

The H100 and H200 use HBM with higher bandwidth and NVLink, which favors large models and multi-GPU tensor parallelism. The RTX Pro 6000 trades some of that bandwidth for a large, cost-effective single-card budget, and it is at its best when the model fits on one card.

What fits on 96 GB

Model size	Fit on one 96 GB card	Notes
7B to 13B	Comfortable	Room for large batches and long context
20B to 35B	The sweet spot	For example `qwen3-coder-30b-a3b-instruct`, `gpt-oss-20b`
Around 70B	Case by case	Usually needs quantization; validate first
235B and above	Not the target	Use an H200 or a multi-GPU or partner path

Model IDs are current EcoHash catalog IDs. See the model catalog for the full list.

Measured numbers

Numbers below are measured on a single RTX Pro 6000, end-to-end through the API, in July 2026. Full data, method, and charts are in ecohash-benchmarks.

Text generation, single stream:

Model	TTFT (p95)	Per-token	Single-stream tok/s
`llama-3.1-8b-instruct`	205 ms	7 ms	~143
`gpt-oss-20b`	170 ms	8 ms	~125
`qwen3-coder-30b-a3b-instruct`	30 ms	9 ms	~111

Under concurrency the card sustains about 11,500 output tok/s on llama-3.1-8b-instruct. With 8k-token prompts, prefill reaches roughly 170k to 220k tok/s, which is why long-context and RAG are the most cost-effective way to use it. Against other providers of the same model, its time to first token is the lowest in the field:

Image generation at 1024×1024:

Model	Steps	Latency	Images/min
`flux2-klein`	4	1.2 s	52
`z-image-turbo`	8	3.2 s	20
`qwen-image`	50	13.4 s	4.6

Three ways to use it

Shared API. Call open models through the OpenAI-compatible API at https://api.ecohash.com/v1, with no GPU to manage. Good for building and for traffic that comes in bursts. See inference.
Dedicated endpoint. Reserve capacity for one model when production traffic needs steadier throughput and predictable latency. See dedicated inference.
GPU workspace. Rent the card by the hour for experiments, fine-tuning, and batch jobs. See GPU compute.

You can start on the API and move to reserved capacity later without touching the model or the code.

Calling a model

The API is OpenAI-compatible, so existing OpenAI SDK code works once you point the base URL and key at EcoHash.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ecohash.com/v1",
    api_key="eco_...",  # create a key at console.ecohash.com
)

resp = client.chat.completions.create(
    model="qwen3-coder-30b-a3b-instruct",
    messages=[
        {"role": "user", "content": "Write a Python function that reverses a linked list."}
    ],
)
print(resp.choices[0].message.content)

curl https://api.ecohash.com/v1/chat/completions \
  -H "Authorization: Bearer $ECOHASH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-coder-30b-a3b-instruct",
    "messages": [{"role": "user", "content": "Write a Python function that reverses a linked list."}]
  }'

Validating the fit

Before you commit a model to this card:

Check that the model plus KV cache fits in 96 GB at your context length and batch size.
Pick a quantization format and compare its output against full precision on your own prompts.
Choose a serving stack such as vLLM or SGLang and pin the version.
Measure latency and throughput on your own traffic, and record the model revision, quantization, context length, framework, and driver so the numbers reproduce.

Where it falls short

96 GB is a single-card budget. Models above roughly 70B usually need quantization or more than one GPU.
Models at 235B and above are not the target. Use an H200 or a multi-GPU or partner path.
We run the card as PCIe capacity, not an NVLink cluster, so workloads that lean on wide tensor parallelism see less benefit.
Real numbers depend on the model, quantization, batch size, and context length. Treat any performance figure as something to reproduce with its method and date.
Small and voice models do not need this card. They run through the API and move onto GPU capacity only when throughput calls for it.

FAQ

Is the RTX Pro 6000 good for AI inference? Yes, for open models in the 20B to 35B range and for smaller models at higher batch sizes. Its 96 GB lets one card serve a mid-sized model with room for context and concurrency.

How much VRAM does the RTX Pro 6000 have? 96 GB of GDDR7, on the Blackwell Server Edition that EcoHash deploys.

How does it differ from an H100 for inference? The H100 has 80 GB of HBM with higher bandwidth and NVLink, which suits large models and multi-GPU serving. The RTX Pro 6000 has 96 GB of GDDR7 over PCIe, which suits single-GPU serving of 20B to 35B models at a lower hourly cost.

What models fit on a single RTX Pro 6000 96GB? Open models up to about 35B, including qwen3-coder-30b-a3b-instruct and gpt-oss-20b. Models near 70B usually need quantization.

Can it run a 70B model? Sometimes, with quantization and reduced context or batch size. Validate the specific model and settings before relying on it.

Does it support FP8 and other low-precision formats? Blackwell supports the low-precision formats used in inference, including FP8. Confirm what your serving stack uses and check output quality before depending on it.

How much does it cost on EcoHash? $1.89 per GPU-hour. See pricing for the current rate.

Do voice or small models need it? No. Models like Kokoro and Whisper run through the API, and move onto GPU capacity only when throughput demands it.

Next steps

Launch a GPU workspace: ecohash.com/gpu-compute
Browse open models: ecohash.com/models
Reserve steady capacity: ecohash.com/dedicated-inference