DEV Community: Runcrate

How to compare GPU cloud prices without fooling yourself

Runcrate — Wed, 21 Oct 2026 00:55:46 +0000

"H100 from $X an hour" is the least useful number in a GPU price list. Two offers with the same headline can differ by a factor of two in what you pay at the end of the month, and two offers with very different headlines can come out the same.

We publish GPU prices at Runcrate, and other people's comparison sites read them, so I spend a lot of time looking at how a price turns into a bill. This is the checklist I use. It works for any provider, ours included. I'll use our public pricing endpoint for the examples because it's the data I can show you with a script, not because the method depends on it.

1. Get to one unit: dollars per GPU-hour

Some lists quote the machine, some quote the GPU. An 8x H100 node at a per-machine price looks eight times worse than a 1x H100 at a per-GPU price. Divide by the GPU count first, always.

Runcrate's endpoint already reports per GPU-hour and needs no API key:

import requests

data = requests.get("https://www.runcrate.ai/api/gpu-pricing").json()
print("fetched at", data["fetched_at"])

for g in data["gpus"]:
    s = g["specs"]
    per_gpu = s["num_gpus"] or 1
    print(
        f'{g["name"]:12} {g["vram"]:>6}  '
        f'${g["min_price_hr"]:.2f} to ${g["max_price_hr"]:.2f} per GPU-hr  '
        f'{g["available_options"]:>2} configs  GPUs per machine {g["gpu_counts"]}  '
        f'~{s["vcpus"] // per_gpu} vCPU / {s["memory_gb"] // per_gpu} GB RAM per GPU'
    )

Part of what it printed when I ran it (fetched at 2026-10-21T00:51 UTC):

H100 SXM       80GB  $3.38 to $7.79 per GPU-hr   7 configs  GPUs per machine [1, 8]  ~20 vCPU / 128 GB RAM per GPU
A100 80GB      80GB  $1.76 to $4.26 per GPU-hr  12 configs  GPUs per machine [1, 2, 8]  ~28 vCPU / 120 GB RAM per GPU
L40S           48GB  $1.14 to $1.95 per GPU-hr  10 configs  GPUs per machine [1, 2, 4, 8]  ~12 vCPU / 72 GB RAM per GPU
A6000          48GB  $0.65 to $2.51 per GPU-hr  18 configs  GPUs per machine [1, 2, 4]  ~28 vCPU / 58 GB RAM per GPU

Notice the ranges. One GPU model can have a dozen configurations with a 2x to 4x spread between them. A comparison that keeps only the minimum hides that the minimum might be one 8-GPU node in one region.

2. Compare the same GPU, not the same name

"A100" is two products (40 GB and 80 GB). "H100" is SXM or PCIe, with different memory bandwidth and interconnect. A "4090" with 24 GB will not hold the model your 48 GB card holds. Before comparing prices, match:

VRAM (the endpoint has vram per GPU type)
form factor and interconnect for multi-GPU work (interconnect, for example NVLink)
GPU count per machine, because you can't split an 8-GPU node

If your model needs 80 GB, the price of a 40 GB card is irrelevant, however low it is.

3. Count what comes with the GPU

A GPU-hour also buys CPU cores, RAM and local disk, and those vary a lot. In the output above, an A6000 can come with about 28 vCPUs, an L40S with about 12. For data loading, tokenization or anything CPU-bound in your pipeline, the "slower" card with more cores can finish first.

So note, per GPU: vCPUs, RAM, local disk, and whether persistent storage costs extra. Runcrate's volumes, for example, are billed separately at $0.03 per GB per month. Your other quotes will have their own line for it, or bundle it.

4. Billing granularity matters more than the rate for short work

This is the one most comparisons skip. Take a 7-minute eval on a $1.14/hr GPU:

billed per second: 1.14 x 7 / 60 = $0.13
billed per started hour: $1.14, nearly 9x more

For long training runs granularity barely matters. For evals, CI jobs, agent experiments and anything bursty, it dominates. Check three things:

The billing increment (second, minute, hour). Runcrate bills most instances by the second and a few GPUs by the minute; the endpoint's min_price_per_minute flag marks the latter.
When the meter starts: at request, at boot, or once the machine is running. Runcrate starts it when the machine is running, so provisioning isn't billed.
When it stops: only when you remember to delete the machine, or also on its own. Idle shutdown and maximum runtimes turn "I forgot it overnight" from a 12-hour bill into a 30-minute one, which is worth more than a few cents on the rate.

5. Listed is not launchable

A price only matters if you can get the machine. Availability moves by the minute: in the same hour I wrote this, a GPU type showed one configuration on the pricing endpoint while a live launch query showed none. When comparing:

try an actual launch, or at least a live availability query, in the region you need
note the count of configurations and regions, not just the headline price
separate on-demand from spot or interruptible capacity, which is cheaper because it can disappear

6. Date every number

GPU prices change weekly, sometimes daily. A comparison without a date is a guess. Put the fetch time next to every price (the endpoint returns fetched_at for that reason), and re-run your script rather than reusing last month's table.

A template you can reuse

For each offer, one row:

Field	Why
GPU model, VRAM, form factor	same product?
GPUs per machine	can you buy just one?
$/GPU-hour	the normalized rate
vCPU, RAM, disk per GPU	the rest of the machine
Billing increment, meter start	cost of short jobs
Auto-stop options	cost of mistakes
Storage and egress	cost of data
Region, availability checked at	can you launch it
Fetched at	how old the number is

Then price your actual workload (hours per run, runs per month, data moved) against each row. The offer with the best headline wins less often than you'd think.

Switch from OpenAI to open models in two lines (streaming, tools and JSON mode still work)

Runcrate — Wed, 21 Oct 2026 00:33:52 +0000

Most LLM code I see calls the OpenAI SDK. That's fine, because the SDK doesn't care whose server is on the other end. If the server speaks the same API, you can move a whole app to open models by changing two lines, and keep everything else: streaming, tool calls, JSON mode, retries.

This post shows the switch with Runcrate's inference API, which serves 223 models behind one OpenAI-compatible endpoint, with open-weight families like DeepSeek, Qwen, Llama, Kimi, GLM and gpt-oss among them. Every snippet below was run against the live API on 2026-10-21.

The two lines

Before:

from openai import OpenAI

client = OpenAI()

After:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.runcrate.ai/v1",
    api_key=os.environ["RUNCRATE_API_KEY"],
)

Then pick a model name from the catalog:

r = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3.2",
    messages=[{"role": "user", "content": "Summarize this stack trace: ..."}],
)
print(r.choices[0].message.content)

Keys start with rc_live_ and come from Dashboard > Keys > API Keys.

Or zero lines

Both official SDKs read the base URL and key from the environment, so for code that already does OpenAI() with no arguments you don't have to touch the code at all:

export OPENAI_BASE_URL=https://api.runcrate.ai/v1
export OPENAI_API_KEY=rc_live_...

I checked this with openai for Python and for Node. Same for TypeScript if you prefer it explicit:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.runcrate.ai/v1",
  apiKey: process.env.RUNCRATE_API_KEY,
});

const r = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "Reply with the word ok." }],
});
console.log(r.choices[0].message.content);

And plain curl:

curl https://api.runcrate.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNCRATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-oss-120b", "messages": [{"role": "user", "content": "Reply with the word ok."}]}'

Picking a model, with its price

models.list() returns every model with its per-token price, so you can choose from code instead of a pricing page. This prints the text models and what they cost per million tokens:

from openai import OpenAI

client = OpenAI()  # OPENAI_BASE_URL and OPENAI_API_KEY set as above
for m in client.models.list():
    if m.output_modalities != ["text"]:
        continue
    p = m.pricing
    print(f'{m.id:55} ${float(p["prompt"]) * 1e6:.3f} in / ${float(p["completion"]) * 1e6:.3f} out per 1M tokens')

On 2026-10-21 that printed 160 text models. A few of them:

Model	Input per 1M tokens	Output per 1M tokens
`openai/gpt-oss-120b`	$0.037	$0.17
`meta-llama/Llama-3.3-70B-Instruct-Turbo`	$0.10	$0.32
`deepseek-ai/DeepSeek-V3.2`	$0.26	$0.38
`Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo`	$0.30	$1.00
`moonshotai/Kimi-K2.5`	$0.49	$2.50

Prices change, so read them from the endpoint rather than from this post. The rest of the catalog is image, video and speech models (FLUX, Whisper and others), on the same key.

What carries over

I ran each of these against deepseek-ai/DeepSeek-V3.2 with the stock Python SDK, no changes:

Streaming

stream = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3.2",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Tool calls. With a get_weather(city) tool and the prompt "What's the weather in Paris?", the response came back with finish_reason="tool_calls" and arguments {"city":"Paris"}, the same shape your existing tool loop already handles.

JSON mode. response_format={"type": "json_object"} returned a parseable object.

Reasoning models. Models that think out loud, such as openai/gpt-oss-120b, put the reasoning in a separate reasoning_content field next to content, so your app still reads content as the answer.

Support for tools and JSON mode depends on the model, as it does anywhere, so test the one you pick with your own prompts before you switch production traffic.

What doesn't carry over

Model names. gpt-4o doesn't exist here; an unknown name returns a 404 with code: "model_not_found". Keep the model name in config so the switch is one value.
Embeddings. There are no embedding models in the catalog right now, so leave your embedding calls where they are.
This post only covers chat completions. If you rely on other OpenAI endpoints, check them first.

A pattern I like: one env var per environment

Because only the base URL, key and model change, I keep all three in env vars and switch per environment without code changes:

OPENAI_BASE_URL=https://api.runcrate.ai/v1
OPENAI_API_KEY=rc_live_...
LLM_MODEL=deepseek-ai/DeepSeek-V3.2

That also makes A/B tests between models a config change, which is how most people end up comparing cost and quality on their own data instead of on a leaderboard.

Give your coding agent a GPU, and a budget: the Runcrate MCP server in Claude Code, Codex and Cursor

Runcrate — Tue, 20 Oct 2026 19:38:02 +0000

Coding agents are good at writing a training script. They are bad at the part that comes next: finding a GPU, getting a shell on it, running the script, and remembering to shut the machine down. That last step is where the surprise invoices come from.

This post shows how to hand that whole loop to the agent you already use (Claude Code, Codex, Cursor or any MCP client) with the Runcrate MCP server, and how to keep it on a leash while it spends money.

What the agent gets

The server lives at https://mcp.runcrate.ai/mcp (Streamable HTTP). It exposes tools for:

GPU instances: list GPU types with live prices, create an instance, check its status, run shell commands on it over SSH, upload and download files, delete it.
Jobs: run one command on a fresh GPU machine that stops itself, and its billing, when the command exits. The agent follows the logs and reads the exit code and cost.
Schedules: start an instance or a job later, once or on a cron.
Storage volumes, SSH keys, firewall rules and HTTPS links for a port on an instance.
Billing: balance, usage, and the monthly budget of the environment it works in.

Sign-in is OAuth in your browser. There is no API key to paste into a config file.

Connect it

Claude Code

claude mcp add --transport http --scope user runcrate https://mcp.runcrate.ai/mcp

Then run /mcp inside Claude Code and pick runcrate. Your browser opens to sign in.

Codex

codex mcp add runcrate --url https://mcp.runcrate.ai/mcp

Cursor: one-click install

VS Code: one-click install

Any other MCP client: follow the setup page.

The leash: one environment, one budget

When you sign in, you pick a workspace and one environment. The agent can act only inside that environment, and the environment's monthly budget caps what it can spend. No MCP tool can create or switch environments or change a budget, so the agent cannot raise its own ceiling. Budgets are set in the dashboard by a workspace owner or admin.

Two more guardrails sit on every instance the agent creates:

max_runtime_minutes (5 minutes to 7 days): the instance is terminated that long after it starts running, with an alert 10 minutes before.
idle_shutdown_minutes (5 minutes to 24 hours): the instance is terminated once GPU utilization stays under 5% with nobody connected for that long.

You do not need to know the parameter names. Say it in the prompt:

Launch an H100, stop it after 4 hours or after 30 idle minutes.

Three prompts that work

Check the hardware before you commit

What GPUs with at least 48 GB of VRAM are available right now, and what do they cost per hour?

The agent calls list_gpu_types and answers from live prices.

Run a training script and come back with the number

Run python train.py on an A4000 for at most 2 hours with my volume training-data, then tell me the final loss.

What happens under the hood:

list_volumes finds the id of training-data.
run_job starts the command with gpu: "A4000", max_runtime_minutes: 120 and the volume attached at /workspace.
get_job_logs follows the output until the job is done, and the agent reads the loss from it.
get_job returns the final status, exit code and cost.

The machine stops when the script exits, so there is nothing left running to forget about.

Keep a box for interactive work

Launch an RTX 4090, install vllm, serve Qwen/Qwen2.5-7B-Instruct on port 8000, and give me an HTTPS link only my workspace can open. Stop it after 3 hours.

Here the agent uses create_instance with a max runtime, ssh_execute to install and start the server, and create_instance_link to expose port 8000 at an https://8000-<instance>.runcrate.app address.

Prices, so you know what you are asking for

From the live pricing endpoint on 2026-10-20: RTX 4090 from $0.52/hr, A100 80GB from $1.76/hr, H100 from $3.55/hr, H200 from $4.24/hr. Booting is never billed, and the balance is prepaid, so there is no monthly fee. The current numbers are at runcrate.ai/pricing.

Not every task needs a GPU of its own

If the agent only needs a model, not a machine, the same account has 200+ open models behind an OpenAI-compatible API. The client you already use works:

from openai import OpenAI

client = OpenAI(base_url="https://api.runcrate.ai/v1", api_key="rc_live_...")
r = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3",
    messages=[{"role": "user", "content": "Summarize this stack trace: ..."}],
)
print(r.choices[0].message.content)

DEV Community: Runcrate

How to compare GPU cloud prices without fooling yourself

1. Get to one unit: dollars per GPU-hour

2. Compare the same GPU, not the same name

3. Count what comes with the GPU

4. Billing granularity matters more than the rate for short work

5. Listed is not launchable

6. Date every number

A template you can reuse

Links

Switch from OpenAI to open models in two lines (streaming, tools and JSON mode still work)

The two lines

Or zero lines

Picking a model, with its price

What carries over

What doesn't carry over

A pattern I like: one env var per environment

Links

Give your coding agent a GPU, and a budget: the Runcrate MCP server in Claude Code, Codex and Cursor

What the agent gets

Connect it

The leash: one environment, one budget

Three prompts that work

Prices, so you know what you are asking for

Not every task needs a GPU of its own

Links