Sooner or later a product stops being happy with somebody else's hosted API. A fine-tune, an open-weights LLM, a vision pipeline nobody sells as a service: at that point you go shopping for inference capacity, and the price list makes no sense.
Here is the part that should bother you. Rates read straight off the vendor pricing pages on August 22, 2026, for the exact same NVIDIA H100:
- Modal: $3.95/hr, charged by the second
- Replicate: $5.49/hr, charged by the second
- Together AI: $5.49/hr on a dedicated endpoint, charged by the hour
- Baseten: $6.50/hr, charged by the minute
Same chip, 65% apart. The gap is not a hardware difference, it is four companies disagreeing about what they are selling you: seconds of compute, managed replicas, a model catalog, or tokens. I put the longer version of this comparison, with the full pricing tables and screenshots, on DevToolLab if you want every number.
Worth noting who is winning the argument financially: Baseten announced a $1.5 billion Series F in June 2026 at a reported $13 billion valuation, and Cloudflare acquired Replicate outright.
The Billing Unit Is the Real Product
| Platform | Charged by | H100 | Free tier | Fits |
|---|---|---|---|---|
| Modal | Second | $0.001097/sec, $3.95/hr | $30/mo in credits | Custom Python, scale to zero |
| Baseten | Minute | $0.10833/min, $6.50/hr | Credits on request | Production replicas with an SLA |
| Replicate | Second, or token | $0.001525/sec, $5.49/hr | Pay as you go | Calling catalog models |
| Together AI | 1M tokens, or GPU-hour | $5.49/hr dedicated, $3.99/hr clusters | Pay as you go | Open LLMs on an OpenAI-shaped API |
| vLLM | Whatever your GPUs cost | Apache 2.0, 89,671 stars | Free | You already own the hardware |
Read that middle column twice. Per-second pricing pays off when traffic arrives in bursts and the box can drop to zero in between. Per-minute pricing quietly assumes a replica that never sleeps. Per-token pricing removes GPUs from your vocabulary entirely, and hands the efficiency upside to the vendor.
Modal
Modal treats GPUs like a language feature. Decorate a function, declare the hardware, deploy. Nothing bills while nothing runs. CPU is $0.0000131 per core-second, memory $0.00000222 per GiB-second, and the accelerator menu spans a T4 at $0.59/hr up to a B300 at $7.10/hr. The free Starter plan includes $30/month of credits; Team is $250/month with $100 of that back as credits.
Checked against modal 1.5.4, a deployable endpoint is about this short:
import modal
app = modal.App("llm-inference")
image = modal.Image.debian_slim().pip_install("vllm")
@app.function(image=image, gpu="H100", timeout=600)
def generate(prompt: str) -> str:
# container loads the model once, then serves many requests
...
# modal deploy app.py -> endpoint that drops to zero between calls
Strengths: cheapest H100 in this group, genuine scale to zero, and cold starts that someone clearly obsesses over. A 40 second job costs 40 seconds of GPU.
Trade-off: this is infrastructure written as Python, not a deploy button on a model page. Modal is also general purpose compute rather than an inference product, so model registries and canary rollouts are things you build.
Baseten
The pitch is the operational gap between a model file and a service with an SLA: replica autoscaling, observability, dedicated deployments that do not share a queue with strangers. Packaging goes through Truss, its open-source format. Investors like the story, first $300 million at a $5 billion valuation in January 2026 per Bloomberg, then the June Series F at a reported $13 billion on roughly $600 million annualized revenue.
Uptime bills per minute: H100 at $6.50/hr, A100 at $4.00/hr, B200 at $9.98/hr, and a sliced H100 MIG at $3.75/hr, training included at the same rates. A separate Model APIs tier serves hosted open-weights models (Kimi K3 and DeepSeek V4 today) per million tokens instead.
Trade-off: you are paying 65% over Modal for identical silicon, per-minute granularity rounds bursty traffic upward, and there is no standing free allowance, just credits if you ask.
Replicate
Replicate is the catalog play: thousands of models, community and commercial, behind a single API. Image generation like flux-1.1-pro at $0.04 per output image, hosted LLMs billed per token. Verified with replicate 1.0.7, calling one really is a single line:
import replicate
output = replicate.run(
"black-forest-labs/flux-schnell",
input={"prompt": "a watercolor map of San Francisco"},
)
The governance news matters more than the pricing news. Cloudflare said on November 17, 2025 that it was buying Replicate, closing inside two months, with the catalog heading into Workers AI. Already on Cloudflare? This just became your obvious pick. Betting on Replicate staying neutral? That bet is settled.
Two billing modes. Public models charge per second of processing, or per token for text. Private models get dedicated hardware, and the pricing page deserves credit for spelling out the catch: setup time, idle time, and active time all bill. H100 at $5.49/hr, A100 at $5.04/hr, L40S at $3.51/hr, multi-GPU stacking linearly. Fast-booting fine-tunes are the one exemption from idle charges.
Trade-off: that idle clause is where surprise invoices come from, since a private model with thin traffic pays through every quiet hour. Public model cold starts also track popularity, because busy models stay warm on shared capacity.
Together AI
Together approaches this from the token end. Serverless open-weights models on an OpenAI-compatible API: DeepSeek V4 Flash at $0.14 in / $0.28 out per million, V4 Pro at $1.32 / $3.96, Kimi K3 at $3.00 / $15.00, with cached input discounts (Kimi K3 cached input falls to $0.30) and a batch toggle. Reported numbers put the company past $1 billion annualized at a $7.5 billion valuation after a $305 million Series B.
Outgrow serverless and the same vendor sells you every rung down the ladder: dedicated endpoints at $5.49 per H100-hour ($8.99 for B200), provisioned throughput for reserved token capacity, and bare GPU clusters at $3.99 per H100-hour on demand, falling to $3.19 with a 91 to 180 day reservation. Client shape, verified on together 2.31.0:
from together import Together
client = Together() # reads TOGETHER_API_KEY
resp = client.chat.completions.create(
model="deepseek-ai/DeepSeek-V4",
messages=[{"role": "user", "content": "Summarize this diff"}],
stream=False,
)
Trade-off: per-token serverless is the quickest start and the least leverage. Deprecation timing, shared-queue latency and margin all sit on Together's side of the table. Need a model that is not in the catalog and you land back in GPU-hour territory with everyone else.
Or Skip All of Them
Each platform above is a wrapper around a serving engine you can run yourself. vLLM (Apache 2.0, 89,671 stars, pushed today) is the default choice, with continuous batching and paged attention turning a single H100 into a serviceable API. SGLang (32,244 stars, also active today) often wins on structured output and multi-turn traffic. Hugging Face TGI (10,887 stars) still runs but has gone quiet, nothing pushed since March 2026.
The break-even is utilization plus staffing. Reserved H100 at $3.19/hr running vLLM near 70% utilization beats every per-token price on this page. The same box at 10% utilization loses to all of them, and you still own the CUDA driver upgrades and the 3 a.m. OOM alerts.
What 730 Hours Actually Costs
One model, one H100 replica, always on, a full month:
| Platform | Monthly |
|---|---|
| Modal | $2,884 |
| Together dedicated | $4,008 |
| Replicate private | $4,008 |
| Baseten | $4,745 |
Now change one assumption. Give that model two hours of real GPU work per day instead of 24, on a platform that can scale to zero. Modal drops to roughly $237/month. Everything holding a warm replica still charges the always-on figure. That is the whole category compressed: whether idle time bills matters more than the hourly sticker, and for spiky traffic scale to zero is a 10x lever. The full write-up walks through the per-workload math in more detail.
Token-shaped work inverts it. At low volume, Together serverless (or Baseten Model APIs, or Replicate per-token models) costs pocket change and needs zero capacity planning. Somewhere past a few billion tokens a month, dedicated GPUs undercut per-token rates.
Picking One
- Custom Python model, unpredictable traffic: Modal. Per-second billing, real scale to zero, cheapest H100 here.
- Steady production load, SLA promises, a team that wants canaries and autoscaling handled: Baseten. Priced as the premium option because that is what it is.
- Mostly calling models somebody else built, across image, video, speech, text: Replicate. A bonus if you are already on Cloudflare, a question to answer if you are not.
- Open-weights LLMs on an OpenAI-shaped API, prototype through reserved cluster: Together AI.
- GPUs already on the books plus engineering capacity: vLLM or SGLang.
Two tools that help before you commit: the latency percentile calculator turns a batch of inference response times into the p50/p95/p99 numbers platforms actually get compared on, and the uptime SLA calculator converts a vendor's promised nines into real allowed downtime per month.
Closing Thought
Price the shape of your traffic, not the H100 sticker. The $3.95 to $6.50 spread is real money, but the gap between billed-idle and scale-to-zero is far wider. It is why one workload costs $237 or $4,745 a month on platforms that look interchangeable in any feature grid.
Meanwhile the market keeps folding upward. Cloudflare owns Replicate, Baseten tripled its valuation in five months, Together crossed a reported billion annualized. Every one of them sells a cheap on-ramp and an expensive destination, and every one of them expects you to make that trip. Re-run your own numbers before they turn out to be right.
Top comments (0)