packet.ai Token Factory: A Managed LLM Inference API from the Team That Cut GPU Prices by 50%+
packet.ai Token Factory is a managed LLM inference API. OpenAI-compatible change
your base_url, swap your API key, and your existing SDK calls run against open-weight
models with no GPU to manage, no serving stack to maintain, and per-token billing that
drops to zero when your app is not generating tokens.
Token Factory is launching soon. The waitlist is live at
packet.ai/token-factory.
Here's why the pricing will be different from what you're used to.
The GPU pricing track record
packet.ai sells NVIDIA GPUs at prices that make most
developers check the rate card twice.
The NVIDIA B200 — 192GB HBM3e, Blackwell — is $3.75/hr
on Dynamic and $6.99/hr on Dedicated. RunPod lists it at $5.89/hr. Lambda Labs at $6.99/hr.
The NVIDIA RTX 6000 Pro — 96GB GDDR7 — is $0.66/hr
on Dynamic. RunPod charges $2.09/hr for the same card. That is a 68% difference on
the same silicon.
Current live rates vs the competition, sourced from packet.ai/pricing:
| GPU | packet.ai | RunPod | Lambda Labs | vs RunPod |
|---|---|---|---|---|
| NVIDIA B200 (192GB HBM3e) | $3.75/hr Dynamic | $5.89/hr | $6.99/hr | 36% cheaper |
| NVIDIA RTX 6000 Pro (96GB GDDR7) | $0.66/hr Dynamic | $2.09/hr | — | 68% cheaper |
| NVIDIA A100 80GB | $1.43/hr Dedicated | $1.49/hr | $1.99/hr | 28% vs Lambda |
| NVIDIA RTX 4090 (24GB GDDR6X) | $0.39/hr Dedicated | $0.69/hr | — | 43% cheaper |
Prices in USD/GPU-hr. Published starting rates, September 2026.
This is what the scheduler makes possible: co-locating
workloads that stress different GPU dimensions so the silicon never contends. Same peak
performance. Same VRAM. Lower price. That is the structural advantage.
The same reasoning is now going into inference.
Why managed LLM inference pricing is broken
If you're building any LLM-powered product at real volume, you've hit one of three walls.
Proprietary APIs. OpenAI, Anthropic, Google. The developer experience is clean.
The per-token cost at scale is not. Output tokens on frontier models compound fast
every agent call, RAG retrieval, and background summarization job adds to the bill.
And you're paying frontier rates for tasks that open-weight models handle just as well.
Self-hosting on GPU compute. Cheaper at steady-state volume. Also a part-time job.
You pick a model, stand up vLLM or TGI, configure batching and quantization, write
autoscaling logic, build monitoring, and respond to the 2am CUDA OOM that takes your
endpoint down. Every hour spent there is an hour not spent on what you're building.
Managed open-weight inference. Fireworks AI, Together AI, Groq, Replicate these
exist. But most land closer to proprietary API rates than to what self-hosting actually
costs. You're paying a GPU margin you can't see, on models whose serving cost has
dropped significantly since 2024.
Token Factory goes into the gap: managed inference, open-weight models, on packet.ai
hardware, priced with the same logic as the GPU products.
What is packet.ai Token Factory?
Token Factory is packet.ai's managed LLM inference API, built on the same GPU
infrastructure that powers the packet.ai cloud.
It exposes the same HTTP API OpenAI uses: POST /v1/chat/completions, the same
request schema (model, messages, temperature, tools), the same response format.
Switching requires three config changes: base_url, api_key, and model name.
Nothing else.
# Before
from openai import OpenAI
client = OpenAI(api_key="sk-...")
# After: same SDK, same code, one line changed
from openai import OpenAI
client = OpenAI(
base_url="<token-factory-endpoint>", # published in dash.packet.ai/docs at launch
api_key="your-packet-api-key"
)
LangChain's ChatOpenAI, LlamaIndex's OpenAI LLM class, any direct API call - all
accept base_url as a single parameter. Chains, agents, RAG pipelines, and tool-calling
loops carry over without structural changes to your application.
What Token Factory includes at launch:
- OpenAI-compatible endpoint (
POST /v1/chat/completions) - Streaming via Server-Sent Events
- Function calling for tool-use and agentic workflows
- Per-token billing with scale-to-zero
- US and EU regions
- No GPU to provision, no serving stack to maintain
Who Token Factory is for
Token Factory is not right for every workload. Here is where it fits clearly and
where it does not.
Good fit:
- Solo developers who cannot justify renting a dedicated GPU for one application
- Teams whose OpenAI bill is growing faster than revenue
- ML engineers building agentic applications with high per-user call volume
- Any team running batch eval, RAG, summarization, classification, or extraction at scale
- Teams reducing single-provider dependency and wanting a clean fallback endpoint
Not the right fit:
- Workloads that require extended frontier-model reasoning (o3, o4 class)
- Fine-tuning runs those need direct GPU access (use a Dynamic POD)
- On-premise compliance requirements
- Teams still under 1,000 calls per day, a proprietary API is cheaper to start
For the detailed breakdown of each developer profile and when self-hosting is
the better call, read Who Is Token Factory For?
Token Factory vs self-hosting vs proprietary APIs
| Factor | Token Factory | Self-host on GPU | Proprietary API |
|---|---|---|---|
| Billing model | Per token, no idle cost | Per GPU-hour (always on) | Per token (frontier rates) |
| Setup time | Minutes (URL + key swap) | Hours to days (vLLM config) | Minutes |
| Scale-to-zero | Yes, zero cost when idle | Manual (stop the pod) | Yes |
| Infrastructure to manage | None | GPU + serving stack | None |
| OpenAI SDK compatible | Yes | Yes, via vLLM | Yes (native) |
| Function calling | Yes | Depends on config | Yes |
| Streaming | Yes | Yes, via vLLM | Yes |
| Best for | Solo devs, growing startups | High-volume steady-state | Prototyping, frontier tasks |
If you outgrow Token Factory and want direct GPU access to run your own stack,
packet.ai's Dynamic GPU tier starts at
$0.66/hr and spins up in under 5 minutes.
How Token Factory will be priced
We are not publishing rates before launch.
Here is what you need to know in the meantime: the team that decided the B200 should
cost $3.75/hr instead of $5.89/hr is building Token Factory. The same reasoning applies.
Open-weight inference has improved faster than the price of calling it has dropped.
There is no structural reason managed open-weight inference has to cost anywhere near
what closed-model APIs charge. We do not think it should.
If you have been watching our GPU pricing, you have a
reference point for the direction Token Factory pricing will take.
Frequently asked questions
What is packet.ai Token Factory?
Token Factory is packet.ai's managed LLM inference API. It serves open-weight models
through a drop-in OpenAI-compatible endpoint, same API format, same SDK, change only
base_url and API key. Per-token billing with scale-to-zero, function calling, and
streaming supported at launch. No GPU or serving stack to manage. Built on packet.ai's
GPU infrastructure across US and EU regions.
How does Token Factory compare to Fireworks AI, Together AI, and Groq?
All four are managed inference options for open-weight models with OpenAI-compatible
endpoints. Fireworks AI and Together AI offer larger model catalogs. Groq is optimized
for raw throughput speed via dedicated LPU hardware. Token Factory's differentiation
is pricing the same cost-structure advantage that makes packet.ai GPU rentals
36-68% below RunPod on the same silicon is the foundation for how Token Factory
will be priced.
Does Token Factory support function calling and streaming?
Yes to both. Streaming runs via Server-Sent Events. Function calling follows the same
tool-use schema as OpenAI's API. Agent loops and tool-use workflows built on the OpenAI
format carry over without changes to application logic.
When does Token Factory launch and how do I get access?
Token Factory is launching soon. Access opens in waves starting with the waitlist.
Join at packet.ai/token-factory we'll email
when your spot is ready.
Get in before the first wave
Token Factory opens access to waitlist members first.
If per-token cost is a real number in your budget, or you have been looking for a
managed inference option that does not charge like it is pre-DeepSeek pricing still,
this is worth 30 seconds.
Join the Token Factory waitlist
Related: Who Is Token Factory For?
the full breakdown on which developer profiles Token Factory is and is not built for,
including when running your own stack on a Dynamic GPU
is the right call instead.
Top comments (0)