TL;DR
- Scality is a data infrastructure software vendor whose object storage platform sits under large AI and media workloads, and its AI Inference Factory is a software stack for running open-weight models on servers that a customer owns.
- Scality announced availability on 8 October 2026 and describes the product as "an open-code software stack for deploying and operating AI inference on enterprise-owned infrastructure."
- Scality validates the whole stack and ships it as one maintained product, which places model versions, serving software and the storage layer under a single support contract.
- The serving layer is disaggregated, so prefill runs in one pool and decode in another and each scales on its own; a cache written by one pool is read back by whichever decode GPU picks up the session.
- Scality ADI is the object storage layer underneath, and its job is holding the KV cache outside GPU high-bandwidth memory so a returning conversation becomes a storage read.
- The measured numbers are the vendor's own: a 14K-token context restored from ADI produced a warm time-to-first-token of 166 ms, 83 ms behind HBM, and retrieval ran 14 times faster than recomputation.
- The commercial case rests on pricing, since the stack competes with per-token cloud inference and puts a licence or a managed service where a metered bill used to sit.
- Availability is a software licence or a fully managed service, on standard servers from Dell, HPE, Lenovo and Supermicro, with validated models including Mistral, Gemma, gpt-oss, Qwen, Kimi, GLM and DeepSeek.
One software stack that Scality validates and ships
Scality's announcement frames the product around a problem that shows up late in an AI programme. A team that has settled on open-weight models and bought the GPUs still has to assemble a serving engine, a cache layer, a request router, a metering plane and enough storage to keep a long context alive, and every one of those pieces moves on its own release cycle. The AI Inference Factory is Scality's answer to that integration burden: a stack the vendor validates end to end and maintains as one product.
Scality's official AI Inference Factory press-release graphic (scality.com, published 8 October 2026, retrieved 8 October 2026), which states the vendor's own positioning for the launch.
The buyer profile in the announcement is specific. Enterprises, government agencies and neo-cloud providers make up the target market, which is the group that has to answer where its prompts and documents are processed and who can change a model version overnight. Scality's positioning follows that audience: organisations want greater control over where inference runs, how models are managed and what happens to their data, and a validated stack is how a vendor sells that control without asking the customer to become an integrator.
The model list keeps the offer concrete. Scality validates Mistral, Gemma, gpt-oss, Qwen, Kimi, GLM and DeepSeek as part of the supported stack, which means version and quantization decisions sit inside the maintenance contract rather than inside a research repository. The harness list runs in the same direction: OpenCode, Hermes, Goose, LangGraph and Pydantic AI stay usable because the stack exposes an endpoint those tools already speak. Nothing about that combination is exotic, and the value is in the packaging rather than in a new inference algorithm.
The serving layer that splits prefill from decode
The architecture decision worth following is the split between prefill and decode. A transformer answering a request does two distinct jobs: it reads the whole prompt and builds the key and value tensors for it, then it generates tokens one at a time against those tensors. Prefill is compute-bound and happens once per prompt; decode is memory-bandwidth-bound and repeats until the answer ends. Running both on the same GPU means the two workloads compete for the same silicon, and the pool has to be sized for whichever phase peaks first.
Scality separates them. The announcement describes a disaggregated inference-serving layer, and StorageReview's report adds that the serving layer is built on vLLM and Dynamo, with each GPU pool scaling on its own. The consequence is a fleet where prefill capacity and decode capacity are purchased and scheduled independently, which is the shape a company wants when its traffic is one long document plus a short answer, or the reverse.
Splitting the phases creates a new problem, and the answer to it is the interesting part of the launch. If the prefill pool computes the cache and the decode pool consumes it, that cache has to live somewhere both can reach, and the obvious answer is storage. StorageReview describes vLLM breaking the cache into fixed-size named blocks that ADI stores as objects, with a connector that replaces the S3 control plane with a stripped-down version using single-pass authentication and dropping XML and multipart framing. The optimisation is mundane and important: a KV block is small, latency-sensitive and written once, so the generic object protocol is mostly overhead.
Netics editorial card built from Scality's announcement: the four dimensions where a validated on-premises stack differs from a metered cloud endpoint, from where prompts are processed to where the context state lives.
Scality ADI keeps the KV cache outside GPU memory
The cache is the scarce resource in a long-context deployment, and the announcement states the storage layer's role directly: Scality ADI is the shared storage layer, "extending key-value (KV) cache beyond limited GPU high-bandwidth memory." A conversation that returns after a pause normally costs a recomputation, because the tensors that describe it were evicted. ADI keeps them instead, which turns the second arrival into a retrieval.
The design underneath that claim comes from StorageReview's report. ADI presents a single namespace that tiers data across TLC flash, HDD and optional tape, so model weights, enterprise data and cold cache each land on media that matches how often they are read. The KV path gets the fastest tier and a connector tuned for its access pattern, which is what makes a cache read competitive with a recomputation in the first place.
The honest framing here is that this is a vendor measuring its own product, and the announcement says so plainly by publishing the test conditions. For a buyer, the direction of the trade matters more than the exact multiplier. Moving the cache out of GPU memory lowers the amount of HBM a deployment has to buy for a given session count, and it moves the cost into storage and network capacity instead. That substitution is what makes the offer legible on a spreadsheet, and it is why the vendor's own numbers are worth reading even with their provenance attached.
The measured latency of a restored context
Scality publishes a set of numbers that describe the substitution in the terms a serving engineer would recognise. Loading Gemma-3 27B from ADI took 1.9 seconds, almost ten times faster than local NVMe, with the model streamed in parallel across the cluster over RDMA. A 14K-token context restore produced a warm time-to-first-token of 166 ms, 83 ms behind the figure the same workload produced from HBM, and the announcement notes no measurable impact on token generation because the restore finishes before the first token is due. Data transfer between GPUs and storage reached 97 percent of network line rate.
The retrieval figures carry the argument further. On a 14K-token context, pulling the cache was 14 times faster than recomputing it, and on a 439K-token context the gap widened to 72 times. The second number is the one that matters for regulated document work, because long contexts are exactly where recomputation is most expensive and where a storage-based cache has the most room. The announcement adds that the cache can exceed a single GPU's memory by more than 80 times and keep up to a thousand concurrent sessions resumable without recomputation.
Netics editorial card built from Scality's measurement of a 14K-token context restored from ADI: 166 ms warm time-to-first-token, 83 ms behind HBM, on the vendor's own test.
The CTO states the design intent in one line: "The KV cache on Scality ADI is fast enough to sit in the serving path." That sentence is the whole claim in miniature. A cache layer on the serving path either meets the latency budget of the phase that reads it, or it becomes an expensive archive. The published numbers exist to argue the first case, and the sentence about restoring a context in the same order of magnitude as GPU memory is how the vendor draws the boundary it claims to have crossed.
The procurement question behind per-token pricing
The commercial argument in the announcement is about predictability, and it is stated in the vocabulary of a budget rather than of a benchmark. Per-token pricing grows with every workflow that turns out to be useful, providers can change model versions and quantization on their own schedule, and a regulated organisation has to send prompts, documents and code to infrastructure it does not run. The stack answers each of those with a contract term: a fixed licence or a managed service, model versions that change when the customer decides, and inference that runs where the data already is.
Whether that trade favours a given company depends on numbers the announcement cannot supply. An on-premises stack converts a variable bill into capital, floor space, power and an operations team, and it adds a maintenance obligation that a cloud endpoint never asks for. The counterweight is the shape of the workload. A team whose inference bill is dominated by a small number of high-value, long-context, data-sensitive workflows is buying exactly the thing the stack sells, and a team running many small, bursty, low-sensitivity requests is buying capacity it will use unevenly.
Two details in the announcement describe how a company would start. Availability is a software licence or a fully managed service, which lets an organisation without a storage engineering group buy the outcome instead of the assembly, and the deployment targets are standard servers from Dell, HPE, Lenovo and Supermicro, so the hardware decision stays separate from the software decision. The reference architecture was demonstrated in Paris on 8 October 2026, which gives an interested team a concrete design to compare against its own.
Scality's own press-release page for the AI Inference Factory (scality.com, published 8 October 2026, retrieved 8 October 2026), including the product description and the vendor's launch commentary.
Our own view is that this class of launch marks a shift in how on-premises AI is sold. A year ago the question was which GPU to buy, and the answer was a component. The interesting question now is where the context lives between requests, and the answer is a storage and scheduling decision that a company can plan around. That is the shift the Netics services are built for, and the same discipline appears in our infrastructure monitoring work, where the goal is a system that explains itself instead of one that merely runs.
Originally published on the Netics blog.
Top comments (0)