DEV Community: Aeza

Beyond the AI Model Race: How OpenAI, Anthropic, Google, and xAI Are Competing for the Future of Computing

Aeza — Mon, 12 Oct 2026 14:56:54 +0000

A technical analysis of AI computing infrastructure, custom accelerators, cloud distribution, API pricing, model economics, and developer ecosystems in 2026.

October 2026

Introduction: When an AI Company Rents Computing Power From Its Competitor

In May 2026, Anthropic announced an agreement that would have seemed unusual just a few years earlier.

The company secured access to Colossus 1, a large AI supercomputer owned by SpaceX, the organization that acquired xAI in February 2026.

Colossus 1 was originally constructed for Grok.

According to the official announcement, the system contains more than 220,000 NVIDIA accelerators, including H100, H200, and GB200 configurations.

The arrangement provides Anthropic with access to substantial additional computing infrastructure, with the agreement described as involving more than 300 MW of capacity.

In practical terms, one of the world's leading AI model developers was acquiring computing resources from the organization behind a competing AI product.

This situation illustrates an important change in the industry.

The competition between major AI companies is no longer determined exclusively by model performance.

Frontier models improve rapidly, and the gap between competing systems can narrow within months.

But the infrastructure required to train, deploy, and operate those models is much harder to reproduce.

Electricity, semiconductors, data centers, cloud agreements, inference efficiency, and developer ecosystems are becoming strategic assets.

Five dimensions now define much of the competition:

Physical computing capacity.
Custom AI accelerators.
Cloud distribution.
Token pricing and inference economics.
Developer tools and ecosystem adoption.

Understanding these dimensions provides a more useful picture of where the industry is heading than examining model benchmarks alone.

1. The First Competitive Weapon: Gigawatts

1.1. Why electricity is becoming a more important metric than GPU counts

Modern AI infrastructure requires more than GPUs.

A large computing cluster depends on several interconnected systems:

AI accelerators.
CPUs and system memory.
High-speed networking.
Power distribution.
Liquid or other cooling infrastructure.
Backup power and electrical protection.
Data center buildings.
Connections to regional electricity grids.

At very large scales, electricity availability can become a major constraint on expansion.

Accelerators can be manufactured and purchased on commercial timelines.

High-capacity power connections, new transmission infrastructure, and large data center construction may require much longer.

This explains why AI companies increasingly discuss future capacity in megawatts and gigawatts.

However, several different measurements need to be distinguished.

Contracted capacity

Power or computing capacity that an organization has secured through agreements.

Under-construction capacity

Infrastructure being developed but not yet available for production workloads.

Operational capacity

Infrastructure installed, powered, cooled, and able to execute workloads.

Utilized computing capacity

The portion of operational infrastructure currently performing useful work.

These categories are not interchangeable.

A company can announce several gigawatts of future infrastructure while having a much smaller amount in production.

1.2. OpenAI: The Stargate approach

OpenAI has publicly described rapid growth in computing resources.

According to figures provided by its leadership:

Year	Reported computing capacity
2023	Approximately 0.2 GW
2024	Approximately 0.6 GW
2025	Approximately 1.9 GW

In April 2026, OpenAI announced that it had secured more than 10 GW of AI infrastructure commitments through Stargate.

The original Stargate objective involved securing 10 GW of infrastructure in the United States by 2029.

Reaching that contractual milestone early represents a substantial expansion of OpenAI's future computing pipeline.

But infrastructure commitments are not equivalent to installed production capacity.

Independent observers tracking large AI infrastructure projects have documented how construction, equipment installation, and electricity connections can delay planned deployments.

The distinction between committed and operational capacity is therefore essential.

1.3. OpenAI's diversified hardware portfolio

OpenAI's infrastructure strategy involves multiple accelerator suppliers and partners.

Its publicly announced relationships include:

NVIDIA systems.
AMD accelerator infrastructure.
Custom silicon developed with Broadcom.
AWS infrastructure, including Trainium arrangements.
Cloud capacity supported by Microsoft, Oracle, and other providers.

Some agreements describe accelerator supply.

Others describe data center development, cloud consumption, or infrastructure capacity.

Their nominal gigawatt figures cannot simply be added together.

The same facility may ultimately contain equipment provided under several related commitments.

OpenAI's strategic objective is to obtain sufficient computing power while reducing exposure to individual infrastructure suppliers.

1.4. Anthropic: Spreading computing across multiple architectures

Anthropic has adopted a multi-platform approach.

Its primary cloud and training partner is AWS, but the company also uses Google TPUs and NVIDIA accelerators.

In October 2025, Anthropic announced an expansion involving access to up to one million Google TPUs.

The company said the arrangement would bring well over one gigawatt of additional capacity online during 2026.

In April 2026, Anthropic expanded its partnership with Google and Broadcom for multiple gigawatts of next-generation TPU capacity, expected to become available beginning in 2027.

Its agreement to use Colossus 1 provided another source of computing resources.

This strategy offers several potential advantages:

Access to different accelerator architectures.
Reduced dependence on a single supplier.
Greater flexibility in matching workloads to hardware.
Potential cost advantages.
Increased resilience against capacity shortages.

However, operating across several hardware and cloud environments also increases complexity.

Different accelerators require distinct software optimizations, deployment procedures, and monitoring systems.

1.5. Google: Full-stack infrastructure

Google has a structurally different position.

Its parent company, Alphabet, operates major cloud infrastructure while Google develops its own TPUs and AI models.

Google's internal ecosystem includes:

Data center facilities.
Custom accelerator development.
Cloud networking.
Model training infrastructure.
Inference platforms.
AI development tools.
Enterprise distribution.

This provides opportunities to optimize computing systems across hardware and software boundaries.

Google still uses NVIDIA accelerators and depends on external manufacturers for parts of its supply chain.

Nevertheless, its ability to design custom chips and operate the surrounding infrastructure is a meaningful competitive advantage.

1.6. xAI: Scaling through NVIDIA infrastructure

xAI's Colossus strategy places significant emphasis on rapid infrastructure deployment.

According to company statements, the original Colossus cluster was assembled in 122 days.

Colossus and Colossus II represent major investments in large-scale AI computing.

The organization has relied heavily on NVIDIA accelerators.

This reduces the need to develop custom hardware before expanding computing capacity.

However, the approach creates dependence on NVIDIA's product availability, pricing, supply chain, and software ecosystem.

The central lesson: Announced computing capacity matters, but operational capacity and its cost matter more.

2. The Second Competitive Weapon: Custom AI Accelerators

NVIDIA remains one of the main beneficiaries of demand for large-scale AI computing.

Its GPU architectures, networking systems, and CUDA software ecosystem have become deeply embedded in AI infrastructure.

But its customers have powerful reasons to develop alternatives.

These include supply diversification, lower inference costs, improved energy efficiency, and greater hardware control.

2.1. Why training and inference require different optimization

AI training and inference have different performance characteristics.

Training

Training large models involves enormous volumes of computation, intensive accelerator communication, and repeated updates to model parameters.

Important considerations include:

Computational throughput.
Memory bandwidth.
Accelerator-to-accelerator communication.
Distributed training efficiency.
Reliability over long-running workloads.

Inference

Inference involves executing a trained model to process user requests.

Important considerations include:

Time to first token.
Output tokens per second.
Concurrent request capacity.
Memory availability for model weights and KV cache.
Latency at different batch sizes.
Cost per request.
Energy efficiency.

A processor optimized for large training jobs may not be the most economical choice for serving interactive applications.

This has encouraged more specialized accelerator architectures.

2.2. Google TPU 8t vs. TPU 8i

In April 2026, Google introduced its eighth-generation TPU family, with separate designs for training and inference.

According to Google's published technical specifications:

Specification	TPU 8t	TPU 8i
Primary purpose	Large-scale training	Inference and reasoning
Peak FP4 performance	12.6 PFLOPS	10.1 PFLOPS
HBM capacity	216 GB	288 GB
HBM bandwidth	6.5 TB/s	8.6 TB/s
On-chip SRAM	128 MB	384 MB
Interconnect topology	3D torus	Boardfly

These are per-chip figures from Google's technical documentation.

TPU 8t emphasizes computing performance for large-scale training.

TPU 8i allocates more resources toward memory and inference-oriented operations.

The 8i architecture also uses a Collectives Acceleration Engine to improve communication performance for supported distributed workloads.

The difference illustrates an important engineering principle.

The fastest training accelerator and the most efficient inference accelerator do not necessarily require the same architecture.

Google has also emphasized software support through JAX, PyTorch, XLA, and specialized kernel development tools.

That software stack is necessary to translate theoretical hardware capability into practical performance.

2.3. AWS Trainium

AWS has developed Trainium as an alternative accelerator platform.

The Trainium family is designed for cloud AI workloads and provides another option for organizations using AWS infrastructure.

Trainium3 became available in late 2025.

The architecture includes 144 GB of HBM3E memory per chip, according to the product information discussed in the original analysis.

AWS benefits from the ability to integrate these accelerators into its cloud infrastructure and offer customers a controlled hardware and software environment.

For large workloads, the economic question becomes whether Trainium can deliver comparable application performance at a lower total cost than alternative accelerators.

2.4. Microsoft Maia

Microsoft is also developing custom AI accelerators.

Maia 200, introduced in early 2026, is part of this effort.

Microsoft's strategy combines its Azure infrastructure, model partnerships, software development, and internally designed hardware.

The objective includes improving inference efficiency and reducing dependence on external accelerator suppliers for suitable workloads.

2.5. OpenAI Jalapeño: Specialized inference hardware

OpenAI and Broadcom unveiled Jalapeño in June 2026.

The accelerator was designed for modern language model inference.

In subsequently published performance measurements, OpenAI reported improvements relative to selected NVIDIA-based comparison systems.

Across three public model workloads, OpenAI reported:

Approximately 1.5 to 1.9 times greater work per watt at peak throughput.
Approximately 1.7 to 3.6 times lower end-to-end latency.
Improvements in interactive inference performance.

The tests included workloads involving GPT-OSS 120B, DeepSeek R1, and Kimi K2.5.

These are OpenAI's reported measurements, not a universal performance guarantee across all workloads.

The relevant performance depends on workload characteristics, batching, sequence length, latency targets, and system configuration.

The strategic significance is broader than an individual benchmark.

By designing inference hardware around its own serving requirements, OpenAI can attempt to optimize the entire processing stack.

2.6. Why custom hardware is difficult

Developing an accelerator requires more than circuit design.

A successful platform needs:

Semiconductor manufacturing capacity.
Memory integration.
Packaging and testing.
Networking.
Compiler support.
Optimized kernels.
Runtime software.
Monitoring and debugging.
Developer adoption.
Production deployment at scale.

Consequently, custom accelerators are more likely to complement NVIDIA infrastructure than replace it immediately across every workload.

3. The Third Competitive Weapon: Cloud Distribution Without Exclusivity

The relationship between AI companies and cloud platforms has changed significantly.

Historically, major AI developers were closely associated with particular infrastructure providers.

OpenAI worked extensively with Microsoft Azure.

Anthropic established AWS as its primary cloud and training partner.

Google developed Gemini within its own broader infrastructure ecosystem.

However, distribution has become more flexible.

3.1. OpenAI expands beyond Azure

In June 2026, OpenAI models and Codex became generally available on Amazon Bedrock.

For AWS customers, this created a route to use OpenAI capabilities through existing cloud infrastructure.

The announcement emphasized enterprise requirements including:

Identity and access management.
Security controls.
Data governance.
Existing billing arrangements.
Deployment workflows.
Cloud monitoring.

The significance is commercial as much as technical.

Customers often prefer to acquire AI capabilities through infrastructure providers they already use.

3.2. Anthropic across the three major clouds

Claude models are distributed through:

Amazon Web Services.
Google Cloud.
Microsoft Foundry.

This gives enterprise customers more deployment choices.

Anthropic benefits from access to customers already operating within those environments.

Cloud platforms benefit from additional model demand and infrastructure consumption.

The arrangement creates an unusual competitive structure.

Cloud providers can invest in an AI company, sell it computing resources, and distribute models from its competitors.

3.3. Why enterprises care about distribution

Consider a multinational financial institution that already uses Azure.

Moving sensitive workloads to an entirely new provider may require extensive reviews of:

Information security.
Compliance.
Identity management.
Data residency.
Network architecture.
Procurement.
Vendor risk.
Business continuity.

Accessing another AI model through its existing cloud may be much easier.

For this reason, enterprise distribution is not simply a marketing channel.

It directly affects whether a model can enter production.

3.4. The cloud platform becomes an AI marketplace

Cloud providers are increasingly becoming marketplaces for multiple AI models.

This creates advantages for customers because they can compare and deploy competing models through a familiar environment.

But it does not completely eliminate vendor lock-in.

Differences may remain in model availability, context limits, API capabilities, caching, tool support, logging, billing, and governance.

3.5. Comparing the main infrastructure strategies

Company	Infrastructure approach	Strategic advantage	Important dependency
OpenAI	Multi-provider contracts and custom silicon	Diversified capacity pipeline	Construction, power delivery, partner execution
Anthropic	AWS, Google TPUs, NVIDIA, additional suppliers	Multiple hardware architectures	External infrastructure providers
Google	Google Cloud, TPUs, NVIDIA, internal systems	Strong full-stack integration	Capital investment and semiconductor supply
xAI	Large NVIDIA-based clusters	Rapid capacity expansion	NVIDIA hardware and large power requirements

These descriptions represent broad strategies, not audited measurements of total operational capacity.

4. The Fourth Competitive Weapon: API Pricing

For most developers, the infrastructure competition becomes visible through API costs.

API prices are determined by multiple factors.

These include hardware acquisition, electricity, model efficiency, caching, serving utilization, and competitive positioning.

4.1. Comparing API prices in October 2026

The following figures represent standard published API prices for shorter-context requests.

All values are in USD per one million tokens.

Model	Input	Cached input	Output
GPT-6 Astra	$10.00	$1.00	$50.00
GPT-6.1 Sol	$2.00	$0.10	$10.00
Claude Fable 5.1	$10.00	$0.25	$50.00
Claude Opus 5.5	$4.00	$0.20	$20.00
Gemini 3.1 Pro Preview	$2.00	$0.20	$12.00
Gemini 3.8 Flash	$0.75	$0.075	$3.75
Grok 4.7	$2.00	$0.50	$6.00

Important pricing conditions

Some providers apply higher rates to longer-context requests.
Faster service tiers may cost more.
Batch processing can be discounted.
Caching terms vary between providers.
Some tools are charged separately.
Data residency may involve additional charges.
Preview and promotional prices may change.

For example, Google states that the standard Gemini 3.8 Flash price will increase on January 1, 2027.

The current input rate of $0.75 per million tokens is scheduled to become $1.50.

The output rate of $3.75 is scheduled to become $7.50.

This is a useful reminder that API pricing is not necessarily permanent.

4.2. The distinction between pricing tiers

Several distinct market segments are emerging.

Premium frontier reasoning

Models such as GPT-6 Astra and Claude Fable 5.1 target demanding workloads where higher per-token costs may be justified by capability.

Balanced intelligence and cost

Models such as GPT-6.1 Sol, Claude Opus 5.5, and competing offerings aim to provide strong performance at lower cost.

High-volume inference

Lower-priced models target tasks where a small cost difference multiplied across millions of requests becomes significant.

These categories are not absolute. A cheaper model may outperform a premium model on particular tasks.

4.3. Why cost per million tokens can be misleading

Consider the basic structure of API spending.

Total model cost = Input cost + Output cost + Cache cost + Tool cost + Retry cost

For a multi-step agent, this becomes more complicated.

An agent may:

Read the user's request.
Inspect a repository.
Call several tools.
Generate a plan.
Modify files.
Run tests.
Diagnose failures.
Retry unsuccessful operations.
Produce a final answer.

Every stage can consume additional tokens and potentially generate external costs.

A model charging less per token can still be more expensive when it requires more reasoning, more attempts, or more human correction.

4.4. Cost per successful task

A better measure is:

Cost per successful task = Total spending on attempts / Number of successfully completed tasks

This incorporates the cost of failures.

For example, a model that costs $0.20 per attempt but succeeds only half the time may be less economical than a model costing $0.30 per attempt with a much higher success rate.

This assumes tasks are comparable and success is measured consistently.

Artificial Analysis has increasingly emphasized cost per task alongside intelligence and response speed.

Its evaluations demonstrate how differences in reasoning token consumption can change the economics of competing models.

4.5. Caching is particularly important for agents

Long-running agents repeatedly process information such as:

System instructions.
Tool definitions.
Conversation history.
Repository context.
Documentation.
Previous results.

Caching can reduce the cost of reprocessing unchanged context.

For example, selected October 2026 cached-input prices include:

GPT-6.1 Sol: $0.10 per million cached tokens.
Claude Opus 5.5: $0.20.
Claude Fable 5.1: $0.25.
Grok 4.7: $0.50.

However, these numbers only apply when requests satisfy each provider's caching conditions.

Cache writes, retention, eligibility, and automatic caching behavior can differ.

Developers should measure effective cache hit rates, not simply compare published cache-read prices.

5. The Fifth Competitive Weapon: Developer Ecosystems

AI models are increasingly embedded into software development environments.

A developer may interact with an AI system through an IDE, terminal, code review platform, deployment workflow, or autonomous coding agent.

This changes the competitive environment.

The model provider is no longer selling only access to an API.

It is also trying to become part of the developer's workflow.

5.1. Coding agents

The four companies offer developer tools with different levels of integration.

Provider	Developer ecosystem
OpenAI	Codex
Anthropic	Claude Code
Google	Gemini CLI and associated development tooling
xAI	Grok Build

These products support workflows beyond isolated question answering.

Capabilities can include:

Repository analysis.
File modification.
Terminal commands.
Tool calling.
Automated debugging.
Multi-step implementation.
Testing assistance.
Development environment integration.

Once a developer becomes familiar with a tool's interface, configuration, and workflow, switching to a competing system can involve additional learning and migration effort.

This creates ecosystem advantages that may last longer than a temporary benchmark lead.

5.2. MCP and interoperability

Anthropic introduced the Model Context Protocol to standardize connections between AI systems and external tools.

The protocol has since gained adoption across multiple AI ecosystems.

In principle, MCP reduces the need to develop completely separate integrations for every model provider.

However, compatibility at the protocol level does not guarantee identical behavior.

Differences can remain in:

Tool permission models.
Authentication.
Context management.
Model reliability.
Error handling.
Supported tool features.
Execution environments.

The value of standardization is therefore real but incomplete.

5.3. Open-weight models

Another distinction is whether models can operate on infrastructure controlled by the developer.

OpenAI has released the gpt-oss family.

Google provides Gemma models.

xAI has also published open-weight Grok releases.

These offerings provide organizations with alternatives to exclusively hosted inference.

Potential use cases include:

Private deployments.
Research.
Specialized applications.
Infrastructure experimentation.
Environments with restrictive data governance requirements.

Self-hosting also introduces operational requirements.

Organizations must manage GPU capacity, inference software, updates, security, and performance tuning.

Anthropic's Claude ecosystem has focused primarily on managed model access rather than offering its flagship models as downloadable open weights.

The important question is not whether open weights are universally better. It is whether the application requires deployment control or benefits more from managed infrastructure.

6. What About the Models Themselves?

Model quality remains important.

However, it has become difficult to identify one universally superior model.

6.1. Benchmark results depend on methodology

Evaluations measure different capabilities.

Some focus on mathematical reasoning.

Others test coding, scientific questions, autonomous agents, professional knowledge work, or human preferences.

The same model may rank differently across different benchmarks.

This is especially important when evaluating reasoning models, because performance can vary with:

Reasoning effort.
Token budgets.
Tool access.
Evaluation harness.
Time limits.
Test contamination.
Model version.

6.2. The early October 2026 benchmark landscape

In September 2026, Artificial Analysis reported that Claude Opus 5.5 reached 58 on its Intelligence Index at maximum reasoning effort.

The original analysis also highlights competing results from GPT-6 Astra, GPT-6.1 Sol, Gemini, and Grok.

These snapshots are useful for observing changes in model capability.

But rankings should not be treated as permanent.

They change when providers release new models or evaluation methodologies are updated.

The distinction between the highest benchmark score and the best production model is particularly important.

A benchmark leader may be slower or more expensive for a given workload.

6.3. Why independent evaluation matters

Before deploying a model in production, teams should construct an evaluation suite that reflects actual requirements.

For a coding agent, this might include:

Correct implementation.
Test success rate.
Regression frequency.
Code maintainability.
Tool execution reliability.
Security.
Completion time.
Total task cost.

For a customer support system, the metrics would be different.

The best model depends on the application.

Model benchmarks are inputs to engineering decisions, not substitutes for those decisions.

7. The Economics of the Entire AI Stack

A frontier AI company faces costs at several levels.

These include:

Model research and training.
Hardware procurement.
Data center construction or leasing.
Electricity.
Networking.
Inference.
Storage.
Engineering.
Security and compliance.
Developer distribution.
Customer acquisition.

Improving efficiency in one layer can influence the economics of the others.

For example, a model that requires fewer inference tokens may reduce accelerator demand.

An improved accelerator can lower electricity consumption.

Better caching can reduce repeated computation.

Cloud distribution can reduce enterprise adoption barriers.

A mature coding ecosystem can increase customer retention.

These advantages reinforce one another.

Why working data centers may become the most durable advantage

A leading model can lose its position following a competitor's release.

API prices can change as hardware and inference efficiency improve.

But large, operational data centers are physical assets.

They depend on land, electricity, construction, cooling, networking, hardware, and maintenance.

Their development cannot be compressed into a model release cycle.

That makes operational infrastructure a potentially more durable advantage.

However, it comes with risk.

If future demand, model efficiency, or hardware economics change, expensive capacity commitments can become difficult to justify.

The same infrastructure that creates competitive advantage can also create significant financial exposure.

8. Strategic Comparison: OpenAI vs. Anthropic vs. Google vs. xAI

Dimension	OpenAI	Anthropic	Google	xAI
Main model ecosystem	GPT	Claude	Gemini	Grok
Key developer tool	Codex	Claude Code	Gemini CLI	Grok Build
Infrastructure strategy	Diversified contracts	Multi-provider computing	Integrated cloud and hardware	Large NVIDIA clusters
Custom accelerator approach	Broadcom partnership	Uses partner accelerators	Develops TPUs	Primarily NVIDIA
Cloud distribution	Multiple cloud channels	AWS, Google, Microsoft	Google Cloud	Own API and selected partner channels
Major advantage	Large infrastructure pipeline	Hardware diversification	Full-stack infrastructure	Rapid cluster deployment
Major challenge	Converting contracts into active capacity	External infrastructure dependence	Large capital requirements	NVIDIA concentration and power demand

This comparison summarizes the companies' approaches rather than establishing a definitive ranking.

9. Practical Recommendations for Developers

Developers cannot control the infrastructure strategies of large AI laboratories.

But they can design applications that remain adaptable.

Recommendation 1: Build a provider abstraction layer

Avoid coupling every application component directly to one model API.

A simple architecture can separate business logic from model-specific integrations.

For example:

Application
    |
    v
Model Router
    |
    +--> OpenAI Adapter
    |
    +--> Anthropic Adapter
    |
    +--> Google Adapter
    |
    +--> xAI Adapter

The router can select models according to task requirements.

However, portability requires more than exchanging endpoint URLs.

Different providers may handle tool calls, reasoning, response formatting, and context differently.

Recommendation 2: Compare successful task costs

Evaluate actual workloads rather than relying exclusively on published token prices.

Record:

Total tokens consumed.
Cached token usage.
Tool execution costs.
Success rate.
Number of retries.
Average latency.
Human correction required.

These measurements are more useful than price per million tokens alone.

Recommendation 3: Use the right model for each task

Not every request requires a frontier reasoning model.

Smaller models may be sufficient for:

Classification.
Simple extraction.
Formatting.
Routing.
Basic text transformations.
Lightweight automation.

More capable models may be appropriate for:

Complex reasoning.
Long-horizon coding.
Multi-step research.
Difficult debugging.
Technical planning.
High-stakes professional work requiring careful verification.

Routing tasks to appropriate models can substantially improve economics.

Recommendation 4: Avoid unnecessary vendor-specific dependencies

Certain provider-specific features can be valuable.

Use them when they deliver clear benefits.

But consider whether those benefits justify higher switching costs.

Prefer interoperable interfaces when equivalent functionality is available.

Recommendation 5: Reevaluate regularly

The AI market changes quickly.

Models improve.

Prices change.

New service tiers appear.

Hardware becomes more efficient.

A model selection that was optimal three months ago may no longer be optimal today.

Build ongoing evaluation into your engineering process.

10. What Will Define the Next Phase of Competition?

Several factors will shape the industry over the coming years.

Computing capacity

Which organizations can bring contracted infrastructure into production efficiently?

Electricity availability

Can data center power supply keep pace with computing demand?

Accelerator diversification

Will custom chips meaningfully reduce dependence on NVIDIA?

Inference efficiency

Can providers reduce the cost of serving increasingly capable models?

Cloud distribution

Will enterprise customers continue consolidating model access through existing cloud platforms?

Developer ecosystems

Which tools become deeply integrated into professional workflows?

Model portability

Will standards and open-weight alternatives meaningfully reduce switching costs?

The companies that succeed will probably be those that combine several advantages rather than relying on one.

Conclusion: The AI Race Has Moved Beyond Intelligence Alone

The competition between OpenAI, Anthropic, Google, and xAI is increasingly a competition between computing ecosystems.

The model remains essential.

But intelligence cannot reach users without infrastructure.

OpenAI is pursuing large-scale computing commitments, hardware diversification, and custom inference processors.

Anthropic is building access to several accelerator ecosystems while expanding cloud distribution.

Google combines frontier AI development with custom TPUs and an established global cloud infrastructure.

xAI is investing in large NVIDIA-based clusters and rapid infrastructure expansion.

Each approach has strengths and weaknesses.

The most enduring advantage may belong to the company that can turn available electricity and computing hardware into useful AI output most efficiently.

For developers, the conclusion is straightforward.

Don't select models based exclusively on rankings.

Don't evaluate costs based exclusively on token prices.

And don't assume that the best provider today will remain the best provider tomorrow.

Design applications for flexibility, compare real workloads, and measure results.

Because in the next stage of AI competition, the winners will not necessarily be the companies with the most impressive models. They will be the companies that can deliver those models reliably, affordably, and at scale.

References and Further Reading

Computing infrastructure

AI hardware

Cloud distribution

API pricing

Benchmarks and model economics

From the Space Race to the AI Chip Race: Inside the US and China's Battle for Computing Power

Aeza — Mon, 12 Oct 2026 14:25:56 +0000

A technical analysis of semiconductor manufacturing, EUV lithography, HBM, advanced packaging, accelerator architectures, interconnect technologies, software ecosystems, and the economics of large-scale AI infrastructure.

Updated: October 2026

Introduction: The Next Technological Race Is Almost Invisible

During the Space Race, technological progress was relatively easy to recognize.

The Soviet Union launched Sputnik in 1957 and sent Yuri Gagarin into orbit in 1961. The United States landed astronauts on the Moon in 1969.

Each achievement provided a visible demonstration of scientific and engineering capability.

Today's competition for leadership in artificial intelligence infrastructure is less visible.

There are no rockets to watch.

Instead, progress is measured through transistor density, semiconductor yields, memory bandwidth, advanced packaging capacity, optical interconnects, and the number of accelerators that can operate together efficiently.

But one similarity remains.

The competition is pushing entire technology ecosystems toward faster innovation and massive industrial investment.

The United States and its international technology partners lead in several critical components of advanced computing infrastructure.

China is investing in alternative manufacturing capabilities, domestic AI accelerators, memory technology, and large-scale system architectures.

The central question is no longer simply who can build the fastest chip.

It is which ecosystem can design, manufacture, package, connect, and operate enough computing hardware to support the next generation of AI models.

To understand this competition, we need to examine the entire infrastructure chain.

1. Semiconductor Manufacturing: The Foundation of AI Computing

Every modern AI accelerator begins with semiconductor manufacturing.

GPU and AI accelerator designers create increasingly complex architectures, but those designs must ultimately be fabricated using advanced industrial processes.

This is one of the areas where access to technology differs most significantly.

1.1. TSMC and the leading-edge manufacturing ecosystem

Taiwan Semiconductor Manufacturing Company, or TSMC, produces advanced semiconductors for customers including NVIDIA, AMD, and other leading chip designers.

TSMC's 2 nm process entered high-volume manufacturing in the fourth quarter of 2025.

During the same quarter:

3 nm accounted for 28% of wafer revenue.
5 nm accounted for 35%.
7 nm accounted for 14%.

In July 2026, TSMC increased its planned annual capital expenditure to $60 billion to $64 billion.

A significant portion of that investment supports advanced manufacturing technologies, while other spending is directed toward specialty processes, packaging, testing, and related operations.

These numbers illustrate the fundamental economics of advanced semiconductors.

Leading-edge manufacturing requires continuous investment in equipment, facilities, process development, and production capacity.

TSMC is headquartered in Taiwan and operates an expanding international manufacturing network, including facilities in Arizona.

However, diversification does not immediately eliminate the concentration of the most advanced semiconductor production in Taiwan.

1.2. SMIC and China's domestic manufacturing progress

China's leading contract semiconductor manufacturer, SMIC, has been developing advanced processes without access to the same equipment ecosystem.

Huawei's Ascend 910B has been associated with SMIC's N+2 process, commonly classified as a 7 nm class technology.

In December 2025, TechInsights published a physical analysis of Huawei's Kirin 9030 processor.

Its findings confirmed that SMIC had progressed to N+3.

This represents an improvement over its previous manufacturing generation.

However, process names should be interpreted carefully.

Terms such as 2 nm, 3 nm, 5 nm, and 7 nm are commercial technology-node labels rather than direct measurements of one physical transistor dimension.

Different manufacturers' node names do not necessarily imply identical transistor density, performance, or energy efficiency.

A meaningful comparison requires examining actual circuit density, power consumption, transistor performance, manufacturing yields, and production cost.

TechInsights characterized N+3 as an important step forward but not equivalent to the leading commercial manufacturing technologies available from TSMC.

1.3. EUV lithography: Why manufacturing equipment matters

Extreme ultraviolet lithography is one of the most important technologies used in leading-edge semiconductor production.

EUV systems use light with a wavelength of 13.5 nm to pattern extremely small features on semiconductor wafers.

These machines are produced by ASML, based in the Netherlands.

They incorporate highly sophisticated optical systems, light sources, vacuum technology, precision positioning, and control systems.

Export restrictions have prevented the sale of ASML EUV systems to Chinese manufacturers.

Without EUV, advanced semiconductor production increasingly depends on techniques involving deep ultraviolet lithography and multiple patterning.

1.4. Why multiple patterning is expensive

Multiple patterning divides a complex circuit layer into multiple exposure and processing steps.

This can extend the capabilities of older lithography technologies.

However, additional steps can introduce:

Longer manufacturing cycles.
Greater equipment utilization.
More materials and consumables.
Increased alignment requirements.
Additional opportunities for defects.
Reduced manufacturing yields.

The result is a complex economic problem.

Even if a manufacturer can produce a functioning advanced chip, commercial competitiveness depends on the number of usable chips produced per wafer and the total manufacturing cost.

1.5. Domestic EUV development

Reuters reported in December 2025 that China had developed an experimental EUV system in Shenzhen.

According to that reporting, the prototype could generate EUV radiation but had not demonstrated commercial chip production.

This is a significant distinction.

Building an experimental light source is one milestone.

Manufacturing high-performance semiconductors reliably at industrial scale is another.

Production equipment must meet demanding requirements for overlay accuracy, throughput, optical performance, process control, maintenance, and uptime.

A working prototype demonstrates technological progress. Commercial yields and sustained throughput determine industrial competitiveness.

2. High-Bandwidth Memory: Why Compute Needs Data

Modern AI accelerators are designed to perform enormous numbers of mathematical operations.

However, their performance depends on keeping processing units supplied with data.

Without sufficient memory capacity and bandwidth, processing resources can spend significant time waiting.

High-bandwidth memory addresses this problem.

2.1. How HBM works

HBM integrates multiple DRAM dies into vertically stacked structures.

Through-silicon vias and high-density interfaces help connect the layers, enabling a very wide memory interface.

The memory stacks are placed close to the compute dies through advanced packaging technologies.

The objectives are:

High memory bandwidth.
Improved energy efficiency per transferred bit.
Greater memory density near the accelerator.
Reduced communication distance compared with conventional external memory systems.

For AI workloads, HBM has become a critical component rather than an optional performance enhancement.

2.2. The global HBM market

According to Counterpoint Research, revenue market shares in Q2 2026 were:

Company	HBM revenue share
SK hynix	50%
Samsung	33%
Micron	18%

Figures are rounded and may not total exactly 100%.

At that point, HBM3E still represented most industry HBM revenue, while HBM4 shipments were beginning to become more significant.

These three companies form the core of the advanced HBM supply chain.

For Chinese accelerator manufacturers, reliance on memory suppliers outside mainland China creates additional complexity under export controls.

2.3. Export restrictions and domestic HBM development

In December 2024, the US Bureau of Industry and Security introduced additional semiconductor export controls.

The measures covered advanced manufacturing equipment, certain semiconductor-related software, and specified categories of high-bandwidth memory.

They also added 140 entities to the Entity List.

This increased the strategic importance of domestic HBM development for Chinese manufacturers.

CXMT has become one of the most important companies in China's memory industry.

Its expansion into advanced DRAM and HBM technologies represents a potential alternative supply route.

However, memory self-sufficiency requires far more than a technically functioning design.

Manufacturers must also establish:

Reliable wafer production.
Advanced DRAM process technology.
Vertical stacking capabilities.
Thermal and mechanical integration.
Testing and quality control.
Packaging compatibility.
Consistent delivery volumes.

A production bottleneck in any of these areas can limit the number of completed accelerators.

2.4. Comparing accelerator memory specifications

Published accelerator specifications illustrate how important memory technology has become.

Accelerator	HBM capacity per accelerator	Peak memory bandwidth
NVIDIA B200	180 GB	Approximately 8 TB/s
AMD MI355X	288 GB HBM3E	8 TB/s
Huawei Ascend 950 generation	144 GB HiZQ 2.0	4 TB/s
NVIDIA Rubin	288 GB HBM4	22 TB/s

These are vendor-published hardware specifications. Availability, implementation, and achieved application performance vary. Rubin and certain Ascend configurations represent newer announced product generations.

The differences are substantial.

But memory bandwidth alone does not establish which accelerator is faster for a real application.

Actual performance also depends on the processor architecture, model structure, precision, memory-access patterns, kernels, and software optimizations.

It is equally important to distinguish memory specifications per accelerator from those per individual HBM stack.

Comparisons must use equivalent units and system boundaries.

3. Advanced Packaging: The Manufacturing Stage Most People Overlook

A modern AI accelerator package may combine multiple compute dies and several HBM stacks.

Connecting these components requires extremely dense and reliable electrical interfaces.

This is the domain of advanced semiconductor packaging.

3.1. Why packaging is now a strategic technology

Traditional semiconductor packaging was primarily associated with protecting chips and connecting them to circuit boards.

Modern advanced packaging performs a much more demanding role.

It enables:

High-density chip-to-chip connections.
Integration of compute dies and HBM.
Large interposers and complex substrates.
High aggregate memory bandwidth.
Multi-die accelerator architectures.

These capabilities are essential to many leading AI accelerator designs.

3.2. TSMC's CoWoS technology

TSMC's Chip-on-Wafer-on-Substrate technology, commonly known as CoWoS, is a major enabler of modern AI computing packages.

CoWoS allows compute components and memory stacks to be integrated within a high-performance package.

But this technology also creates manufacturing dependencies.

Even with enough compute dies and HBM memory, a shortage of advanced packaging capacity can prevent those components from becoming completed accelerator products.

As demand grows, packaging availability can become a limiting factor in the entire supply chain.

TSMC has been expanding its advanced packaging capacity, and its leadership acknowledged capacity pressure during 2026.

3.3. China's advanced packaging industry

Chinese companies including SJ Semiconductor, JCET, Tongfu, and Huatian are developing and expanding advanced packaging technologies.

Their challenge involves not only increasing capacity but also supporting increasingly complex accelerator packages.

This requires specialized equipment, high-density substrates, thermal management, process integration, and dependable yields.

The strategic lesson is that semiconductor leadership depends on manufacturing completed systems, not simply producing silicon dies.

4. AI Accelerator Architectures: Comparing the Building Blocks

Accelerator specifications often emphasize peak floating-point performance.

However, FLOPS comparisons can be misleading if they combine different numerical formats or sparsity assumptions.

4.1. FP8, BF16, and FP4 are not interchangeable

Modern AI processors support different numerical formats.

BF16

A 16-bit floating-point format commonly used for model training.

FP8

An 8-bit floating-point format designed to reduce memory and compute requirements for supported workloads.

FP4

A 4-bit format used in increasingly aggressive quantization and low-precision computing.

Hardware specifications may also include performance with structured sparsity.

This can increase theoretical throughput under qualifying conditions.

A comparison between BF16 throughput on one accelerator and FP8 throughput on another is not a direct performance comparison.

Likewise, sparse and dense FLOPS should not be mixed without explanation.

4.2. Selected AI accelerator specifications

Accelerator	Memory	Memory bandwidth	Performance reference
NVIDIA B200	180 GB	About 8 TB/s	About 4.5 PFLOPS dense FP8
AMD MI355X	288 GB	8 TB/s	5 PFLOPS dense FP8
Huawei Ascend 910C	128 GB	About 3.2 TB/s	Estimated BF16 performance, not directly comparable to FP8
Huawei Ascend 950DT	144 GB	4 TB/s	Huawei-announced next-generation specifications
NVIDIA Rubin	288 GB	22 TB/s	New architecture with support for lower-precision formats

Figures describe selected vendor specifications and independent estimates. They are not a unified benchmark dataset. Performance comparisons require matched precision, sparsity, workload, and deployment configurations.

The important observation is not simply which accelerator has the largest number.

It is that accelerator performance depends on the combination of compute throughput, memory bandwidth, connectivity, and software.

5. China's Alternative Strategy: Scaling at the System Level

China's semiconductor manufacturing constraints have contributed to a distinct approach to AI infrastructure.

If individual accelerators cannot immediately match the strongest available competitors, one alternative is to connect more of them.

Huawei's large-scale accelerator systems demonstrate this strategy.

5.1. CloudMatrix 384

Huawei's CloudMatrix 384 uses 384 Ascend 910C processors.

SemiAnalysis compared the design with NVIDIA's GB200 NVL72 system.

The comparison found several notable differences.

Metric	CloudMatrix 384 relative to GB200 NVL72
Accelerator count	384 vs. 72
Aggregate dense BF16 compute	Nearly 2 times higher
Aggregate memory capacity	About 3.6 times higher
Aggregate memory bandwidth	About 2.1 times higher
System power consumption	About 4.1 times higher
Performance per watt	Significantly lower

These are SemiAnalysis estimates from its April 2025 system comparison. They are not universal application-level benchmark results.

This illustrates a fundamental engineering trade-off.

The system uses more accelerators and a larger interconnect network to provide high aggregate computing capability.

But the energy cost rises substantially.

5.2. Why interconnect architecture matters

Large-scale AI training frequently requires accelerators to exchange parameters, gradients, activations, and other data.

Network communication can become a significant fraction of total execution time.

An accelerator that waits for data from another processor is not fully using its computing capacity.

Consequently, system-level performance depends on:

Interconnect bandwidth.
Communication latency.
Network topology.
Collective communication efficiency.
Memory accessibility.
Congestion behavior.
Hardware and software coordination.

Huawei's UnifiedBus is designed to improve communication across large accelerator systems.

The objective is to make multiple physical processors operate as parts of a more tightly integrated computing environment.

5.3. The energy trade-off

More accelerators require more electrical power.

They also create additional demands for cooling, racks, networking hardware, and maintenance.

System efficiency therefore becomes increasingly important.

In the CloudMatrix comparison, the larger aggregate computing capability came with substantially higher total power consumption.

This may be acceptable in environments where large amounts of electricity are available at relatively low cost.

In power-constrained locations, however, infrastructure efficiency becomes more important.

The useful metric is not simply total peak FLOPS. It is useful computing output relative to electricity, cost, and operational complexity.

6. The Next Generation: UnifiedBus, Atlas SuperPoDs, and Optical Interconnects

Huawei has continued expanding its system-level strategy.

Its September 2025 roadmap outlined ambitious configurations for future Atlas systems.

The Atlas 950 SuperPoD roadmap described configurations reaching 8,192 processors.

A larger Atlas 960 roadmap described a system reaching 15,488 processors.

These figures were roadmap targets, not evidence that every configuration had already entered large-scale commercial deployment.

6.1. The September 2026 update

At HUAWEI CONNECT 2026, Huawei introduced the Atlas 960E SuperPoD.

Its announced architecture supports:

Up to 4,096 AI processors.
8 EFLOPS of FP8 computing capability.
16 EFLOPS of FP4 computing capability.
Up to 1 PB of HBM.
UnifiedBus-based interconnection.
Near-packaged optical modules.

These figures are theoretical manufacturer specifications.

Actual performance depends on the implemented system, workload, software, and operating conditions.

6.2. Hi-ONE and near-packaged optics

One of the most technically interesting developments is Huawei's Hi-ONE optical technology.

The company uses near-packaged optics, or NPO, to improve interconnection density and efficiency.

Traditional large-scale optical networks rely heavily on pluggable optical transceivers.

As clusters grow, these modules consume electrical power, occupy physical space, and introduce additional complexity.

Huawei states that the Atlas 960E design uses approximately 5,500 Hi-ONE units rather than around 48,000 conventional 800G optical modules.

The company claims this reduces interconnect power consumption by more than 550 kW for the compared configuration.

These are vendor-provided estimates and should be interpreted within the scope of the stated comparison.

Nevertheless, the engineering direction is important.

Reducing networking overhead can improve the practical economics of very large AI systems.

6.3. Updated accelerator roadmap

In September 2026, Huawei announced that development of its Ascend 960 processors was progressing ahead of schedule.

The updated plan includes:

Ascend 960DT availability in Q1 2027.
Ascend 960PR availability in Q3 2027.
Additional Ascend generations planned for 2028 and 2029.

This demonstrates continued investment in individual processor capabilities alongside system architecture.

However, roadmap specifications must be distinguished from production shipments and independent benchmarks.

7. NVIDIA's Platform Strategy: Beyond the Individual GPU

Huawei is not alone in shifting attention toward system-level computing.

NVIDIA has increasingly presented its AI infrastructure as an integrated computing platform.

Its architectures combine processors, memory, high-speed interconnects, networking equipment, and software.

7.1. Blackwell and rack-scale systems

NVIDIA's Blackwell generation emphasized multi-GPU computing, high aggregate memory bandwidth, and tightly connected accelerator systems.

The GB200 NVL72 design connects 72 GPUs within a rack-scale configuration.

This enables large AI workloads to use a substantial pool of accelerator resources.

System design is especially important for large-model training and inference, where communication overhead can limit scaling.

7.2. Rubin and NVLink 6

NVIDIA's Rubin generation further develops this approach.

Its published specifications include:

288 GB HBM4 per GPU.
Up to 22 TB/s of HBM bandwidth.
NVLink 6 with 3.6 TB/s of scale-up bandwidth.
Integration with the wider Vera Rubin platform.

These capabilities target the increasing demands of large-scale AI training and inference.

NVIDIA is attempting to improve the efficiency of the complete computing platform rather than treating every GPU as an isolated component.

7.3. Why hardware integration matters

A large AI computing system contains several sources of overhead:

Data movement between accelerators.
Memory access.
Synchronization.
Network communication.
Power delivery.
Cooling.
Distributed workload scheduling.

Optimizing individual processors while ignoring these costs can limit real system performance.

The strongest infrastructure designs address these constraints together.

8. Software Ecosystems: CUDA, ROCm, and CANN

Hardware performance is only useful when applications can access it efficiently.

This is where software becomes a strategic advantage.

8.1. NVIDIA CUDA

CUDA is one of NVIDIA's most important strengths.

Over many years, developers have built software around the platform.

The ecosystem includes:

Optimized computational libraries.
Machine learning framework integrations.
Compiler and kernel development tools.
Distributed training support.
Debugging and profiling utilities.
Broad deployment experience across AI infrastructure.

This creates an advantage that cannot be captured in a simple accelerator specification table.

8.2. AMD ROCm

AMD is also investing in accelerator software through ROCm.

Its objective is to provide tools, libraries, and framework support for AMD GPU computing.

ROCm is an important component of AMD's effort to compete in large-scale AI infrastructure.

As with any accelerator ecosystem, performance depends on workload support, maturity, and optimization.

8.3. Huawei CANN

Huawei is developing its CANN software ecosystem for Ascend accelerators.

In September 2026, Huawei reported that Ascend supported more than 90 third-party open-source projects, including PyTorch, Triton, and vLLM.

The company is also pursuing more open development of its software stack.

This can reduce barriers to adoption.

However, framework compatibility should not be confused with complete performance parity.

A workload may run correctly while still requiring substantial optimization.

8.4. What developers actually need

Before selecting an accelerator platform, engineering teams should evaluate:

Framework compatibility.
Kernel performance.
Compiler maturity.
Distributed training support.
Available debugging tools.
Memory management behavior.
Infrastructure orchestration.
Vendor support and ecosystem activity.

The real performance of an accelerator is determined by hardware and software working together.

9. Export Controls and Industrial Policy

The semiconductor supply chain has become increasingly influenced by trade restrictions and industrial policy.

The US introduced major advanced computing and manufacturing export controls in October 2022.

Additional measures followed in subsequent years.

Selected developments

Period	Development
October 2022	Major US restrictions on advanced computing chips and semiconductor manufacturing technologies
December 2024	Expanded US controls covering specified HBM, equipment, software, and additional entities
April 2025	Additional licensing restrictions affected selected NVIDIA AI accelerators
December 2025	Reuters reported development of an experimental domestic EUV prototype in China
2026	Limited licensed shipments of NVIDIA H200 hardware to China were reported
September 2026	Huawei announced an updated Ascend roadmap and Atlas 960E architecture

These developments have created a complex environment for manufacturers, suppliers, and customers.

The United States has used export licensing as one means of restricting access to selected advanced technologies.

China has increased investment in domestic alternatives while also using trade and regulatory measures affecting the technology supply chain.

Neither side operates independently of the rest of the world.

9.1. The international nature of semiconductor manufacturing

A modern AI accelerator can depend on contributions from several countries.

For example:

US companies design accelerators.
Taiwanese foundries manufacture leading-edge dies.
Dutch lithography systems enable advanced fabrication.
South Korean and US memory suppliers produce HBM.
Japanese companies supply important equipment and materials.
Packaging operations may span several countries.

This specialization supports technological progress but creates supply-chain concentration.

9.2. Semiconductor equipment spending

According to SEMI, semiconductor manufacturing equipment spending in China reached approximately $49.3 billion in 2025.

This was higher than the amount reported for any other individual region.

Large equipment spending does not automatically mean access to the most advanced manufacturing technology.

Much of the investment also supports mature-node production and other manufacturing capabilities.

Nevertheless, it demonstrates the scale of China's effort to expand its domestic semiconductor industry.

10. Comparing the Two Technology Ecosystems

A useful comparison must look beyond individual chip specifications.

Technology layer	US and international partner ecosystem	China's domestic ecosystem
Advanced manufacturing	TSMC high-volume 2 nm production	SMIC N+3 development
EUV lithography	Commercial ASML EUV equipment	Experimental domestic development
Advanced HBM	SK hynix, Samsung, Micron	CXMT and domestic alternatives
Advanced packaging	CoWoS and other established solutions	Expanding domestic 2.5D packaging
AI accelerators	NVIDIA, AMD, and other designers	Huawei Ascend and other local developers
Interconnects	NVLink, Infinity Fabric, networking platforms	UnifiedBus and optical interconnect development
Large computing systems	Blackwell and Rubin platforms	CloudMatrix and Atlas SuperPoDs
Software ecosystems	CUDA, ROCm, and supporting frameworks	CANN and expanding open-source integration
Key constraints	Advanced packaging, infrastructure power, geographic concentration	Leading-edge lithography, HBM supply, manufacturing yields, software maturity

This table is a simplified picture.

The ecosystem is highly international, and the capabilities described are not necessarily available at equivalent production volumes.

11. Five Metrics That Will Matter More Than Peak FLOPS

Peak theoretical FLOPS remain useful for understanding processor capabilities.

But they are insufficient for evaluating large AI infrastructure.

11.1. Useful throughput

How many training or inference operations can the system complete under realistic workloads?

For inference, this may include tokens generated per second under specified latency constraints.

For training, it may include time to complete a training run or processing throughput for a defined model.

11.2. Performance per watt

How much useful computation does the system deliver for the electricity consumed?

This matters as data centers face constraints in power availability and cooling.

11.3. Memory efficiency

Does the system have sufficient memory capacity and bandwidth to support its workloads?

A processor with high theoretical computing throughput can be underutilized when memory access becomes the bottleneck.

11.4. Communication efficiency

How effectively can processors exchange data?

A large cluster with poor communication efficiency may deliver disappointing scaling even when aggregate theoretical compute is enormous.

11.5. Total cost of ownership

The cost of operating AI infrastructure includes far more than the purchase price of accelerators.

A proper analysis should account for:

Hardware acquisition.
Electricity.
Cooling.
Networking.
Facility requirements.
Software development.
Maintenance.
Reliability.
Utilization.
Replacement cycles.

The winning architecture is not necessarily the one with the highest theoretical performance. It is the one that delivers the required work reliably and economically.

12. What Happens Next?

Several developments will shape the next phase of competition.

Manufacturing

Can China improve domestic advanced-node production yields and develop commercially competitive lithography equipment?

Can international suppliers expand leading-edge capacity while reducing geographic concentration?

Memory

Will Chinese manufacturers be able to produce advanced HBM in sufficient volumes?

Can global memory suppliers expand output fast enough to match accelerator demand?

Packaging

Will advanced packaging capacity grow quickly enough to avoid limiting finished accelerator supply?

Interconnects

How much can optical technologies and improved network architectures reduce communication overhead?

Software

Can alternative accelerator ecosystems achieve the maturity needed to support demanding production workloads?

Energy

Will data center electricity availability become a more important constraint than raw processor production?

These questions are closely connected.

Progress in one area can expose bottlenecks in another.

Conclusion: Two Paths Toward the Same Computing Problem

The AI chip race is becoming a competition between industrial systems.

The US and its international partners possess important advantages in advanced semiconductor manufacturing, HBM supply, accelerator design, and software maturity.

China is developing its own manufacturing capabilities, memory supply, accelerator designs, packaging, and software.

It is also pursuing a system-level strategy that uses larger numbers of interconnected processors to compensate for some individual hardware limitations.

That strategy has demonstrated substantial aggregate computing potential.

But it also creates trade-offs in electricity consumption, cooling, networking complexity, and infrastructure cost.

The most important lesson is that the future of AI computing cannot be understood through one GPU benchmark.

Compute performance depends on an entire chain, from lithography and manufacturing yields to memory bandwidth, packaging, interconnects, software, and power availability.

During the Space Race, technological competition was measured through launch vehicles and missions.

In the AI chip race, the outcome may be measured through a much less visible metric.

How efficiently can an entire computing ecosystem convert electricity and semiconductor technology into useful AI performance?

That may ultimately matter more than which manufacturer produces the fastest individual chip.

References and Further Reading

Semiconductor manufacturing and lithography

Memory and AI accelerator specifications

Counterpoint: Global DRAM and HBM Market Share
[AMD: Instinct MI355X Specifications](https://www.amd.com/en/products/accelerators/instinct/mi350

PostgreSQL Monitoring Is Broken: Too Many Metrics, Not Enough Signals

Aeza — Mon, 12 Oct 2026 14:12:13 +0000

A practical guide to actionable PostgreSQL alerts, latency percentiles, locks, PgBouncer, replication lag, WAL retention, and transaction ID wraparound.

The incident every on-call engineer recognizes

It's Friday evening at a streaming platform. A highly anticipated episode has just dropped. Twenty minutes later, customers begin contacting support: search takes ten seconds, the Watch button spins indefinitely, and some viewers cannot find the new episode in the catalog.

The on-call engineer opens Grafana.

Everything is green.

PostgreSQL CPU utilization is around 33%.
The disk is only half full.
Health checks are passing.
The replica is connected.
There are roughly 300 dashboard panels, and none has raised an alert.

The graphs are not lying. They are simply answering the wrong question. The infrastructure looks healthy, but customers are waiting.

Our argument is that database monitoring is frequently designed around what is easy to measure rather than what an engineer can act on during an incident. The problem extends beyond PostgreSQL. In a February 2026 NeuBird survey of 1,039 SRE, DevOps, and operations professionals, 78% reported having experienced an incident that triggered no alert, and 57% said fewer than 30% of their alerts were actionable. This is a vendor-sponsored survey, not a neutral industry census, but it illustrates a familiar problem: too many notifications, too little useful signal.

So what should trigger a page? And what should remain on a dashboard until an engineer needs to investigate?

1. Three failures that can happen while CPU is idle

Scenario A: A migration creates a lock queue

At 8:55 p.m., someone deploys an ALTER TABLE migration to add a column to the catalog table. The operation needs a strong table lock. But a long-running analytical report is still reading that table.

The migration waits for the report. New queries, including ordinary SELECT statements from the search service, can queue behind the waiting migration. They are not performing expensive computation. They are waiting for locks.

Customer symptom: Search and playback-related operations become slow.

What the CPU dashboard shows: Plenty of free CPU.

Scenario B: PgBouncer has exhausted its server connections

The application connects through PgBouncer. All 50 server connections are occupied by slow queries or transactions that have been left open. New client requests wait in PgBouncer before reaching PostgreSQL.

Customer symptom: Requests stall while acquiring a database connection.

What PostgreSQL may show: Modest CPU usage and no dramatic increase in active work.

Scenario C: A read replica is stale

The catalog reads from a replica, but replay has fallen behind. The new episode exists on the primary and has not yet appeared on the read replica.

Customer symptom: Users see outdated content even though requests complete successfully.

What ordinary health checks show: The database responds, and nothing looks obviously broken.

All three cases are diagnosable. The harder question is whether you should page on database internals, user-visible symptoms, or both.

2. Two competing alerting philosophies

Approach A: Page on database state

This approach assumes that PostgreSQL often provides warning signs before users notice a problem. Examples include:

Connection counts approaching a limit.
Replication lag increasing.
Old transactions remaining open.
WAL files accumulating because a replication slot is inactive.
Free disk capacity falling.

The advantage is early warning. The weakness is poor threshold design.

An alert such as "connections exceed 80%" may indicate routine operation for one database and an imminent incident for another. Reusing generic thresholds across every cluster produces noise, and people eventually learn to ignore it.

The stronger version of this approach uses database-specific baselines and rates of change. For example: "This replication slot is retaining 40 GB of WAL, increasing by 5 GB per hour." That describes a concrete failure mode and gives the operator useful context.

Approach B: Page on user-visible symptoms

This approach follows the reasoning in Google's Site Reliability Engineering book: monitoring should distinguish what is broken from why it is broken.

Google's four golden signals are latency, traffic, errors, and saturation. For a PostgreSQL-backed service, the most important paging signals might be:

p99 latency of critical user operations.
Error rate of those operations.
Time spent waiting for a database connection.
Failed or stale reads that directly affect users.

Locks, checkpoints, disk statistics, and internal session counts are then diagnostic signals. Engineers open them to explain a symptom instead of receiving a page for every fluctuation.

This approach reduces noise, but it has limits. Instrumenting every service takes work, and some dangerous database conditions grow silently for days or weeks. A symptom-only policy could discover an exhausted resource only when PostgreSQL begins rejecting writes.

Our position: Page on user impact, with a small and deliberate set of predictive alerts for irreversible or rapidly escalating database risks.

3. Why averages and even p95 can hide an incident

Imagine 1,000 requests to the Watch operation during one minute:

980 complete in 20 ms.
20 wait on a lock and complete in 3 seconds.

The average is:

(980 × 20 ms + 20 × 3000 ms) / 1000 = 79.6 ms

That appears healthy against a 200 ms average-latency threshold.

The p95 is also 20 ms. But p99 is approximately 3 seconds. Two percent of viewers are experiencing a three-second delay.

This is why latency distributions matter. But they must be interpreted correctly:

Measure each critical operation independently. Fast search requests can conceal slow playback operations in an aggregate metric.
Display request counts beside percentiles. Percentiles calculated from five requests per hour are not particularly informative.
Separate connection acquisition from query execution. A slow request may never have reached PostgreSQL.
Use stable histograms. Application-side Prometheus histograms are a common way to aggregate and analyze request latency.

Do not expect pg_stat_statements to provide p95 or p99 per query. It tracks counts and timing statistics such as mean, minimum, maximum, and variability, but not those latency percentiles. Instrument the application or relevant proxy layer for request-level distributions.

4. Lock contention: Find the blocker, not just the blocked

PostgreSQL uses locks constantly. An alert saying "locks exist" is therefore not useful.

You need answers to three questions:

Which sessions are waiting?
How long have they waited?
Which sessions are blocking them?

You can investigate using pg_stat_activity and pg_blocking_pids():

-- Find sessions waiting on locks or left idle in a transaction.
SELECT
    pid,
    now() - xact_start AS xact_age,
    now() - state_change AS in_state,
    state,
    wait_event_type,
    pg_blocking_pids(pid) AS blocked_by,
    left(query, 60) AS query
FROM pg_stat_activity
WHERE wait_event_type = 'Lock'
   OR state LIKE 'idle in transaction%'
ORDER BY xact_start;

Pay special attention to idle in transaction. A query may have finished, but the surrounding transaction remains open while the application waits for an external service. That session can retain locks and interfere with cleanup of old row versions.

Two controls are worth understanding:

idle_in_transaction_session_timeout closes sessions that remain idle within an open transaction too long.
transaction_timeout, available starting with PostgreSQL 17, limits total transaction duration. Apply carefully, preferably to suitable application roles, because it can also interrupt legitimate long-running work.

Make schema migrations fail quickly instead of blocking traffic

For a migration requiring a strong lock, a short lock_timeout can prevent the migration from sitting in a queue indefinitely:

BEGIN;
SET LOCAL lock_timeout = '2s';
ALTER TABLE titles ADD COLUMN age_rating text;
COMMIT;

If the lock cannot be acquired within two seconds, the operation fails and the transaction must be retried deliberately. PostgreSQL does not automatically retry the migration. The point is to reduce the risk of a waiting schema change obstructing normal traffic. For more context, see Xata's explanation of migrations and exclusive locks.

With state-based monitoring, an unusually old transaction might produce an early warning, though normal reports can also be long-running. With symptom-based monitoring, an increase in playback p99 would page the on-call engineer, who could then inspect the lock queue.

log_lock_waits is another diagnostic option. As of the PostgreSQL 19 beta series, PostgreSQL 19 also proposes expanded lock observability through pg_stat_lock; version-specific behavior should be confirmed against the PostgreSQL 19 release notes.

5. PgBouncer: When the queue is outside PostgreSQL

Database connection counts are frequently compared with max_connections. That is a useful capacity metric, but it does not capture the entire request path. PostgreSQL also reserves connection slots for privileged access, so ordinary clients may fail before total occupancy reaches 100%.

If PgBouncer sits in front of the database, inspect its own queue:

SHOW POOLS;

Two fields are particularly important in the PgBouncer documentation:

cl_waiting: Clients waiting for an available server connection.
maxwait: How long the oldest waiting client has waited, measured in seconds.

A pool that is 80% occupied but has no waiting clients may be healthy. A growing maxwait can indicate real user-facing delay even if the database is only moderately busy.

Do not automatically increase default_pool_size or max_connections. If slow queries and abandoned transactions already occupy connections, allowing more work to compete for CPU, RAM, and disk can make the situation worse.

A more useful alert combines connection acquisition latency, waiting-client count, and queue age with the affected application operation.

6. Replication lag: The replica can respond while returning stale data

pg_stat_replication exposes replay_lag, but this is not a universal freshness guarantee.

When the replica catches up and there are no new commits, lag measurements may eventually become NULL. If a replica disconnects entirely, its row can disappear from the primary's replication statistics. An alert that only checks whether replay_lag exceeds a threshold can miss both cases.

For critical read paths, consider measuring freshness at the application level. One approach is to update a heartbeat timestamp on the primary every few seconds and read that same row from the replica. Comparing the stored timestamp with the current time estimates how stale the replicated data is, provided the clocks are synchronized.

This tells you something closer to what a customer experiences: Can a read from the replica observe recent writes?

7. WAL retention: A failure that stays invisible until it is urgent

A replication slot tracks the WAL position still needed by its consumer. When the consumer stops advancing, the slot may keep WAL files from being removed.

An abandoned logical subscriber or disconnected standby can therefore produce sustained WAL growth. If no effective retention limit is configured, WAL can continue consuming storage until disk exhaustion threatens availability.

Inspect retention from the primary:

-- Inspect WAL retained by replication slots on the primary.
SELECT
    slot_name,
    active,
    wal_status,
    pg_size_pretty(
        pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)
    ) AS retained,
    safe_wal_size
FROM pg_replication_slots;

safe_wal_size indicates how much more WAL can be written before a slot risks losing necessary files when max_slot_wal_keep_size is configured. It may be NULL depending on slot state and configuration.

PostgreSQL 18 introduced idle_replication_slot_timeout. It is disabled by default and can invalidate qualifying inactive slots after the configured duration, with invalidation applied during a checkpoint. That protects against some abandoned-slot scenarios, but it also requires operational planning because affected consumers may need their slots recreated.

Why this belongs in alerting: Users may see normal latency right up until a disk or WAL retention problem reaches a dangerous limit.

8. Transaction ID wraparound: The risk p99 cannot predict

PostgreSQL uses 32-bit transaction IDs and relies on vacuuming and freezing old transaction identifiers to avoid wraparound problems.

If that maintenance falls too far behind, PostgreSQL first emits warnings and eventually refuses commands requiring new transaction IDs to protect data integrity. The practical outcome can be a database that stops accepting writes.

Monitor the age of the oldest unfrozen transaction ID in each database:

SELECT
    datname,
    age(datfrozenxid) AS xid_age
FROM pg_database
ORDER BY xid_age DESC;

By default, autovacuum_freeze_max_age is 200 million transactions. PostgreSQL's protective limits are much farther away, but waiting for critical warnings is not an acceptable monitoring strategy. Consult the PostgreSQL documentation on routine vacuuming for the relevant thresholds and safeguards.

This is not a theoretical failure mode:

In February 2019, Mailchimp's Mandrill service experienced a major outage after transaction ID exhaustion affected a database shard. The disruption lasted roughly 40 hours.
Sentry documented a transaction ID wraparound incident in 2015 that disrupted its service for a substantial part of the US working day.

In Mandrill's case, engineers had identified the growing risk months earlier, but planned monitoring work remained in the backlog. The alert arrived after the incident had already been resolved.

An early XID-age warning, for example around one billion with suitable follow-up escalation, can be more useful than waiting for user-facing errors. The precise threshold must be chosen for the database's transaction rate, vacuum behavior, and operational response time.

9. Disk I/O and checkpoints: Correlate them with latency

Checkpoints write dirty data to disk and can contribute to I/O pressure, but a checkpoint should not automatically trigger a page.

Starting in PostgreSQL 16, pg_stat_io provides useful I/O statistics. Relevant timing fields require track_io_timing, which is disabled by default for most database I/O, and WAL timings use track_wal_io_timing where applicable. I/O timing can include operations served by the operating system's page cache, so use host-level tools such as iostat or node_exporter to understand physical device behavior.

Frequent checkpoints caused by WAL pressure can create log warnings. PostgreSQL's WAL configuration documentation cautions against overreacting to a single warning; repeated warnings combined with rising write latency deserve investigation.

Do not blindly increase max_wal_size. More WAL capacity can reduce some checkpoint pressure but also consumes disk space and can increase recovery time after a failure.

Better diagnostic question: Are I/O latency and checkpoint activity correlated with slower writes or failed user operations?

10. Comparison: State-first vs. symptom-first alerting

Question	Approach A: Database state	Approach B: User symptoms
What triggers a page?	Connection thresholds, replication, WAL, locks, capacity	p99 latency, error rates, connection wait time
When do you find out?	Potentially before users notice	Once service quality degrades
Main failure mode	Excessive noise and alert fatigue	Gradual risks can stay invisible too long
Where is it strongest?	WAL retention, XID age, replication-slot health	Lock queues, pool waits, slow operations
What is required?	PostgreSQL exporters and meaningful per-cluster thresholds	Application instrumentation, histograms, agreed service objectives

Neither strategy is sufficient on its own.

11. The alerting policy we would actually use

Our preferred model is symptom-first paging with a small set of high-consequence predictive exceptions.

Page when users are affected

p99 latency of a critical operation exceeds its objective for a meaningful interval and volume.
The error rate of a critical operation increases.
Connection acquisition time or PgBouncer queue age affects requests.
Replica freshness falls behind an application's accepted limit.

Page before a silent resource failure

WAL retention is increasing toward available disk or configured limits.
A replication slot has stopped advancing and creates a credible retention risk.
Transaction ID age is approaching an escalation threshold.
Critical disk headroom is projected to run out before a human can safely respond.

Keep on dashboards unless correlated with risk or impact

Ordinary lock counts.
Raw CPU utilization.
Connection occupancy without a waiting queue.
Routine checkpoints.
Normal variations in disk I/O.

Make every page actionable

Compare these notifications.

Poor alert:

PostgreSQL active connections: 80%.

Useful alert:

Watch operation p99 is 4.2 seconds, compared with a normal 300 ms. Thirty-seven sessions are waiting on titles. The queued ALTER TABLE (PID 81234) is blocked by an analytical report (PID 80112).

The second tells an engineer what customers are experiencing, how many sessions are affected, and where to investigate first.

For each proposed page, ask a simple question: If this reaches someone at 3 a.m., is there a clear action to take?

If the answer is only "look at it and close it," it probably belongs on a dashboard instead.

There is one important exception to the instrumentation ideal. If you operate an old monolith and cannot safely add service-level metrics, state-based monitoring is still better than no monitoring. Just tune its thresholds to the actual database, not to a generic configuration copied from the internet.

Conclusion: Monitor outcomes, investigate causes

A healthy CPU graph does not prove that PostgreSQL is serving users well.

A full dashboard does not prove that your monitoring is effective.

The useful distinction is between signals that should wake someone up and metrics that help that person investigate.

Start with customer-facing latency, errors, and connection waits. Add carefully chosen predictive alerts for replication, WAL, XID wraparound, and genuine capacity exhaustion. Keep the rest available for diagnosis.

Fewer pages. Better context. Faster decisions.

Over to you

What PostgreSQL alert last prevented a real outage in your environment? And which alert has been firing for months without anyone taking action?

If you prefer state-based paging, what incident convinced you it was better than a symptom-first approach?

Shared, Dedicated, or Storage: Which Aéza Server Should You Choose for Your Project?

Aeza — Mon, 12 Oct 2026 13:15:38 +0000

A project starts out on the cheapest VPS available. A couple of months later, the website begins slowing down in the evenings. The team upgrades to a plan with “room to grow,” usually interpreted as “more CPU cores.” But the actual bottleneck may be insufficient memory or competition for CPU time with other virtual machines. The hosting bill goes up, and the performance problems remain.

To avoid this, it helps to answer four questions before ordering a server. Does your project need dedicated CPU resources, or will shared cores do? How much memory and storage will it need six months from now? Where are its users located? And how long will it run—a few days for testing, or years in production?

Shared vs. Dedicated: What Is the Actual Difference?

At first glance, the specifications of the two VPS ranges look identical. Entry-level configurations offer 1 CPU core, 2 GB of RAM, and 30 GB of NVMe storage, while the largest configurations go up to 16 cores, 32 GB of RAM, and 480 GB of NVMe storage. The fundamental difference is how CPU time is allocated.

With Shared VPS, CPU cores are shared. The physical processor divides its time among multiple virtual machines, so when neighboring workloads are busy, your VM may receive less CPU time. RAM and storage capacity, however, are allocated to your instance rather than shared in the same way. For a website with relatively steady traffic, a bot, or a testing environment, this is often sufficient: these workloads typically leave the CPU idle for much of the time.

With Dedicated VPS, a full physical CPU core is reserved for your server, and disk operations are additionally accelerated by an in-memory cache (RAM Cache). Other customers' VMs do not take away your allocated CPU time. This option makes sense when CPU demand is sustained or when users are sensitive to latency—for example, with e-commerce platforms, game servers, and databases.

Both ranges use KVM virtualization. Each virtual machine is isolated from the others, you have full control over the guest operating system's kernel and firewall, and tools such as Docker, Podman, and LXC run as usual. Nested virtualization is disabled, however, so you cannot run a hypervisor such as Proxmox or create your own KVM virtual machines inside the VPS. For that, you would need a physical dedicated server, which we will discuss later.

At the entry level, Dedicated VPS costs approximately 1.4 times as much as Shared VPS for an equivalent configuration of 1 core, 2 GB of RAM, and 30 GB of NVMe storage. Check Aéza's website for current prices and billing currencies.

How Much Capacity Should You Plan For?

Start with memory, because shortages often become noticeable first. You need enough RAM for the operating system, the application itself, and the data it accesses frequently. If available memory regularly approaches zero and the system begins swapping or triggers the OOM killer, adding CPU cores will not solve the problem. You can check available memory with free:

# Check the available column; 'free' memory is often low on Linux because of caching.
free -h

CPU requirements depend on the workload. A web application handling many concurrent requests can often make effective use of multiple cores. A single-threaded task, such as a game loop or an individual resource-intensive report, benefits more from higher per-core clock speed. Aéza's Shared and Dedicated VPS ranges use AMD Ryzen 9 processors with frequencies of up to 5.7 GHz.

Both ranges use NVMe storage. A 30 GB disk may be enough for the operating system and a small project, but logs, user uploads, and local backups can grow faster than expected.

If none of the five predefined plans fits your needs, you can build a custom configuration when ordering: 1–16 CPU cores, 2–32 GB of RAM, and 10–480 GB of NVMe storage. This is useful when your project needs plenty of RAM but few CPU cores, or the reverse.

Where Should You Host Your Server?

Virtual servers are available in international locations including Helsinki, Stockholm, Amsterdam, Vienna, Paris, London, and Charlotte (USA). As a general rule, hosting closer to your users reduces network latency. The order form displays the ping from your computer to each location, and you can inspect the route to a specific data center through Looking Glass.

Hardware also varies by location. In Charlotte, configurations use AMD Ryzen 9 9950X processors and offer network links of up to 25 Gbps. Many European locations use Ryzen 9 7950X3D processors, with network links of up to 1 Gbps. Available hardware, network capacity, and prices may differ by location, so check the configuration shown during ordering.

Other features are not location-dependent. Traffic is unlimited, and each server includes one IPv4 address and an IPv6 /48 subnet. Basic DDoS protection is included and runs continuously, with up to 2 Tbps of coarse filtering and 600 Gbps of fine-grained filtering. For gaming workloads and more complex cases, enhanced L3–L4 and L7 protection is available as a separate service, configured with support for the specific application.

All Server Options at a Glance

Product	Specifications	Billing	Recommended use
Shared VPS	Shared AMD Ryzen 9 cores, NVMe, 1–16 cores	Hourly or monthly	Websites, bots, development, and testing
Dedicated VPS	Dedicated physical CPU cores, NVMe with RAM Cache	Hourly or monthly	E-commerce, game servers, and databases
Promo VPS (Stockholm)	1 AMD Ryzen 5 3600 core, 4 GB RAM, 10 GB NVMe, up to 300 Mbps	Promotional plan; check availability	Landing pages, bots, and learning projects
Storage VPS	Intel Xeon Platinum 8260, 2–16 TB of HDD storage in ZFS RAID, up to 10 Gbps	Monthly; check current rates	Backups, archives, and media files
Physical Dedicated Server	AMD Ryzen 7 7700, 8 cores, 96 GB RAM, 2 TB storage	Monthly; check current rates	Running your own VMs and hosting large databases
Web Hosting	Control panel, SSL, MySQL, and email	Monthly; check current rates	WordPress and other CMS-based websites

Recommended Configurations for Common Workloads

Website or Blog

If you do not want to manage a server yourself, web hosting is the simpler option. The Start plan supports one website, three databases, email mailboxes, and an SSL certificate. Our specialists can install WordPress or migrate an existing WordPress site. Check current introductory offers and trial availability on the official website.

If you need your own technology stack—for example, Node.js, Docker, or custom web server settings—a Shared VPS with 2–4 GB of RAM is a better fit. Current pricing is available on Aéza's website.

Bot, Pet Project, or Test Environment

A bot, pet project, or test environment will often run comfortably on a Shared VPS with 1–2 CPU cores and 2–4 GB of RAM. For environments that only need to run for a few hours or days, hourly billing is convenient: a server is provisioned in about a minute, and you pay only for the hours you use. Servers can also be managed through a REST API, including from CI pipelines.

For particularly lightweight workloads, such as a landing page or a simple bot, the Promo plan in Stockholm may be sufficient for basic projects. It has a network link of up to 300 Mbps and a less powerful CPU, so Shared VPS is a better starting point for production services with meaningful load.

E-commerce and Business Applications

E-commerce platforms and business applications often depend on database queries, disk access, and caching. Some workloads are also sensitive to per-core performance. Dedicated VPS can provide more predictable CPU availability because the allocated cores are not shared with neighboring VMs.

For a small online store or business application serving a handful of users, a reasonable starting configuration is 4 cores and 8 GB of RAM. As the product catalog and order volume grow, consider moving to 8 cores and 16 GB of RAM, guided by actual resource measurements.

Game Server

Players notice lag immediately, which makes shared CPU cores a less attractive option. Many game servers, including Minecraft servers, execute their main game loop on a single thread. That means per-core clock speed often matters more than the total number of cores.

For a vanilla Minecraft server or a Paper-based server with a few dozen players, the same Dedicated VPS configuration with 4 cores and 8 GB of RAM is a reasonable starting point. Modpacks generally require considerably more memory.

Choose a location close to your players and test real latency to that data center before deploying. Basic DDoS protection is included, and enhanced filtering can be arranged through support for the requirements of a particular game.

Database Server

For PostgreSQL or MySQL, the key is to have enough memory to hold frequently accessed data. This allows most reads to be served without reaching the disk, while NVMe storage with RAM Cache handles the remaining I/O efficiently.

Start by considering Dedicated VPS, and size RAM according to your working set—the frequently accessed data and indexes—rather than the total size of the database.

Backups can be configured in the control panel: up to five copies on an automated schedule, at 20% of the server price. For critical databases, it is also useful to maintain separate database dumps, for example on a Storage VPS, as described below.

Storage for Backups and Files

Aéza offers a separate product for this use case: Storage VPS. These virtual servers use HDD storage in ZFS RAID, with configurations ranging from 2 TB to 16 TB. Check Aéza's website for current pricing, availability, and supported locations.

The network link offers up to 10 Gbps, so uploading and restoring database dumps will rarely be limited by storage-server network bandwidth. However, HDD storage is not suitable for databases or other workloads requiring fast random disk access. Those are better hosted on NVMe.

When a VPS Is No Longer Enough

A physical dedicated server makes sense if your project needs more than 32 GB of RAM or 480 GB of disk space, if you want to run your own virtual machines, or if you need the entire physical machine to yourself.

Available configurations include an AMD Ryzen 7 7700 with 96 GB of RAM and 2 TB of storage. Another option offers 192 GB of RAM and a 2.5 Gbps network link. Ask support about current configurations, custom builds, and whether you can announce your own autonomous system (AS) and network prefixes.

How to Reduce Hosting Costs

Start with a smaller plan. You can increase CPU, RAM, and disk capacity through the control panel without losing your data. The server reboots and becomes available again in approximately one minute. There is no need to buy excess capacity far in advance.

Use hourly billing for short experiments. Suppose you need to run a load test over a weekend. With a Dedicated VPS configured with 8 cores and 16 GB of RAM, a 48-hour test only requires paying for those hours rather than a full monthly period, if hourly billing is offered for that configuration. Compare the current hourly and monthly rates before ordering.

If you are unsure whether Shared VPS still provides enough CPU performance, check CPU steal time. It shows how much CPU time the hypervisor assigned to other virtual machines while your instance was waiting to run:

# The st column shows the percentage of time taken by the hypervisor for other VMs.
# Ignore the first row, which is the average since system boot.
vmstat 1 5

If st consistently reaches several percent or more while your application is CPU-bound, it may be time to switch to Dedicated VPS. If st stays near zero, investigate memory, disk I/O, or application-level bottlenecks instead.

Prepay for long-running projects. Longer billing terms may qualify for discounts: 5% for three months, 9% for six months, and 12% for a year under the tariff terms described here. Confirm current offers for your account and chosen location before paying in advance.

How to Order a Server

You can order a server through your account dashboard. First, choose a location. Then select a standard plan or a custom configuration, choose the operating system, and set the billing period.

Available Linux distributions include Ubuntu 26.04, 24.04, and 22.04; Debian 13 and 12; AlmaLinux; Rocky Linux; CentOS Stream 9; Arch Linux; and Alpine Linux.

You can pay hourly, weekly, monthly, or in advance for 3, 6, or 12 months to receive the corresponding discount. You can also enable automatic renewal and order several identical servers at once. The server is generally ready to use approximately one minute after payment.

If you prefer not to handle server setup and maintenance yourself, you can also request Aéza's server administration service through the dashboard.

Key Takeaways

Shared VPS is a good choice for websites, bots, and test environments where the CPU is idle much of the time. Dedicated VPS is better for sustained workloads or applications where users immediately notice latency: online stores, game servers, and databases. Storage VPS is designed for file storage and backups, while a physical dedicated server is the next step when you need more than 32 GB of RAM, more than 480 GB of storage, or your own hypervisor. If you just need a website without server administration, web hosting may be the most convenient option.

You can build your configuration on the virtual servers page. If you are unsure which plan is right for your project, contact support. The team is available 24/7 and can help you choose the most appropriate server.

VPS Server: What It Is, How It Works, and Who Needs It

Aeza — Mon, 14 Sep 2026 19:50:25 +0000

A VPS server provides a separate operating system and administrative access without renting an entire physical machine.

This solution is suitable for services requiring a permanent IP address and predictable workloads.

When choosing a VPS, important factors include virtualization technology, CPU, memory, storage, network limits, and responsibility boundaries. Two plans with identical descriptions may have completely different real-world conditions.

What is a VPS in simple terms?

A VPS is a virtual machine with a guest operating system running on physical hardware through a hypervisor.

The user receives:

administrative access;
an IP address;
virtual disks;
allocated resource limits.

The provider manages the physical host and virtualization layer.

Actual isolation and performance guarantees depend on:

virtualization technology;
resource allocation model;
provider terms.

What is included in the service and what remains your responsibility?

A physical server is divided into multiple virtual machines.

Each VM runs its own operating system, filesystem, users, processes, and network configuration.

Processes from one VM cannot access another VM's files or memory, although all machines share the same physical hardware.

A VPS plan usually includes:

vCPU;
RAM;
storage;
IP address;
traffic limit or network bandwidth.

The provider panel usually manages:

power operations;
OS reinstallations;
emergency console access.

Inside the operating system, administration remains your responsibility:

web server;
database;
updates;
users;
backups.

Administrative access gives flexibility but also responsibility.

A wrong SSH configuration, exposed database, or deleted file exists inside the guest system and will not be fixed by the hypervisor.

Before ordering, verify:

who installs updates;
who handles incidents;
who restores data.

How does a hypervisor create multiple servers?

On Linux, KVM uses CPU virtualization extensions and provides an interface for running virtual machines.

Each VM usually runs as a QEMU process.

QEMU executes guest code through KVM and provides virtual hardware.

The management layer, commonly libvirt, defines:

vCPU;
memory;
disks;
network interfaces;
startup rules.

The virtualization stack:

Physical CPU, RAM, NVMe, NIC
        ↓
Linux kernel with KVM
        ↓
QEMU + virtio devices
        ↓
Guest OS
        ↓
Application

The host scheduler distributes CPU time between virtual machines.

CPU pinning, quotas, and overcommit depend on the provider configuration.

The guest does not access hardware directly. It uses virtual resources provided by QEMU.

Disk and network operations use paravirtualized virtio queues.

Virtual machines are isolated from each other, but physical CPU, storage, and networking remain shared.

Therefore, comparing VPS providers requires understanding resource allocation and testing real workloads.

What do vCPU, RAM, NVMe, IOPS, and bandwidth mean?

vCPU shows how many virtual processors the guest sees.

You should also evaluate:

CPU generation;
frequency;
CPU quotas;
host contention.

For continuous workloads, long-term performance and scheduling latency matter.

Linux %steal shows when a VM wanted CPU time but the hypervisor did not provide it.

RAM defines available memory.

Behavior during memory pressure depends on:

memory guarantees;
ballooning;
overcommit;
swap policy.

Frequent swapping increases latency.

For databases, guaranteed RAM with additional capacity is often better than a large advertised amount without guarantees.

NVMe performance depends on:

IOPS;
throughput;
block size;
queue depth;
read/write ratio.

Example:

3000 IOPS with 4 KiB blocks
≈ 11.7 MiB/s random workload

Real performance is affected by filesystem overhead, queues, and contention.

Network resources include:

port speed;
traffic limits;
packets per second;
outgoing bandwidth;
routing quality.

A cloud VPS may provide APIs and automatic resource creation, but scaling capacity depends on the actual infrastructure, not the product name.

VPS vs shared hosting vs dedicated server

A VPS provides:

its own operating system;
administrative access;
resource limits.

Shared hosting provides a managed environment with less control.

A dedicated server provides the entire physical machine to one customer.

Managed services, CPU guarantees, memory guarantees, and storage guarantees differ between providers.

Managed vs unmanaged VPS

With unmanaged VPS:

The provider manages:

physical hardware;
hypervisor;
power;
external network.

The customer manages:

OS;
accounts;
firewall;
applications;
updates;
logs.

The exact responsibility boundary depends on the contract.

Managed VPS may include:

monitoring;
updates;
alerts;
recovery assistance.

However, the exact service must be checked before purchase.

A control panel simplifies management but does not automatically make a VPS managed.

Snapshots are not always backups because they may exist on the same platform and may not guarantee application recovery.

Common VPS use cases

VPS servers are commonly used for:

websites;
APIs;
bots;
monitoring systems;
Git runners;
testing environments.

They are useful when applications require:

a permanent IP;
background processes;
operating system control.

Heavy databases and workloads requiring GPUs may require different solutions.

Workload	Example profile	Critical metric	Suitable option
Website/API	2 vCPU, 4 GB RAM	p95 latency, RAM	VPS
Database	High I/O workload	IOPS, disk latency, RAM guarantees	VPS with guarantees or dedicated
Bots	Small CPU, constant network	bandwidth, PPS	VPS
GPU workloads	CUDA/ROCm	GPU model, VRAM	GPU instance/dedicated
Traffic spikes	Rapid RPS growth	scaling speed	Cloud infrastructure

Testing is still required because the same hardware can behave differently depending on workload.

How to choose a VPS plan

Start with measurements.

For CPU:

usage per core;
saturation time;
request latency.

For memory:

working set;
page faults;
swap usage.

For storage:

IOPS;
throughput;
p95/p99 latency.

For network:

average speed;
peak speed;
PPS;
outgoing traffic.

If no baseline exists, start with the smallest suitable plan and run repeatable tests.

Upgrade only the resource that becomes the bottleneck.

A plan such as:

2 vCPU
4 GB RAM
60 GB storage
3000 IOPS
100 Mbps port

is not enough information without knowing:

CPU model;
CPU quota;
storage type;
throughput limits;
traffic rules.

The name VPS or VDS alone does not describe virtualization quality.

Suitable VPS tasks

Websites, APIs, and small databases with stable workloads.
Bots, monitoring systems, Git runners, and test environments.
Applications requiring their own OS and administrative access.

Initial VPS security setup

After first connection:

Verify the SSH fingerprint through a trusted channel.
Create a separate administrator account.
Add a public SSH key.
Test key-based login.
Disable password authentication and root login only after recovery access is confirmed.

Configure:

firewall rules;
OS updates;
application updates;
least-privilege services.

A VPS isolates the machine, not the software.

A vulnerable plugin, leaked token, or incorrect access rule remains your responsibility.

Backups should survive:

VM loss;
account loss;
infrastructure failure.

Store backups separately, encrypt them, and regularly test restoration.

Monitoring should alert about:

disk usage;
stopped services;
errors;
expiring certificates.

Critical systems need a recovery procedure describing:

console access;
backup location;
rollback process;
recovery validation.

VPS, VDS, cloud, and dedicated servers

VPS and VDS are often marketing terms.

The name alone does not show:

CPU allocation;
overcommit policy;
storage guarantees;
virtualization method.

Evaluate:

CPU model;
quotas;
memory guarantees;
IOPS;
network limits;
SLA.

Cloud platforms are useful when infrastructure must be created through APIs, distributed across zones, or connected with managed services.

Dedicated servers provide the entire physical machine and are suitable for:

high constant workloads;
large databases;
specialized hardware requirements.

The correct choice depends on:

required control;
recovery speed;
workload stability;
downtime cost.

A VPS server is suitable when a project needs:

its own operating system;
network control;
predictable resources;

but a dedicated physical machine is unnecessary.

Before purchasing, verify technical parameters, responsibility boundaries, and recovery procedures.

If monitoring confirms that the workload fits within limits with sufficient resource reserve, a VPS can provide the required infrastructure without unnecessary complexity.

Choosing a Game Server Location Based on Ping

Aeza — Mon, 14 Sep 2026 19:44:28 +0000

Choosing a game server location based on ping is determined not by distance alone, but by the networks used by future players. A nearby city may perform worse if packets reach it through an overloaded interconnection between operators. A proper comparison requires several networks, different times of day, and a test gameplay session, because service-level ping requests may behave differently from actual game traffic. Measurements should not be taken only from your own network. Instead, players themselves should run them: several people from each city and from each ISP run the same test, and you aggregate the results.

Which latency tests should be done before choosing a location?

Before choosing a location, you need ping series and traceroutes from player networks, followed by a test session on the game server. Together, these help evaluate latency, jitter, and packet loss at different times of day. Responses to service-level requests may differ from the behavior of actual game traffic, so ping alone is not enough.

Which players are we choosing the server for?

Player groups differ by country, city, ISP, and connection type. Even neighbors using different providers may have different routes to the server. Therefore, measurements must be taken from the actual networks used by your audience. ASN, the Autonomous System Number, helps distinguish networks.

Testing game server location latency should rely on several volunteers from each group. Their results can be stored under codes such as G1-P1 and G1-P2, along with the city, ISP, and connection method. If a player uses Wi-Fi in practice, they should measure the same connection they actually use. If one player’s result is significantly worse than the others in the same group, an additional wired test can show whether the issue is in the home network or in the route itself.

Before measuring, define thresholds: what latency, what packet loss rate, and how many session disconnects are considered acceptable. These thresholds depend on the game and the expectations of the community. Requirements should account for every significant group, because a good average can easily hide problems affecting a minority.

Which servers should be compared?

Location should remain the only difference between candidates. The test VMs should match in CPU resources, network limits, game version, and firewall settings. On an overloaded machine, latency may be caused by resource limitations rather than the network. The candidate list should include the domain name, actual IP, game port, and the period of access to the test environment.

Providers often publish a test IP that can be pinged before renting a location. That is enough to rule out obviously distant regions, but not enough to choose between nearby ones. It is worth asking support whether the address belongs to the required site and whether it passes through the same DDoS protection path as the future VM. Final measurements should be taken on the rented machine.

Ping, jitter, and packet loss answer different questions: how quickly a response arrives, how stable its timing is, and how many responses never arrive at all. A low average can coexist with spikes and losses. IPv4 and IPv6 routes may differ, so if both protocols are supported, they should be tested separately.

How to collect comparable measurements

A comparison will show which country to choose for a VPS only if every participant performs the same scenario for all candidates. For an initial selection, several five-minute series are sufficient: in the morning, in the evening, and during usual gaming hours. If results are close, additional days of observation are required.

The script below is intended for Linux with Bash, the ping utility from the iputils package, and MTR. Save the code as test-region.sh. In the launch command, the IP should be the address of the test VM. Results and utility versions will be saved into a separate folder with a UTC timestamp.

If MTR refuses to open sockets, run the script with sudo, because some builds require root privileges.

#!/usr/bin/env bash
# Usage: bash test-region.sh IP
export LC_ALL=C
ip="${1:?Specify the IPv4 address of the test VM}"
out=$(mktemp -d "probe-$(date -u +%Y%m%dT%H%M%SZ)-XXXX")
printf '%s\n' "$ip" > "$out/target.txt"
ping -V > "$out/versions.txt" 2>&1
mtr --version >> "$out/versions.txt" 2>&1
ping -4 -n -D -c 300 -i 1 "$ip" > "$out/ping.txt" 2>&1
mtr -4 -n -r -w -c 60 "$ip" > "$out/mtr.txt" 2>&1

Ping will send 300 requests at one-second intervals, then MTR will collect a route report. The whole run will take more than five minutes. The order of candidates should be changed between runs so that one region is not always tested before the others. Command parameters are described in the ping documentation and the MTR manual.

How much more important are jitter and packet loss than average ping?

A low average ping does not compensate for frequent latency spikes and packet loss. Because of them, data arrives unevenly or does not reach the game at all, even if most responses are fast. The importance of each metric depends on the game mechanics and how it handles loss, so the final comparison requires testing a real gameplay session.

What the numbers in the report mean

RTT is the time from sending a request to receiving a response. The median, or p50, divides the measured values into two halves. The p95 value indicates the latency not exceeded by about 95% of the received responses. This makes slow responses visible even when the average looks good. The number of requests and responses is needed to evaluate the sample.

For consistency, p95 can be calculated as follows: sort RTT values in ascending order and take the value at position 0.95 × N, rounded up, where N is the number of responses. A short series is not enough to draw confident conclusions about p99. A graph of RTT over time will show whether spikes were concentrated in a single episode.

The term jitter is used for different measures of latency variation. Here it means the average absolute RTT difference between consecutive requests for which both have a response. This calculation is neither the same as mdev from ping output nor the one-way delay variation defined in RFC 3393. Numbers from different sources should not be compared directly.

The percentage of unanswered requests is (tx − rx) / tx × 100%, where tx and rx are the number of requests and unique responses. A lost request is neither turned into zero RTT nor into RTT equal to the timeout. A hosting provider’s looking glass can be used to check connectivity from the provider side, but player-side measurements are still necessary.

How to investigate an unexpected route

If one region performs significantly worse, a traceroute to the game server will help reveal the responding intermediate hops. MTR repeats such probes and collects statistics. Reports from several networks during the problematic period will show whether the degradation affects only one operator or several.

Asterisks and high loss percentages on an intermediate hop do not automatically mean that it is dropping forwarded traffic. A router may rate-limit service replies while still forwarding packets normally. The suspicion becomes stronger when degradation is also visible at the final destination. The actual cause of the issue should be confirmed by the operator.

Hop names may hint at transit through another city, although geography inferred from names or IP databases may be inaccurate. A reverse traceroute from the server to a reachable player address provides information about the opposite direction. Forward and reverse paths may differ, and RTT includes both. A home address hidden behind NAT may limit such verification.

Testing inside the game itself

After network probes, a gameplay session is required on every candidate. The servers should match in game version, mods, map, and settings. Participants should ideally repeat similar actions with the same number of players. The report should include client-side latency, disconnects, and any available packet loss indicators. Some games calculate built-in ping using their own smoothing and internal processing, so it does not have to match ICMP RTT.

VM load requires separate observation. For example, in Minecraft, MSPT means milliseconds per tick, while FPS refers to the client’s frame rate. Slow ticks or FPS drops can ruin the experience even if the network is fine, so these metrics should be stored separately from RTT.

DDoS protection may change the traffic path. Some providers reroute traffic through a scrubbing center — a dedicated site where packets are filtered before being sent to the server. Others filter directly on edge nodes, in which case no visible detour appears. Any such testing must be coordinated with the provider, without independently organizing attacks. After a known enablement or change in filtering, it is useful to repeat both the gameplay session and the network probes.

Why can the closest region on the map be slower?

Traffic exchange between operators, or peering, may go through a distant point.
A congested segment on the route adds queueing delay even when the physical distance is small.
Traffic filtering may change the route to the server and increase latency.

How to choose based on the results

In the table, each combination of player group, candidate region, and test time should have its own row: a good daytime run must not hide a bad evening one. The four main columns are p50, p95, RTT variation, and packet loss.

In the summary CSV, the group field links the row to a city and operator, trace stores the traceroute filename, and game contains the result of the gameplay session.

player,group,region,utc,tx,rx,p50,p95,jitter,loss,trace,game

The player field stores a participant code. Latencies are in milliseconds, and loss is in percent. One row corresponds to one measurement series. An overall p95 cannot be obtained by averaging the p95 values of individual participants.

Candidates that violate mandatory requirements for any significant group are removed from the comparison. The remaining candidates are scored using predefined scales for each metric, for example from 0 to 5, where 5 means the best result. Weights should be defined before calculation. An example is shown below and should be adapted to your own game.

Metric	Weight	What is evaluated
p50 RTT	20%	Typical latency
p95 RTT	40%	Slow responses
RTT variation	20%	Stability of latency
Packet loss	20%	Unanswered requests

The score of each metric is multiplied by its weight, and the sum gives the group score. Group scores are then weighted according to each group’s share of the audience. If scores are close, route stability becomes the deciding factor: a routing path that changes transit provider from one series to another is riskier than a stable path, even with the same p95. The availability of measurement endpoints also matters, because a region that cannot be rechecked after launch is less reliable in practice. If no single region is suitable for everyone, it may be worth considering several servers for different groups.

When to repeat the tests

After launch, the same participants and the same scenario help verify quality on the permanent IP. New measurements are required after changes to the address, provider, filtering settings, or when complaints appear from a specific network.

Player geography also changes. A new large group may change the optimal choice even if route quality remains the same. Therefore, the list of cities, ISPs, and the shares of active participants should be updated regularly.

The report should include the author, dates and time zone, test addresses, VM configuration, utility versions, and game version. A summary CSV without raw data cannot be rechecked, so RTT series, traceroutes, and gameplay notes should be preserved alongside it. Before publication, home IPs and other personal data should be removed from the logs. The measurement date should be shown next to the results. Routing changes over time, so output becomes outdated as the network evolves.

Choosing a game server location based on ping should end with a decision that clearly states the conditions: which groups the region is suitable for, where limitations remain, and when the verification took place. If results are close, the test should be extended. Before the final migration, it is wise to preserve the ability to return to the previous location in case the new route proves unstable.

SSH Hardening Checklist: How to Configure Access and Verify the Result

Aeza — Thu, 10 Sep 2026 12:23:51 +0000

An SSH hardening checklist helps eliminate unnecessary login methods on a VPS. Hardening means configuring the server to reduce unnecessary exposure and improve access security. A secure sshd_config setup comes down to leaving only the authentication methods that are actually required and making sure access can be restored if something goes wrong.

The examples below are intended for Ubuntu 24.04 LTS with OpenSSH from the operating system repositories. If you use another OS or have specific audit requirements, verify the relevant parameters separately.

Which sshd_config parameters matter most during an audit?

The main parameters are PermitRootLogin for direct root access, PasswordAuthentication and KbdInteractiveAuthentication for authentication methods, and AllowUsers or AllowGroups for restricting who is allowed to connect.

When multi-factor authentication is used, AuthenticationMethods becomes important because it defines which authentication methods must succeed before access is granted.

The configuration should be verified with real login attempts for both permitted and prohibited scenarios.

What Exactly Needs to Be Protected

A secure sshd_config configuration starts with identifying everyone who actually needs SSH access: the VPS owner, colleagues, automation, backup systems, and other service accounts.

Passwords can be brute-forced, keys can be stolen, and former employees may retain working access long after it should have been revoked.

Key-only authentication removes password guessing as an attack path. A stolen key still has to be revoked, unnecessary accounts need to be disabled, and a second factor such as a one-time code can reduce the risk associated with a compromised key.

Another risk is losing trustworthy logs. If SSH authentication events exist only on the same server, an attacker who obtains root privileges may modify or remove them, making later incident reconstruction much harder.

Configuration errors can also lock out the administrator who is applying the hardening.

Before changing anything, verify that the provider console gives you administrative access to the filesystem or an equivalent recovery mechanism. In a team environment, it may be useful to explicitly assign someone responsibility for confirming that recovery access exists.

Check Which Configuration the Server Is Actually Using

An SSH audit checklist should include the operating system version, OpenSSH version, and effective configuration before any changes are made.

You can inspect them with:

cat /etc/os-release
sudo /usr/sbin/sshd -V
sudo /usr/sbin/sshd -T

On Ubuntu, /etc/ssh/sshd_config includes files from /etc/ssh/sshd_config.d/.

Usually, the first value read for a parameter takes precedence, and the order depends on the names of included files. Because of that, adding a setting near the end of the main configuration file may not produce the result you expect.

More details are available in the Ubuntu OpenSSH documentation.

Match blocks create exceptions for specific users, addresses, or other connection properties.

The -C option lets you evaluate the effective configuration for a particular connection:

sudo /usr/sbin/sshd -T \
  -C user=admin,addr=198.51.100.10,host=client.example

sudo ss -ltnp

Replace the username, client address, and hostname with the values from your environment.

If a Match condition also depends on the server address or port, include laddr and lport.

The output of sshd -T represents the configuration read from files on disk.

Existing SSH sessions continue operating under the conditions established when they were created, so changes must be tested using a new connection.

Listening ports shown by ss should also be compared against firewall rules on the VPS and any network filtering configured in the provider control panel.

Identify Who Owns Every Account and SSH Key

Create an inventory of all accounts that can log in and execute commands.

Separately identify users that can obtain administrative privileges through sudo.

The authorized_keys file contains public keys that are permitted to authenticate. Every key should have a known owner. A comment written next to a public key is useful metadata, but it does not prove who actually controls the corresponding private key.

Disabling SSH password login should generally happen only after the account and key inventory has been completed.

Personal administrator accounts make it easier to distinguish individual users in logs.

Automation should use dedicated keys. For example, a backup process can have its own key rather than sharing an administrator credential.

The command= option in authorized_keys can restrict a key to a specific command, while restrict disables additional capabilities, including forwarding.

After configuring these restrictions, verify that the intended automation still works.

When rotating an SSH key, first install and test the replacement key. Only after confirming that the new key works should the previous one be removed from all servers.

Deleting a key does not terminate sessions that were authenticated earlier, so compromised active sessions may require separate investigation and termination.

How Do You Disable Password Login Without Losing Access?

First open a new SSH session using a key with a dedicated user that has working sudo access.

Keep the existing session open.

Before applying any changes, validate the configuration, verify provider console access, and prepare a rollback procedure.

After applying the new settings, open another new connection and confirm that key authentication still works.

If multi-factor authentication is enabled, verify the second factor as well.

Decide Which Authentication Methods Should Remain

The following example permits authentication only with public keys.

Replace admin with the username for which you have already tested SSH access and sudo:

# Key-only authentication; no second factor through PAM
PubkeyAuthentication yes
AuthenticationMethods publickey
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
AllowUsers admin

AllowUsers should contain every human administrator and service account that genuinely needs access.

If group-based management is more convenient, use AllowGroups instead.

MFA and an SSH allowlist can work together: the allowlist restricts which accounts are eligible to connect, while the second factor reduces the risk of a stolen key.

PermitRootLogin no disables direct root login completely.

PermitRootLogin prohibit-password, by contrast, still permits root authentication with a public key.

If authentication requires both a public key and a one-time code through PAM, use:

UsePAM yes
KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactive:pam

This form of multi-factor authentication only works if an appropriate PAM module has already been configured.

keyboard-interactive itself does not necessarily mean MFA. Depending on the PAM stack, it may still prompt for a normal password.

For that reason, setting only:

PasswordAuthentication no

is not sufficient to guarantee that every password-based authentication path has been disabled.

How to Apply Changes and Roll Them Back Safely

Before modifying the server, verify key-based login from a new connection.

Run the following from your own machine:

ssh -o ControlPath=none -o PreferredAuthentications=publickey \
  admin@SERVER

This creates a new connection and permits only public-key authentication.

If MFA through PAM is used, specify:

publickey,keyboard-interactive

in PreferredAuthentications.

Inside the new session, verify that sudo works.

Keep the old session open.

In this example, configuration changes are placed in a new file:

/etc/ssh/sshd_config.d/00-local-access.conf

Make sure both this name and the corresponding .disabled filename are unused.

Save the output of sshd -T before making changes, then compare the effective values after modification, including any relevant Match conditions.

You should also determine how SSH is being started on the system.

Run:

systemctl is-enabled ssh.socket

On Ubuntu, socket activation has been enabled by default since version 22.10.

When socket activation is used, sshd is started for incoming connections and reads the configuration for new connections, so a service reload is not required. Testing with a new session is enough.

If socket activation is disabled and a persistent service is running, apply the changes with:

systemctl is-enabled ssh.socket
sudo /usr/sbin/sshd -t && sudo systemctl reload ssh.service

sshd -t prints nothing when the configuration is valid.

If validation fails, the command after && will not execute.

Afterward, verify the root login restriction, normal key authentication, sudo, and every login scenario that should be denied.

For rollback, rename the added configuration file from the existing session or provider console so that it no longer matches *.conf:

sudo mv /etc/ssh/sshd_config.d/00-local-access.conf \
  /etc/ssh/sshd_config.d/00-local-access.conf.disabled

sudo /usr/sbin/sshd -t

This rollback affects only the configuration file introduced in this example.

PAM settings, firewall rules, and SSH port changes need their own rollback procedures.

Do not close the original working SSH session until every expected login and rollback scenario has been tested successfully.

What to Do With Tunneling and Cryptographic Algorithms

SSH security on a VPS also depends on what users are allowed to do after authentication.

An SSH tunnel forwards another application's traffic through the encrypted SSH connection.

AllowTcpForwarding controls TCP forwarding.

X11Forwarding allows graphical applications running on the server to display windows on the client.

AllowAgentForwarding allows processes on the server to interact with the user's local SSH agent.

Capabilities that are not required should be disabled.

However, these settings should not be treated as complete containment. A user with a normal shell may be able to implement equivalent traffic forwarding using another program.

With a supported OpenSSH version, it is usually better to start from the default cryptographic settings.

Before manually changing cipher, MAC, or key exchange algorithm lists, verify client compatibility.

Regular updates are generally more useful than maintaining a custom cryptographic configuration without a specific requirement.

MaxAuthTries limits authentication attempts within one connection.

Setting it too low may cause legitimate clients to fail before they have a chance to offer the correct key.

ClientAliveInterval controls how often the server checks whether the client is still reachable. A user simply pausing while typing commands does not by itself cause the connection to be terminated.

Are Fail2ban and a Non-Standard SSH Port Enough?

No.

Moving SSH to another port often reduces automated login attempts, but a port scan can still discover the service.

Blocking addresses after repeated authentication failures can reduce brute-force traffic, but it does not help when an attacker already has a valid private key.

A stolen working key can authenticate successfully on the first attempt.

Proper SSH hardening therefore requires more than hiding the port or installing Fail2ban.

You still need account and key management, access restrictions, updates, reliable logging, and, depending on the risk model, multi-factor authentication.

Restrict the Network and Preserve Authentication Events

A firewall can allow SSH only from administrator networks.

Test access both from an allowed network and from an external network that should be rejected.

Do not forget IPv6 if it is enabled.

Fail2ban can temporarily block addresses after repeated failed authentication attempts.

After installation, verify that it is actually reading SSH logs and applying bans.

Be careful when testing: your own failed login attempts can trigger a block against your client address.

Logs should record successful logins, failed authentication attempts, and administrative actions performed through sudo.

Recent Ubuntu events can be inspected with:

sudo journalctl -u ssh.service --since "30 minutes ago"
sudo journalctl SYSLOG_IDENTIFIER=sudo --since "30 minutes ago"

An attacker with root access can alter local logs.

Keeping a copy on another system makes later incident reconstruction more reliable.

Verify that the records are actually arriving at the remote logging destination, that clocks on both systems are synchronized through NTP, and that logs are retained for as long as required by your audit or operational policy.

Administrators of the VPS ideally should not have permission to delete the remote copy of those logs.

How to Prove That the Hardening Works

An audit report should contain the operating system and OpenSSH versions, the list of included configuration files, the inventory of users and keys, and the configuration values before and after modification.

When comparing configuration files, include the reason for each change.

Private keys and second-factor secrets must never be included in the report.

The collected configuration and test results form the audit evidence that demonstrates whether the access policy is actually enforced.

Test the following scenarios on the server:

Administrator listed in AllowUsers with a valid key: login succeeds and sudo works
The same user using only a password: login is rejected
Root with a valid key: login is rejected
User outside AllowUsers with a valid key: login is rejected
Connection from a prohibited network when firewall filtering is configured: blocked by the firewall
Rollback to the previous configuration: new connections work according to the previous rules

To test password-based authentication from your own computer:

ssh -o ControlPath=none -o PubkeyAuthentication=no \
  -o PreferredAuthentications=password,keyboard-interactive \
  admin@SERVER

This test disables public-key authentication.

When testing a user outside AllowUsers, use a valid key. Otherwise, the reason for rejection will remain ambiguous because the connection could simply be failing due to invalid credentials.

For MFA configurations, test a correct code, an incorrect code, and a missing code.

Record the date, operator, client address, and corresponding log entry alongside each result.

A firewall rejection can also be confirmed by checking whether the packet or connection counter for the relevant rule increases.

Repeat the access verification after system updates and whenever the composition of the administrator team changes.

A good result is simple: permitted users can log in, prohibited scenarios are rejected, and rollback has already been tested instead of being improvised during an outage.

Keep the SSH hardening checklist together with the recovery procedure.

Every exception should have a reason, an owner, and a review date.

That allows the next administrator to distinguish a legitimate operational requirement from a temporary rule that was added months earlier and forgotten.

Cloud Server: How It Differs from a VPS and When to Choose the Cloud

Aeza — Thu, 10 Sep 2026 12:21:21 +0000

A cloud server often looks like a familiar VPS: you choose CPU, memory, and disk, receive an address, and connect over SSH. The difference becomes noticeable when you need to quickly add servers, replace a failed instance, or attach separate storage. These capabilities should be evaluated before migration.

Configuring cloud servers involves networking, access permissions, and data storage. On a platform with separate managed services, this may require more work than with a traditional VPS. At the same time, a single machine remains a single point of failure: if it becomes unavailable, the application will also stop responding.

How does a cloud server differ from a VPS?

A cloud server usually operates as part of a platform with programmable management of machines, storage, and networks. This makes it easier to create resources and adjust their number according to load.

VPS services can also provide APIs and hourly billing, so the real differences should be evaluated through the capabilities and conditions of specific services.

What Is Hidden Behind the Terms VPS and Cloud

A VPS is a virtual machine with its own operating system and a defined share of resources from a physical server. In cloud environments, such a machine is often called an instance. Both services may use the same virtualization technology.

A cloud platform combines compute, storage, and networking under common management. Users can request resources themselves and release them when they are no longer needed.

According to the NIST definition of cloud computing, cloud platforms are characterized by on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service. Elasticity here means the ability to increase or decrease allocated resources as workload changes.

In practice, the boundary between VPS and cloud services is blurred. For example, DigitalOcean provides APIs and autoscaling groups for its virtual machines.

The useful question is not simply whether an API exists, but what it can actually do: only reboot an instance, or also create disks, configure networks, and replace machines.

If infrastructure settings rarely change, automation may remain unused, and cloud server administration may differ very little from operating a regular VPS.

If a team creates test environments every day, however, an API can eliminate a large amount of repetitive work in the control panel.

How to Configure a Cloud Server: Images, Disks, and Access

Through an API, software can instruct the platform to create a machine from an image, attach a disk, or assign an address.

An image contains a prepared operating system, while a startup script installs the application and its dependencies. Images and startup scripts make cloud server deployment reproducible.

Passwords and other secrets should be stored separately from the shared image and provided to applications with limited permissions.

Access to platform resources is usually controlled through IAM. It defines permissions for employees and software.

For example, a backup script does not need permission to delete production servers.

Resources are also commonly grouped into projects so that teams can manage them more clearly.

Before selecting storage, determine what remains after the machine itself is deleted.

A local disk is tied to a physical host, while an attached network volume may continue to exist independently of the instance.

Whether the volume survives deletion depends on the platform and configuration. For example, Amazon EC2 uses the DeleteOnTermination attribute to determine whether an EBS volume should be deleted together with an instance.

Object storage is often better suited for user files. An application accesses it through an API rather than as a normal filesystem disk.

A database can be placed on a suitable volume or delegated to the provider through a managed database service.

When Rapid Scaling Actually Works

Increasing the memory or CPU count of a single machine is called vertical scaling.

Changing the size of an instance may require downtime, so the exact resize procedure should be checked beforehand.

Horizontal scaling means adding additional application instances.

A load balancer distributes incoming requests among them.

Every instance must have access to shared data such as user sessions, files, and orders.

If an online store keeps a shopping cart only in the memory of one machine, a second instance will not see it.

When Does Cloud Elasticity Justify the Added Complexity?

Elasticity is useful when load varies significantly and additional application instances can start and begin processing traffic before the system becomes overloaded.

After demand falls, unnecessary resources can be released to reduce costs.

If the application cannot be distributed across multiple machines, or if the traffic spike ends before additional instances can start, automatic scaling provides little benefit.

After a machine is created, the operating system still needs to boot, the application has to start, and frequently used data may need to warm the cache.

In AWS, for example, instance warmup defines a period during which metrics from a newly launched instance are excluded from some autoscaling calculations.

Readiness to accept traffic should be checked separately through health checks. A load balancer needs a reliable way to determine whether the application is actually ready.

The scaling metric should represent a real shortage of capacity.

For background workers, the number of queued jobs per instance can be useful.

For compute-heavy workloads, CPU utilization may be appropriate.

Launching from a prepared image can reduce startup time but does not guarantee that a new server will appear immediately.

Deployment may still fail because of account quotas or insufficient capacity in a selected zone.

Maximum instance counts limit spending, while a pre-launched reserve can help absorb short spikes.

Why One Machine Can Still Fail

During a restart after failure, a service may remain unavailable.

To continue serving traffic after losing one machine, requests must be accepted by other instances.

To protect against failure of an entire location, instances can be deployed across different availability zones.

A zone may contain one or more data centers, but the actual independence of power and networking should be verified with the provider.

For example, a web application can use a load balancer in front of two application instances placed in separate zones.

The application uses a shared database, with a primary instance in one zone and a standby in another.

User files are stored in object storage replicated across multiple zones.

The load balancer itself must also tolerate a zone failure, and the surviving zone must have enough capacity to handle the full workload.

Storage architecture has to follow the same logic.

A network volume may be available only inside one zone. Moving database operations to another zone requires replication and failover.

For example, Amazon RDS provides Multi-AZ deployments for this purpose.

Backups remain necessary even when replicas exist, because accidental deletion or logical corruption can be replicated to the standby system as well.

An SLA defines the provider's obligations, including availability guarantees for resources.

The advertised availability percentage of one instance cannot automatically be applied to the entire website because the application still depends on DNS, networking, databases, and other components.

Amazon EC2, for example, defines different SLA commitments depending on the deployment model.

The SLA should therefore be checked for compensation rules, exclusions, and the responsibilities that remain with your own team.

How to Compare Performance and Test Recovery

Even servers with the same number of virtual CPUs and the same amount of memory can perform differently.

Performance depends on factors such as:

physical CPU model
available CPU time
disk performance
network distance
burst behavior
IOPS
throughput
latency

A valid comparison requires the same application version, identical data, and the same request profile.

The load generator must also be powerful enough to generate the required traffic without becoming the bottleneck itself.

For a web application, useful metrics include requests per second, error rate, and response time.

The p50 median shows the response time under which approximately half of requests complete.

p95 and p99 represent thresholds under which 95% and 99% of requests complete.

Results under normal and peak load should be compared with CPU, disk, and database utilization.

A short benchmark may run entirely inside a temporary burst period.

Some instance types provide extra performance through accumulated credits. Once those credits are exhausted, performance may return to a lower baseline.

Storage can be constrained both by IOPS and by data throughput.

With large operations, the throughput limit may be reached before the IOPS limit. Amazon EBS documents this distinction in its I/O characteristics.

If the service allows temporary burst performance, the test should continue long enough to measure behavior after the burst ends.

The next step is to verify whether data survives instance replacement.

On a test copy of the service, write known data to the database and file storage.

Then perform three separate operations:

Reboot the operating system.
Stop and start the machine.
Replace it with a new instance created from the image.

After each operation, check the data, addresses, and application availability.

In EC2, for example, instance store data can survive a reboot but disappears when the instance is stopped or replaced.

To test fault tolerance, stop one application instance while the system is under load and verify that the second instance continues serving requests.

Record recovery time, client errors, and the final state of the data.

Losing an entire availability zone requires a separate test.

What Makes Up the Bill

Hourly billing is convenient for temporary workloads.

A machine running for the entire month accumulates the full monthly compute cost, and storage and other platform services are added on top.

A monthly calculation should include:

Virtual machines: runtime hours for each instance × hourly rate, including reserve capacity
Disks and copies: storage volume, additional IOPS, snapshots, and backups
Networking: outbound traffic, cross-zone traffic, and paid addresses
Platform services: load balancers, databases, log storage, and queries
Team work: migration, support, upgrades, and recovery

For illustration, assume an arbitrary rate of 10 units per hour.

One machine running for 30 days, or 720 hours, costs 7,200 units.

If a second machine is required for four hours on 20 days, it adds 800 units, producing a total compute cost of 8,000 units.

Two continuously running instances would cost 14,400 units for the same period.

This example includes only compute. Storage, network traffic, backups, and other services still have to be added.

Some expenses continue after a machine is stopped.

In EC2, for example, compute billing stops for a stopped instance while retained EBS volumes continue generating storage charges.

Rules for addresses, snapshots, and other retained resources depend on the service.

Resource limits and automatic cleanup of unused infrastructure should therefore be configured deliberately.

Which Workloads Are Better Left on a Fixed VPS?

A small website with stable traffic that fits comfortably within one selected plan and can tolerate downtime during recovery.
A bot or internal tool with predictable load and no requirement to remain available during a machine failure.
An application that cannot yet operate across multiple instances, as long as a single VPS provides enough resources.

How to Choose the Right Option for Your Workload

How to create a cloud server is usually explained clearly in the provider's documentation.

The harder part is preparing an application to run across several machines.

Shared file storage, session data, monitoring, deployment, and recovery procedures all need to be designed.

The engineering effort required to build and maintain this architecture should be included in the decision.

A typical comparison looks like this:

Management
- Fixed VPS: control panel; API capabilities depend on the service
- Cloud platform: creation and management of related resources through APIs
Load growth
- Fixed VPS: upgrade to a larger plan or add more VPS instances
- Cloud platform: machine groups and scaling policies
Storage
- Fixed VPS: disk usually included in the plan
- Cloud platform: separate volumes, object storage, and managed database services
Networking
- Fixed VPS: addresses and networking options included in the selected plan
- Cloud platform: private networks, load balancers, and availability zones
Availability
- Fixed VPS: recovery according to the service conditions
- Cloud platform: multiple zones when the application architecture supports them
Costs
- Fixed VPS: often a fixed monthly amount plus optional extras
- Cloud platform: accounting for multiple independently consumed resources

An online store with occasional advertising campaigns may be fine with temporarily increasing VPS capacity before a promotion.

If load fluctuates frequently and unpredictably, automatic instance creation may be more useful, provided the application is designed for it.

Cloud infrastructure can also be useful under constant load when a team needs managed databases or more convenient recovery mechanisms.

Several VPS instances behind a load balancer can also serve a distributed application.

Both a cloud platform and a multi-VPS architecture still require operational work.

How to Test the Idea Before a Full Migration

A pilot migration is easier to start with a component that does not contain unique local data.

For example, an image-processing worker can read source files from shared storage and write results to another location.

Such a component is easier to recreate in the cloud or move back to the VPS.

Moving the main database during the first stage makes rollback much more difficult.

Before launching the pilot, define limits for:

response time
error rate
recovery time
monthly budget

Comparing p95 latency against the target threshold shows whether the configuration is suitable under a real workload.

Costs should be projected for an entire month, including traffic peaks, even if the pilot itself lasts only one week.

The pilot plan should record:

region and availability zones
instance types
disk types
operating system version
application version
machine image
startup procedure

Logs of resource creation and deletion, request results, and billing exports make the test reproducible.

You should also write an operational instruction for another team member: how to access the cloud server, where to request permissions, and how to recover the component without the original operator being present.

At the end of the experiment, return the application to its previous deployment.

Required data must remain intact and processing must continue correctly.

A worker may receive the same task more than once after a failure. For example, Amazon SQS standard queues use an at-least-once delivery model, so processing should be designed to avoid duplicate side effects.

The experiment should be stopped if it causes data loss, exceeds the defined budget, or requires constant manual intervention.

After the pilot, remove unused disks, addresses, snapshots, and other resources that can continue generating charges.

Finally, compare the actual benefit with the cost of migration and ongoing operations.

Cloud infrastructure pays off when the team uses it as a platform: provisioning resources through APIs, running applications across multiple zones, delegating part of the operational work to managed services, and recovering systems through documented procedures.

A single virtual machine purchased from a cloud platform instead of a VPS remains a single machine with the same basic point of failure and may simply produce a more complicated bill.

If a cloud server does not solve any current problem, a working VPS can stay where it is.

Migration can be reconsidered when there is a concrete reason: increasing load, stronger availability requirements, or too much manual operational work.

Why Your Minecraft Server Lags at 5 GHz: Single-Core Performance, MSPT, and Stable TPS

Aeza — Mon, 07 Sep 2026 14:44:48 +0000

A CPU advertised at 5 GHz can still run a laggy Minecraft server. Clock speed matters, but it is only one part of the performance equation.

Stable tick processing also depends on instructions per clock, cache and memory latency, sustained boost behavior, Java garbage collection, server software, plugins, mods, chunk activity, and contention on the host.

The practical question is not, “Which processor has the highest GHz?”

It is: Can the server complete its critical tick-processing work within the available time budget under sustained, representative load?

For a server targeting 20 ticks per second, the nominal budget is 50 milliseconds per tick. When processing regularly exceeds that budget, the server cannot maintain 20 TPS.

This article explains how to diagnose Minecraft server lag, interpret MSPT, and compare CPUs without relying on advertised boost clocks.

Why single-core performance matters for a Minecraft server

In conventional Minecraft Java server implementations, much of the simulation is coordinated by the main server thread.

During a tick, the server processes world state: entities, scheduled block updates, plugin or mod callbacks, and other required tasks. The exact execution order depends on the implementation and version.

If the critical work takes too long, the server falls behind.

Additional cores are still useful. Networking, packet compression, garbage collection, chunk-related work, and explicitly asynchronous tasks can run on other threads. Extra CPU capacity also reduces competition between the main thread and background activity.

However, additional cores do not automatically shorten a serialized critical path. Doubling the number of cores will not necessarily halve tick time.

Single-core performance matters when the critical thread is computation-bound. Core count matters when parallel work needs additional execution capacity.

This distinction is implementation-dependent. Folia, for example, uses regionized multithreading rather than one conventional world-wide tick thread. Independent regions can run concurrently, but individual busy regions can still become bottlenecks.

See the PaperMC documentation for the distinction between Paper and Folia.

TPS vs. MSPT: what should you measure?

These metrics answer different questions:

TPS ticks per second: Is the server maintaining its target simulation rate?
MSPT milliseconds per tick: How long does tick processing take?

At the standard 20 TPS target:

1,000 milliseconds / 20 ticks = 50 milliseconds per tick

A lightly loaded server does not need to spend the full 50 ms processing each tick. It can finish its work early and wait until the next scheduled tick.

A server that regularly requires more than 50 ms cannot sustain that target indefinitely.

Why average MSPT is not enough

Averages can hide intermittent stalls.

For example, normal ticks may finish quickly while occasional chunk loads, plugin operations, or garbage-collection pauses produce much longer delays.

Record the distribution, not just its mean:

Median MSPT.
p95 MSPT.
p99 MSPT.
Maximum observed tick duration.
The proportion of ticks exceeding 50 ms.
TPS over the same measurement window.

The p99 is the tick duration at or below which approximately 99% of observations fall. It is not the maximum and does not describe every outlier.

Also record how long the test ran. A percentile from a short, quiet sample is not equivalent to the same percentile measured during a sustained production workload.

Stable TPS tells you the server is keeping pace. Tick-time percentiles reveal how consistently it is doing so.

Why identical GHz can produce different server performance

Clock frequency measures cycles per second. It does not measure useful work completed per cycle.

Two processors running at the same frequency can produce different Minecraft tick times because of differences in:

Instructions per clock, or IPC.
Branch prediction.
Cache capacity and latency.
Memory access latency.
Execution resources.
Microarchitecture.
Power and thermal behavior.

Minecraft server workloads often traverse interconnected objects, execute conditional logic, and access data with imperfect locality.

A cache miss can make the processor wait for data. A branch misprediction can discard speculative work. Higher frequency does not eliminate either problem.

IPC is also workload-dependent. A processor does not have one universal IPC value that predicts performance across every game, plugin, or server configuration.

Your world changes the workload

An empty world is not a substitute for a production environment.

Entities, farms, hoppers, redstone, custom mobs, plugins, and player distribution all change the amount and shape of the work.

A generic single-thread benchmark can help shortlist processors. It cannot establish which CPU will deliver the best p99 MSPT on your actual server.

Advertised boost clock vs. sustained effective clock

The maximum boost frequency in a specification is a peak operating condition, not a promise that the CPU will maintain that frequency throughout a long server workload.

Sustained behavior depends on:

Temperature and cooling.
Power limits.
Firmware configuration.
CPU frequency-management policy.
The number of active cores.
Other workloads sharing the machine.
Virtualization and compute-allocation limits.

A short benchmark can finish before the system reaches a stable thermal state. A longer run may reveal reduced frequency or inconsistent CPU availability.

That is why effective clock and MSPT should be measured during the same interval.

What changes on a VPS?

Inside a virtual machine, frequency readings and hardware telemetry can be incomplete or misleading.

The guest may report a nominal frequency without exposing the host’s actual thermal limits, power constraints, or scheduling decisions.

For a Minecraft VPS, inspect more than the advertised CPU model:

Are vCPUs shared or dedicated?
Is compute capacity burstable or sustained?
What limits apply to CPU use?
Is host contention visible through steal-time metrics?
Are repeated performance measurements consistent?

Where available, CPU steal time indicates time during which the virtual CPU was not executing because the hypervisor was servicing other work. Missing or low reported steal time does not prove that every other resource limit is absent.

High MSPT does not automatically mean a slow CPU

The server thread may be executing code, or it may be waiting.

Those situations require different fixes.

Possible causes of Minecraft server lag include:

Expensive synchronous plugin or mod code.
Entity AI and pathfinding.
Large numbers of block, hopper, or redstone updates.
Chunk generation and loading.
Synchronous disk operations.
Lock contention.
Java garbage-collection pauses.
Memory pressure or swapping.
CPU throttling.
Host-level scheduling contention.

A faster CPU can help a computation-bound workload. It may do little for a thread blocked on storage, waiting for another task, or paused by garbage collection.

Identify the bottleneck before buying hardware.

How to investigate server lag with spark

The spark profiler helps connect performance problems to the code executing during a workload.

For a conventional Minecraft server, the Server thread is usually the first thread to inspect.

The spark viewer documentation explains how to navigate threads and call trees.

Capture a profile while the problem is happening. A quiet-period profile will not explain a busy-period slowdown.

Useful commands include:

/spark health
/spark profiler start
/spark profiler stop

If a background profiler is already running, follow the installed version’s instructions for opening or stopping that session.

To investigate slow ticks specifically, spark supports:

/spark profiler start --only-ticks-over 50

This filters profiling to ticks exceeding the specified duration. It is useful for investigating spikes, but a filtered profile should not be treated as a complete description of normal server activity.

Command behavior and availability depend on the installed version and platform. Refer to the official spark command reference.

Keep the profile’s context

A profile without workload information is difficult to compare.

Record:

Server implementation and exact build.
Minecraft version.
Java version.
JVM flags and heap configuration.
World snapshot identifier.
Plugin or mod versions.
Player count.
View distance.
Simulation distance.
Test actions and duration.
Relevant garbage-collection logs.

Collect MSPT statistics, profiles, and GC events over matching time windows.

A call tree shows where samples were attributed. It does not automatically prove that every delay was active CPU execution. Interpret it alongside the profiler mode, thread state, and operating-system telemetry.

How to benchmark a Minecraft server CPU properly

A meaningful comparison changes the CPU or hosting environment while keeping the workload as consistent as possible.

Use identical copies of the world and match the server build, Java runtime, heap size, plugins, mods, and configuration.

1. Define the target before testing

“Runs Minecraft well” is not a measurable requirement.

An example acceptance target could be:

Maintain 20 TPS with p99 MSPT below 45 ms during the normal-load scenario, without sustained tick backlog during the heavy-load scenario.

The 45 ms threshold is an example, not a universal standard. Choose a target that matches your service requirements and leaves appropriate headroom.

2. Separate workload scenarios

Test these independently:

Idle: Background activity and loaded chunks without players.
Normal load: Representative players, farms, entities, and plugin activity.
Chunk generation: A controlled workload involving new terrain.
Heavy event: The largest legitimate event expected in production.

Do not combine all scenarios into one average. Each can expose a different bottleneck.

Player count alone is insufficient. Fifty players exploring separate areas may create a different workload from fifty players gathered around one complex farm.

3. Warm up the server

Allow the JVM and hardware to reach a reasonably stable operating state.

Warm-up matters because JIT compilation, cache state, temperature, and power behavior can affect early results.

Use the same warm-up policy for every candidate.

4. Record measurements together

During each measured interval, capture:

Tick-time distribution.
TPS.
Main-thread profile.
GC events.
CPU utilization.
Effective clock, where available.
Temperature and power-limit information, where available.
CPU steal time on virtual machines.
Storage latency when I/O is suspected.

On Linux, the following commands can help record machine configuration:

lscpu -e=CPU,CORE,SOCKET,NODE,ONLINE > topology.txt
pgrep -a -x java

Identify the correct server process before setting its PID:

JAVA_PID=12345  # Replace with the actual server process ID

taskset -pc "$JAVA_PID" > affinity.txt
cpupower frequency-info > governor.txt
sudo timeout 180 turbostat --interval 1 > turbo.txt 2>&1

In this form, taskset -pc reports the affinity mask without changing it.

These commands require the relevant utilities to be installed. turbostat needs supported hardware and appropriate permissions. Useful telemetry may be unavailable inside a VPS.

Do not interpret missing sensor output as evidence that thermal or power limits are irrelevant.

5. Repeat the tests

Use at least three warmed-up repetitions as a starting point, with longer or additional runs when variability remains high.

Alternate the order of candidates where practical.

Restore the same initial world state when a scenario modifies it. Otherwise, the second candidate may receive a materially different workload.

Preserve raw results, not only screenshots.

6. Compare latency tails and consistency

A CPU with lower average MSPT may still produce worse p99 latency.

Compare:

Performance in each scenario.
Run-to-run variation.
Frequency of over-budget ticks.
Severity and duration of stalls.
Cost and available resource headroom.

Do not claim a meaningful improvement from a small difference that falls within normal measurement variability.

How many CPU cores does a Minecraft server need?

There is no universal answer.

A conventional server benefits from strong sustained per-core performance and enough additional capacity for background work.

A balanced configuration provides:

Fast execution of the critical tick-processing work.
Additional cores for parallel and background tasks.
Sufficient RAM without host memory pressure.
Consistent storage latency.
Cooling and power limits suitable for sustained load.

More cores can also help when running multiple independent server instances on the same machine.

However, a large core count does not guarantee that one overloaded main thread will run faster.

For shared hosting, the allocation policy matters as much as the advertised vCPU count.

When a CPU upgrade will not fix the problem

A CPU upgrade is most likely to help when profiling shows sustained computation on the critical thread.

Other findings suggest different actions:

Blocking plugin operations: Inspect or replace the relevant plugin.
Slow chunk generation: Evaluate pre-generation and workload limits.
Storage stalls: Investigate I/O latency and synchronous operations.
GC pauses: Inspect allocation behavior, heap configuration, and GC logs.
Lock contention: Investigate synchronization and plugin architecture.
Host contention: Compare compute guarantees and repeated measurements.
High network latency with healthy ticks: Investigate the network separately.

These are diagnostic directions, not automatic fixes. Confirm the cause with evidence.

More RAM is not a universal solution either. It helps when insufficient memory is part of the problem; it does not directly accelerate a computation-bound server thread.

Choosing the best CPU for your Minecraft server

Use single-thread benchmarks to narrow the shortlist, then test candidates on a representative world.

For published comparisons, look for:

Exact software versions.
World and workload details.
Player behavior, not only player count.
Java and memory configuration.
Test duration and repetitions.
Thermal conditions.
Tick-time distributions.

A screenshot showing 20 TPS on an unspecified world is not a meaningful capacity benchmark.

The best CPU for a Minecraft server is the one that delivers consistent tick times on your workload, within your budget not necessarily the one with the highest advertised GHz.

Frequently asked questions

Does Minecraft use only one CPU core?

No. Minecraft servers can use multiple threads for networking, garbage collection, chunk-related work, and background tasks.

However, conventional implementations coordinate substantial simulation work through a main thread. Regionized implementations such as Folia use a different model.

Is a 5 GHz CPU good for a Minecraft server?

Potentially, but frequency alone is insufficient.

Architecture, cache, memory behavior, sustained clock speed, compute allocation, and the workload all affect performance.

Measure the server instead of judging only the specification.

What is a good MSPT?

For a 20 TPS target, the nominal budget is 50 ms per tick.

Lower processing times provide headroom, but examine the distribution as well as the average. Occasional long ticks can still cause noticeable disruption.

Why does my server lag with low overall CPU usage?

Aggregate CPU utilization averages activity across the available processors.

One critical thread can saturate a logical CPU while much of the machine remains idle. Alternatively, the thread may be waiting rather than executing.

Inspect per-thread activity and profile the workload.

Should I measure TPS or MSPT?

Use both.

TPS shows whether simulation is keeping pace. MSPT describes processing time, while profiling and supporting telemetry help explain the cause of slow ticks.

Conclusion

Clock speed contributes to Minecraft server performance, but it is not a complete performance metric.

Reliable diagnosis requires a representative world, repeatable scenarios, warmed-up measurements, tick-time percentiles, and profiles captured during the actual problem.

First determine whether the critical thread is computing, waiting, or being paused. Then decide whether the right change is a faster CPU, better resource guarantees, a plugin fix, memory tuning, or storage work.

That approach avoids paying for a higher GHz number while leaving the real bottleneck untouched.

References

Ollama vs vLLM for Concurrent Inference: How to Choose the Right Server

Aeza — Thu, 03 Sep 2026 14:13:06 +0000

Choosing between Ollama and vLLM is not simply a matter of asking:

Which inference engine is faster?

The answer depends heavily on the workload.

A personal chat interface with one active user has very different requirements from a multi-user API handling dozens of concurrent generations.

For one workload, simple model management may matter more than maximum throughput.

For another, the deciding factors may be:

continuous batching
queue behavior
KV-cache utilization
TTFT
p99 latency
GPU saturation
overload handling

That means a useful Ollama vs vLLM comparison cannot be reduced to a single tokens/s number.

In this article, we'll build a practical benchmark methodology for comparing the two under real concurrent load.

By the end, you'll know:

when vLLM's batching can provide a measurable advantage
why Ollama may still be the better choice for smaller workloads
how to build a fair concurrency sweep
which latency and throughput metrics actually matter
how to detect queueing and saturation
how to choose the server that stays inside your SLO

The Short Answer

There is no universal concurrency level at which vLLM suddenly becomes better than Ollama.

vLLM begins to gain an advantage when enough requests are active or waiting for its scheduler to form useful batches while the GPU still has sufficient compute and memory capacity.

Conceptually:

Low concurrency
      ↓
Little opportunity for batching
      ↓
Ollama may be completely sufficient

Higher concurrency
      ↓
More simultaneous requests
      ↓
Continuous batching becomes useful
      ↓
vLLM can improve aggregate throughput

Too much concurrency
      ↓
Queue grows
      ↓
TTFT and p99 increase
      ↓
OOM / timeout / rejected requests

The real question is therefore not:

Which engine reaches the highest tokens per second?

It is:

Which engine delivers the required throughput while keeping latency, errors, memory usage, and operational complexity inside acceptable limits?

Start with the Workload, Not the Inference Engine

Before testing either server, define what the service actually needs to handle.

Three common scenarios illustrate why this matters.

Personal Chat

A personal chat usually has:

one active user
low concurrency
occasional model loading
relatively tolerant startup latency
no large server-side request queue

In this environment, maximum aggregate GPU throughput may not matter very much.

Operational simplicity often matters more.

Small Internal API

An internal API may receive:

several simultaneous users
short request bursts
varying context lengths
occasional concurrency spikes
moderate latency requirements

Both Ollama and vLLM may fit this workload.

The correct choice depends on the actual traffic curve.

High-Concurrency Inference Service

A public or heavily used internal service may have:

many simultaneous requests
sustained queues
strict TTFT targets
p95/p99 latency requirements
high GPU utilization targets
overload conditions that must be handled predictably

This is where continuous batching and queue-aware scheduling become much more important.

Record the Workload Before Benchmarking

At minimum, record:

Input tokens — determine prefill cost.
Output tokens — determine how long a request occupies generation resources.
Average concurrency — represents normal load.
Peak concurrency — reveals saturation behavior.
Streaming ratio — affects user-perceived latency.
Context length — directly affects KV-cache consumption.
Request bursts — determine queue behavior.
Target SLO — defines when performance becomes unacceptable.

Long contexts consume more KV-cache.

Long generations keep execution resources occupied for longer.

Average concurrency can also hide dangerous workload patterns.

For example, imagine a service that usually has four active requests.

That sounds moderate.

But if several long conversations overlap at the same moment, the system may suddenly experience:

KV-cache pressure
+
longer queue
+
higher TTFT
+
higher p99

even though average concurrency still looks reasonable.

Important: Higher server throughput does not automatically mean a better user experience.

A configuration can produce more total tokens per second while making individual requests noticeably slower.

Why Ollama Is Attractive for Smaller Workloads

Ollama combines several parts of the local model lifecycle into one workflow:

model downloading
model storage
Modelfiles
configuration
local HTTP API
model loading
model unloading

For a developer or small team, this can make deployment and maintenance significantly easier.

You can replace a model quickly, modify the system prompt, and control how long a model remains loaded without building a large serving stack.

Cold-start performance should still be evaluated separately from warm inference.

Important Ollama Parameters

In a warmed-up environment, Ollama behavior depends on settings such as:

OLLAMA_NUM_PARALLEL
OLLAMA_MAX_QUEUE
OLLAMA_MAX_LOADED_MODELS
keep_alive
context size

Parallel requests increase memory consumption because each active request may require additional context state.

A configuration that works perfectly with short prompts may behave very differently with longer contexts.

For example:

Short context
+
4 parallel requests
=
works normally

may become:

Long context
+
4 parallel requests
=
larger memory usage
+
queue growth
+
possible failure

So do not treat parallelism as an isolated setting.

Context length matters too.

Ollama's Main Advantage

For many workloads, Ollama's strongest advantage is not peak benchmark performance.

It is operational simplicity.

That can be extremely valuable for:

personal assistants
development environments
prototypes
small internal APIs
teams that frequently change models

If the workload does not create enough concurrency for continuous batching to matter, a more complicated serving stack may provide little practical benefit.

Why vLLM Benefits from Concurrency

vLLM is designed around server-side inference workloads.

Its scheduler can continuously adjust active batches as requests arrive and complete.

Instead of waiting for one static batch to fully finish, the engine can use newly available capacity for other requests.

This is commonly referred to as continuous batching.

Conceptually:

Request A ────────────────>
Request B      ────────────────>
Request C           ────────────────>

Scheduler continuously updates
the active batch as requests
arrive and finish.

This becomes useful when:

several requests are active
more requests are waiting
the GPU supports the required backend
enough memory remains available
batching improves accelerator utilization

With one short request, there may simply be nothing useful to batch.

Important vLLM Parameters

Record the server configuration with every benchmark.

Important settings include:

--max-num-seqs
--max-model-len
--gpu-memory-utilization
--dtype

Also record whether chunked prefill is enabled and how it is configured.

For multi-GPU deployments, record:

--tensor-parallel-size

Tensor parallelism can increase the effective memory available for larger models, but it also introduces communication between GPUs.

That communication is not free.

So:

More GPUs

does not automatically mean:

Linear performance scaling

Monitor vLLM's Server-Side Behavior

In addition to normal application metrics, monitor:

request queue length
KV-cache utilization
GPU memory utilization
preemption
rejected requests
OOM events

Both Ollama and vLLM should also be monitored for:

queue growth
memory pressure
timeouts
failed requests

The important difference is that vLLM exposes more of the machinery involved in high-concurrency serving.

Which Server Is Easier for One User?

For a single user, Ollama will usually be easier to operate.

The model lifecycle and API are integrated into one workflow.

vLLM becomes more attractive when you specifically require:

high concurrent throughput
continuous batching
detailed production metrics
specific GPU backends
larger request queues
a workload expected to grow rapidly

The decision should follow the workload rather than the popularity of either tool.

Make the Benchmark Fair

A benchmark only makes sense if both systems are processing equivalent work.

This sounds obvious.

In practice, it is easy to get wrong.

Try to keep the following equivalent:

model weights
model architecture
precision
quantization
tokenizer
chat template
generation parameters

Different weight formats may require different inference paths.

For example:

GGUF

and:

safetensors

do not automatically represent identical runtime behavior.

If exactly the same model build cannot be used by both engines, document that clearly.

The benchmark conclusion then applies only to the tested configurations.

Record Model Sources and Hashes

For reproducibility, save:

Model name
Model version
Weight source
File hash
Quantization
dtype

Otherwise, a future benchmark may use slightly different weights and produce results that appear inconsistent.

Keep Generation Parameters Identical

Use the same values for parameters such as:

max_tokens
temperature
top_p
stop
stream

For deterministic benchmark workloads, you may use:

temperature: 0

But matching generation settings is still not enough.

Verify Tokenization and Chat Templates

The same chat payload can produce different token sequences if the two servers use different tokenizers or templates.

For example:

{
  "messages": [
    {
      "role": "user",
      "content": "Explain TCP slow start."
    }
  ]
}

may be transformed differently before reaching the model.

That means two systems may appear to be benchmarking the same request while actually processing different input lengths.

Compare input-token counts before trusting the results.

If one server sees:

312 input tokens

and another sees:

356 input tokens

the workload is not identical.

Record the Benchmark Environment

Every run should include:

Inference server version
Model name
Model hash
GPU model
VRAM
CPU
RAM
Operating system
Driver version
Context length
Generation settings

A result such as:

8,200 output tokens/s

has little value without the environment that produced it.

Separate Cold and Warm Performance

Cold-start performance and steady-state inference answer different questions.

Measure them separately.

Cold Test

A cold test includes:

model loading
initialization
first request latency

This matters for workloads where models are frequently unloaded.

Warm Test

A warm test measures:

steady-state throughput
TTFT
token generation
latency
concurrency scaling

Do not combine cold and warm measurements into one average.

They describe different behavior.

Handle Prompt Caching Consistently

Prompt caching can significantly change benchmark results.

Either:

disable prompt caching on both systems

or:

use it consistently on both systems

Do not allow one server to reuse cached prompt state while the other processes every prompt from scratch.

Make Sure the Load Generator Is Not the Bottleneck

Your client must have enough:

CPU
network bandwidth
connection capacity

to generate the intended load.

Otherwise, you may accidentally benchmark the load generator rather than the inference server.

Monitor the client machine during heavy tests too.

Build a Concurrency Sweep

A simple benchmark matrix could look like this:

input_tokens:
  - 256
  - 2048

output_tokens: 128

concurrency:
  - 1
  - 2
  - 4
  - 8
  - 16
  - 32

repeats: 5
stream: true
temperature: 0

The exact values should match your real workload.

If production requests usually have:

8k input tokens
+
1k output tokens

then a benchmark using:

256 input tokens
+
64 output tokens

may tell you very little about production behavior.

Increase Concurrency Until Saturation

Start at:

concurrency = 1

Then increase gradually.

For every level:

warm up the server
execute several repeated runs
collect latency
collect throughput
collect errors
collect queue metrics
record GPU utilization
record GPU memory usage

Continue until you reach the first unacceptable condition.

Examples include:

OOM
Timeout
HTTP 503
Rejected request
p99 > SLO
GPU memory exhaustion
Queue growth without recovery

That point is much more useful than the absolute maximum throughput.

Closed-Loop vs Open-Loop Testing

There are two common ways to generate concurrent load.

They measure different things.

Closed-Loop Load

A closed-loop generator maintains a fixed number of active requests.

For example:

Concurrency = 16

Whenever one request finishes, another starts.

This answers questions such as:

How does the server behave with 16 continuously active clients?

Open-Loop Load

An open-loop generator sends requests at a defined arrival rate.

For example:

20 requests/second

Requests continue arriving regardless of how quickly previous requests finish.

This is useful for exposing queueing and overload behavior.

If the server can process:

15 requests/second

while receiving:

20 requests/second

then approximately:

5 requests/second

are being added to the queue.

The queue may continue growing until:

latency becomes unacceptable
requests time out
requests are rejected
memory is exhausted

Do not combine closed-loop and open-loop measurements into a single curve.

They answer different questions.

Throughput Alone Is Not Enough

Aggregate tokens per second describes server capacity.

It does not fully describe the user experience.

For every concurrency level, collect:

requests per second
output tokens per second
TTFT
ITL
TPOT
end-to-end latency
successful-request rate
queue length
memory usage

You need both throughput and latency.

TTFT: Time to First Token

TTFT measures:

Request sent
      ↓
First actual generated token

Do not automatically treat the first streaming chunk as the first token.

Some APIs may send metadata or other fields before actual generated text appears.

Measure the first real generated output.

TTFT strongly affects perceived responsiveness in chat applications.

A user may tolerate a long generation if text begins appearing quickly.

A long silent wait before the first token often feels much worse.

ITL: Inter-Token Latency

ITL describes the delay between generated tokens.

Conceptually:

Token 1
   ↓ 40 ms
Token 2
   ↓ 43 ms
Token 3

This describes how smoothly the response streams after generation begins.

If the client measures network chunks rather than actual tokens, call the measurement:

inter-chunk latency

instead.

TPOT: Time per Output Token

TPOT can be represented conceptually as:

generation time after first token
---------------------------------
remaining output tokens

It helps separate initial responsiveness from steady-state generation speed.

Measure End-to-End Latency

End-to-end latency measures:

Request sent
      ↓
Queueing
      ↓
Prefill
      ↓
Generation
      ↓
Final response

This is often the metric most directly connected to the total user wait time.

Always Measure Tail Latency

Do not report only the average.

At minimum, report:

p50
p95
p99

Why?

Because batching and queueing may affect a minority of requests much more severely than the median request.

Suppose:

p50 = 1.8 s
p95 = 4.2 s
p99 = 12.7 s

An average of:

2.4 s

would hide an important part of the user experience.

For production inference APIs, p99 often matters much more than the mean.

Understand the Throughput-Latency Trade-Off

Continuous batching can improve aggregate throughput.

But that does not mean every request becomes faster.

Consider:

Concurrency 4
Throughput: 3,000 tok/s
p99 TTFT: 400 ms

and:

Concurrency 32
Throughput: 7,500 tok/s
p99 TTFT: 4.8 s

The second configuration produces much more aggregate throughput.

But it may violate the application's SLO.

If your requirement is:

p99 TTFT < 2 seconds

then the higher-throughput operating point is not acceptable.

The best benchmark result is therefore not necessarily the highest point on the throughput curve.

It is the highest useful operating point inside the SLO.

Measure Queueing Explicitly

Queueing is one of the most important parts of concurrent inference.

Imagine requests arriving faster than they can be processed:

Arrival rate
     ↓
[Request]
[Request]
[Request]
[Request]
     ↓
Server

If the server cannot keep up:

Queue length ↑
TTFT ↑
p99 ↑
timeouts ↑

This is why a large or unlimited queue does not solve overload.

It only changes:

rejected request

into:

very slow request

Backpressure and overload behavior should therefore be part of the benchmark.

Test Fairness Between Short and Long Requests

A realistic workload rarely contains identical prompts.

Create a mixed workload with:

short inputs
long inputs
short outputs
long outputs

Then check whether short requests are excessively delayed by long ones.

For example:

Request A:
256 input tokens
64 output tokens

Request B:
8,000 input tokens
2,000 output tokens

A scheduler should not allow a large request to make every small request unreasonably slow.

Also observe whether large prefill operations create visible pauses for requests already generating tokens.

Store Raw Benchmark Data

A professional benchmark should produce reproducible artifacts.

For example:

benchmark/
├── requests.jsonl
├── run.json
├── results.csv
├── server.log
└── gpu.csv

A minimal result schema could look like:

server,run,concurrency,input_tokens,output_tokens,ttft_ms,itl_ms,e2e_ms,status,error

You may also want:

timestamp,gpu_utilization,gpu_memory_used,queue_length

Screenshots alone are not sufficient.

Raw data allows you to:

recalculate percentiles
change aggregation windows
inspect outliers
compare future server versions
reproduce the analysis

Graph Throughput and Latency Together

A useful benchmark graph uses concurrency on the X-axis.

Then plot metrics such as:

Requests/s
Output tokens/s
TTFT p50
TTFT p95
TTFT p99
End-to-end p99
Queue length

The important shape often looks like:

Throughput
    /
   /
  /
 /_____
       \
        saturation

Latency
        /
       /
______/

At low concurrency:

throughput increases
latency remains stable

Near saturation:

throughput growth slows
queueing increases
p99 rises rapidly

That knee in the curve is often more important than the absolute maximum.

Compare API Compatibility Before Migrating

Both Ollama and vLLM may expose OpenAI-compatible interfaces.

But:

OpenAI-compatible

does not mean:

behaviorally identical

Verify the features your application actually uses.

For example:

Chat Completions
streaming
structured output
embeddings
tool calling
generation parameters
stop sequences
error responses
token limits

Do not discover incompatibilities after production traffic has already been switched.

Production Readiness Is More Than the Inference Engine

A working HTTP endpoint is not automatically a production-ready inference service.

The surrounding platform may need to provide:

TLS
Authentication
Rate limiting
Logging
Monitoring
Health checks

A health check should also distinguish between:

Server process is running

and:

Model is loaded and ready to serve

A listening TCP port does not guarantee that the next request will meet the latency SLO.

Compare Observability

For concurrent production workloads, observability becomes part of the product decision.

Useful vLLM signals may include:

queue state
KV-cache utilization
TTFT
request latency
memory pressure
preemption

Ollama also exposes timing information around:

model loading
prompt evaluation
generation

Regardless of the server, collect the same client-side metrics.

Client-side instrumentation provides the most direct view of what users actually experience.

Test Upgrades Before Production

Inference-server updates can change:

memory consumption
scheduler behavior
inference backends
configuration defaults
chat templates
model compatibility

Before upgrading production, replay a realistic workload against the new version.

Keep the previous:

Container image or package
Configuration
Model hash

available for rollback.

A benchmark that is valid for one version should not automatically be assumed valid for the next.

Use a Shared API Layer to Keep Migration Easy

If possible, avoid coupling application logic directly to one inference engine.

Keep:

model names
generation parameters
timeout logic
error handling

behind an internal adapter.

Then make the server endpoint configurable.

Conceptually:

Application
     ↓
Internal inference adapter
     ↓
Ollama OR vLLM

This makes it much easier to:

benchmark alternatives
canary a new server
roll back
migrate later

Canary the Migration

Do not send 100% of production traffic to the new server immediately.

Start with a small percentage.

For example:

5% new server
95% old server

Measure:

latency
errors
throughput
queue growth
memory usage

Then increase gradually if the results remain inside the SLO.

Keep the old server available for rapid rollback.

A Practical Benchmark Workflow

Here is a reusable process.

Step 1: Define the Real Workload

Record:

Input length
Output length
Average concurrency
Peak concurrency
Request rate
Streaming ratio
SLO

Step 2: Normalize the Models

Match:

Weights
Precision
Quantization
Tokenizer
Chat template

Step 3: Match Generation Settings

Use identical:

max_tokens
temperature
top_p
stop
stream

Step 4: Record the Environment

Save:

Server version
Model hash
GPU
VRAM
CPU
RAM
OS
Driver

Step 5: Warm Up the Server

Separate cold-start measurements from steady-state results.

Step 6: Run the Concurrency Sweep

For example:

Step 7: Measure Latency and Throughput

Collect:

Requests/s
Tokens/s
TTFT
ITL
TPOT
E2E latency

Step 8: Measure Tail Latency

Calculate:

p50
p95
p99

Step 9: Watch Queue and GPU State

Collect:

Queue length
GPU utilization
GPU memory
KV-cache
OOM events

Step 10: Stop at the SLO Boundary

The benchmark should stop being considered "better" once:

p99 > SLO

even if tokens per second continue increasing.

Starting Recommendation by Scenario

Single User

Start with:

Ollama

Confirm:

the model is supported
cold start is acceptable
memory consumption is acceptable
latency is acceptable

The simplicity may be worth more than extra concurrent throughput.

Small Internal API

Start with:

Ollama or vLLM

Benchmark:

request bursts
moderate concurrency
queue behavior
p95/p99
overload handling

Either system may be the better fit.

High-Concurrency GPU Service

Start with:

vLLM

Then confirm that:

throughput improves with concurrency
p99 remains inside the SLO
TTFT remains acceptable
KV-cache remains healthy
queue growth remains controlled
OOM does not occur

Do not assume the answer before measuring it.

Practical Checklist

Before declaring one server faster than the other:

[ ] Use comparable model weights
[ ] Match precision and quantization
[ ] Match tokenizer behavior
[ ] Match chat templates
[ ] Match generation parameters
[ ] Record software versions
[ ] Record GPU and VRAM
[ ] Separate cold and warm tests
[ ] Use realistic input lengths
[ ] Use realistic output lengths
[ ] Sweep concurrency
[ ] Test open-loop and closed-loop load separately
[ ] Measure requests per second
[ ] Measure output tokens per second
[ ] Measure TTFT
[ ] Measure ITL or TPOT
[ ] Measure end-to-end latency
[ ] Calculate p50/p95/p99
[ ] Monitor queue length
[ ] Monitor GPU utilization
[ ] Monitor GPU memory
[ ] Record OOM and rejected requests
[ ] Preserve raw benchmark data
[ ] Evaluate results against the SLO

Final Takeaway

Ollama and vLLM solve overlapping problems, but they optimize for different priorities.

Ollama is attractive when you value:

fast setup
simple model management
a straightforward local lifecycle
moderate concurrency
lower operational complexity

vLLM becomes more attractive when you need:

continuous batching
higher concurrent throughput
efficient KV-cache management
high GPU utilization
larger request queues
detailed production observability

The correct comparison is not:

Which server produces the most tokens per second?

It is:

Which server delivers the throughput you need while keeping TTFT, p99 latency, queueing, errors, memory usage, and operational complexity inside acceptable limits?

If vLLM produces higher aggregate throughput but pushes p99 beyond your SLO, that is not a win.

If Ollama already handles your real traffic comfortably, migrating to a more complex serving stack may provide little practical benefit.

Benchmark the workload you actually have.

Find the saturation point.

Measure the queue.

Measure the tails.

Then choose the simplest system that satisfies the SLO.

CPU Steal Time Explained: What %st Really Means on a Virtual Machine

Aeza — Thu, 03 Sep 2026 14:11:28 +0000

CPU steal time is one of those Linux metrics that looks simple until you actually need to diagnose a performance problem.

You open top, notice that %st is above zero, and the obvious conclusion seems to be:

The host is overloaded.

Or perhaps:

Another VM on the server is stealing my CPU.

Both explanations are possible.

Neither can be proven from %st alone.

A useful diagnosis requires at least three things:

the steal-time measurement itself
CPU pressure inside the VM
measurable impact on the application

In practice, that means correlating %st with metrics such as the run queue, p95/p99 latency, throughput, per-vCPU utilization, and cgroup throttling.

In this article, we'll build a practical workflow for doing exactly that.

By the end, you'll know:

what %st actually measures
how to collect steal-time data correctly
when high steal time is worth investigating
how to separate host contention from guest CPU saturation
what evidence to collect before contacting your hosting provider
how to monitor steal time continuously

The Short Answer

%st represents time during which a virtual CPU was ready to run but did not receive physical CPU time from the host.

But a high value only becomes operationally meaningful when it is:

abnormal for that specific VM
sustained across multiple measurement intervals
correlated with CPU pressure or application degradation

A single spike is not enough.

A much stronger signal looks like this:

Elevated %st
+
Growing CPU run queue
+
Increasing p99 latency
+
Falling throughput

That combination is worth investigating.

What Does `%st` Mean in `top` and `vmstat`?

In a virtual machine, %st — or steal time — represents the percentage of a measurement interval during which a vCPU was ready to execute but did not receive physical CPU time.

You may see it in top:

%Cpu(s): 12.7 us, 6.4 sy, 0.0 ni, 72.2 id,
          3.1 wa, 0.0 hi, 0.0 si, 5.6 st

In this example:

5.6 st

means that approximately 5.6% of the measured CPU interval was accounted as steal time.

The important part is understanding why this can happen.

A VM does not control the physical CPU directly.

There is another scheduling layer underneath it.

A Virtual Machine Has Two CPU Schedulers

To understand steal time, separate the guest scheduler from the host scheduler.

The guest scheduler

The operating system inside the VM decides:

Which process should run on this vCPU?

For example:

nginx
postgres
python
node

may all compete for CPU time inside the guest.

The host scheduler

The physical host or hypervisor decides something different:

When does this vCPU get access to an actual physical CPU?

That gives us two scheduling layers:

Application process
        ↓
Guest OS scheduler
        ↓
       vCPU
        ↓
Host / hypervisor scheduler
        ↓
Physical CPU

Suppose the guest has a runnable process.

The guest scheduler wants to execute it.

The vCPU is therefore ready.

But the host scheduler does not immediately schedule that vCPU onto a physical core.

The guest waits.

That waiting time can be recorded as steal time.

Conceptually:

Guest has runnable work
        ↓
vCPU is ready
        ↓
Physical CPU is not assigned
        ↓
Guest waits
        ↓
Time is recorded as steal

This is why %st applies to the vCPU as a whole rather than to one particular process.

Do Not Confuse `%st` with Other CPU Metrics

Several CPU metrics can indicate performance pressure, but they describe different things.

`%usr`

Time spent executing user-space code.

Examples:

application logic
JavaScript
Python
database execution
compression
model inference

`%sys`

Time spent executing kernel code.

`%idle`

CPU time during which the guest had no runnable work.

`%iowait`

Time associated with waiting while I/O is pending.

`%st`

Time during which a runnable virtual CPU did not receive physical CPU time.

A useful mental model is:

%usr / %sys → guest is actively using CPU

%iowait     → work is waiting around I/O

%st         → guest wanted CPU but the host did not provide it

%iowait is therefore not another form of steal time.

CPU Throttling Is Also Not the Same as Steal Time

A process can be CPU-limited inside the VM even when %st is zero.

One common cause is a cgroup CPU quota.

With cgroup v2, inspect the configured quota:

cat /sys/fs/cgroup/cpu.max

Then inspect CPU statistics:

cat /sys/fs/cgroup/cpu.stat

Useful counters can include:

nr_throttled
throttled_usec

If these counters increase while the workload is running, the process or cgroup may be hitting a CPU quota.

That means the CPU restriction originates inside the guest environment, not necessarily from hypervisor scheduling.

So this situation is possible:

%st = 0

while application latency still increases because the process is being throttled.

This distinction matters when trying to determine whether the VM itself is overloaded or whether the underlying platform is contributing to the problem.

A Zero `%st` Does Not Prove the Host Is Healthy

Another common mistake is assuming:

%st = 0

means:

No host-side scheduling problems exist

That conclusion is too strong.

A zero value only tells you that the guest kernel did not account for steal time during that particular interval.

The availability of steal-time accounting depends on factors such as:

hypervisor
guest kernel
architecture
platform configuration

KVM can expose steal time on supported systems, but the guest still needs access to the relevant accounting information.

Other forms of host-side delay may also exist without appearing as %st.

So think of zero as:

No steal time was recorded here.

Not:

The physical host is definitely healthy.

Measure Steal Time over Intervals

One of the easiest ways to misread steal time is to use the wrong measurement window.

For example, the first row printed by vmstat can represent averages since system boot unless you suppress it.

A since-boot average is nearly useless when investigating a short incident.

Imagine that your VM has been running for 30 days.

A two-minute scheduling problem may almost disappear inside the long-term average.

For incident analysis, collect interval-based measurements instead.

Collect One Minute of `vmstat`

A useful starting point is:

TZ=UTC LC_ALL=C vmstat -y -t 1 60 | tee vmstat.txt

This gives you:

1 sample per second
×
60 samples

Using -y skips the misleading initial since-boot row.

The output allows you to observe steal time together with metrics such as:

r
us
sy
id
wa
st

during exactly the same intervals.

Collect Per-vCPU Data with `mpstat`

A system-wide CPU average can hide an important detail.

Suppose a four-vCPU machine looks like this:

CPU 0 → heavily delayed
CPU 1 → mostly idle
CPU 2 → mostly idle
CPU 3 → mostly idle

The average across all CPUs may look relatively normal.

To inspect individual virtual CPUs, use:

TZ=UTC LC_ALL=C mpstat -P ALL 1 60 | tee mpstat.txt

This lets you compare each vCPU separately.

That becomes especially important for workloads that rely heavily on a small number of threads.

What Does the `r` Column in `vmstat` Mean?

The r field represents runnable tasks.

Conceptually:

r =
tasks currently executing
+
tasks ready and waiting for CPU

It is not an exact queue size for each individual CPU core.

But it is useful as a CPU-pressure signal.

Consider these two situations.

Situation A

%st: brief spike
r: normal
p99: normal
throughput: normal

There may be nothing operationally important happening.

Situation B

%st: elevated
r: increasing
p99: increasing
throughput: falling

Now the evidence is much stronger.

The second case tells us that:

the VM is waiting for CPU
runnable work is accumulating
users are experiencing measurable degradation

That is the kind of correlation worth investigating.

Record the Environment with Every Incident

Raw performance numbers are much less useful without environmental context.

At minimum, collect the timestamp:

date -u --iso-8601=seconds

Kernel version:

uname -r

Number of processors visible to the guest:

nproc

Virtualization type:

systemd-detect-virt

And hypervisor information where available:

LC_ALL=C lscpu | sed -n '/Hypervisor vendor/p;/Virtualization type/p'

Store at least:

UTC timestamp
Kernel version
Number of vCPUs
Virtualization type
VM plan
CPU limits
Region

This matters because two results such as:

%st = 8%

may describe completely different environments.

An 8% reading on a one-vCPU burstable instance is not automatically comparable to 8% on an eight-vCPU VM with a different CPU policy.

Establish a Baseline Before Calling `%st` High

There is no universal steal-time threshold.

You will sometimes see rules such as:

%st > 5% = bad

or:

%st > 10% = overloaded host

These rules are convenient.

They are also too simplistic.

A better question is:

Is this value abnormal for this VM under a comparable workload?

Build a baseline using measurements collected during:

low-load periods
normal-load periods
comparable hours
comparable weekdays
similar request volumes

Then calculate values such as:

median
p95
p99

for the steal-time series.

The goal is to understand the normal distribution for the machine.

Why the Maximum Is Often a Bad Metric

Suppose you observe:

Maximum %st this month: 35%

That sounds alarming.

But what if the 35% value lasted for one second and nothing happened to the application?

Now compare that with:

%st: 7–10%
Duration: 12 minutes
p99 latency: +70%
Throughput: -20%

The second event may be much more important even though its maximum value is lower.

For performance incidents, persistence and application impact often matter more than the highest isolated value.

Rebuild the Baseline After Major VM Changes

The old baseline may stop being meaningful after changing:

number of vCPUs
kernel version
VM class
service plan
CPU limits

For example, moving from:

2 vCPU

to:

8 vCPU

changes thread distribution and the way aggregate CPU statistics should be interpreted.

Treat significant configuration changes as the beginning of a new baseline period.

When Should You Investigate High Steal Time?

A useful rule is to investigate when all three conditions begin to appear.

1. `%st` is above normal

Not simply above an arbitrary internet threshold.

It should be above the historical level for that VM.

2. The condition persists

Several consecutive elevated intervals are more important than a single isolated sample.

3. Something measurable becomes worse

For example:

CPU run queue increases
p95 latency increases
p99 latency increases
throughput decreases
errors increase

A weak signal might look like:

%st: 12%
Duration: 1 second
p99: unchanged
Throughput: unchanged

A stronger signal might look like:

%st: elevated for 8 minutes
r: increased
p99: +80%
Throughput: -25%

The second case is much more actionable.

Separate Host Contention from Guest CPU Saturation

Steal time does not prevent the guest from also being CPU-bound.

Both can happen at the same time.

Start by examining:

%usr
%sys
%idle
r
%st

Suppose you observe:

High %usr
High %sys
Low %idle
r > number of vCPUs

The guest itself may already be CPU saturated.

That does not mean %st is irrelevant.

It means you may have multiple sources of delay.

Inspect Individual vCPUs

Run:

TZ=UTC LC_ALL=C mpstat -P ALL 1 60

This can expose per-vCPU imbalance that disappears from the average.

For example:

CPU 0 → 100% busy
CPU 1 → 20%
CPU 2 → 15%
CPU 3 → 10%

A single CPU-bound thread may saturate one virtual CPU while the rest remain mostly free.

Inspect Application Threads

Use pidstat to inspect thread-level CPU usage:

TZ=UTC LC_ALL=C pidstat -u -t -p PID 1 60

This helps answer questions such as:

Is one thread consuming nearly all available CPU?
Is CPU usage distributed evenly?
Is the application itself creating the bottleneck?

Only after checking these guest-side causes does it become safer to focus on the host.

Do Not Ignore I/O, Locks, or External Dependencies

High latency does not automatically mean CPU contention.

Other possibilities include:

storage latency
database locking
application locks
garbage collection
network latency
external APIs
DNS
remote databases

So remember:

High latency ≠ automatically CPU contention

And:

High %st ≠ automatically a noisy neighbor

A useful diagnosis should eliminate alternative explanations before assigning a cause.

Correlate `%st` with Application Metrics

The most useful analysis begins when system metrics and application metrics share the same timeline.

Use UTC for everything.

For each interval, compare:

%st
r
request rate
throughput
error rate
p50 latency
p95 latency
p99 latency

This allows you to answer a much better question than:

Was steal time high?

You can ask:

Did application performance become worse during exactly the same period?

Do Not Compare Different Aggregation Windows

Imagine this comparison:

%st → one-second samples

p99 → five-minute window

A CPU scheduling event lasting 30 seconds may disappear almost completely inside a five-minute application aggregate.

The metrics then appear unrelated even when they describe the same incident.

Try to align:

timestamps
collection intervals
aggregation windows

as closely as possible.

A "Noisy Neighbor" Is a Hypothesis, Not a Measurement

One of the most common explanations for steal time is another VM consuming physical CPU resources on the same host.

That is the classic "noisy neighbor" scenario.

It may be correct.

But guest metrics alone cannot tell you:

which VM caused the delay
whether another tenant caused it
whether the host itself was busy
whether a scheduler policy was involved

So instead of saying:

A noisy neighbor caused our outage.

A technically defensible statement would be:

During this interval, the VM experienced increased CPU steal time while the run queue and application latency also increased.

That statement describes what you actually measured.

Host telemetry is needed for stronger attribution.

When Does the Host-Contention Hypothesis Become Stronger?

The hypothesis becomes more convincing when degradation:

repeats under similar traffic
coincides with elevated %st
is not explained by a deployment
is not explained by garbage collection
is not explained by I/O
is not explained by cgroup throttling

Host-side scheduling metrics provide the strongest confirmation.

Store Raw Data, Not Just Screenshots

A useful incident dataset might include:

incident/
├── metadata.txt
├── vmstat.txt
├── mpstat.txt
├── application.csv
└── README.md

A minimal application/system CSV could use:

timestamp_utc,st,r,usr,sys,p95_ms,p99_ms,rps,error_rate

Raw data lets you:

recalculate percentiles
change aggregation windows
inspect individual events
compare multiple incidents
reproduce the analysis later

A screenshot cannot do that.

Once metrics have been compressed into an image, much of the original information is gone.

Test the Hypothesis with Controlled CPU Load

A controlled test can help you understand how the VM behaves under a known CPU workload.

First record the stress-ng version:

stress-ng --version | tee stress-ng-version.txt

Then run a CPU-bound workload pinned to one vCPU:

taskset -c 0 stress-ng \
  --cpu 1 \
  --cpu-method matrixprod \
  --timeout 120s \
  --metrics-brief \
  2>&1 | tee stress-ng.txt

At the same time, collect system metrics:

TZ=UTC LC_ALL=C vmstat -y -t 1 120 | tee vmstat.txt

And:

TZ=UTC LC_ALL=C mpstat -P ALL 1 120 | tee mpstat.txt

Production warning: Do not deliberately saturate CPU on a production VM unless you have an approved test or maintenance window.

A controlled test is useful because one known variable is introduced:

Predictable CPU workload

You can then observe how the VM and its metrics respond.

Compare Like with Like

Do not compare stress-ng numbers from unrelated systems as if they were equivalent.

Try to keep the following constant:

stress-ng version
CPU method
operating-system image
CPU class
vCPU count
CPU limits

Otherwise, differences may come from the test environment rather than from the platform behavior you are trying to measure.

What Can the Hosting Provider See That You Cannot?

Inside the VM, you only see guest-level evidence.

The infrastructure team may have access to:

vCPU thread wait time
host oversubscription
scheduler statistics
physical CPU utilization
vCPU pinning
host events
hypervisor-level contention

Those measurements are much closer to the actual scheduling layer.

If possible, ask the provider to compare its host-side data with the exact same UTC interval from your guest logs.

A Practical Diagnostic Workflow

Here is a simple sequence you can reuse during an incident.

Step 1: Measure `%st`

Collect interval-based vmstat data:

TZ=UTC LC_ALL=C vmstat -y -t 1 60

Step 2: Check the run queue

Look at:

Ask whether runnable work is accumulating.

Step 3: Inspect each vCPU

TZ=UTC LC_ALL=C mpstat -P ALL 1 60

Look for imbalance.

Step 4: Inspect application threads

TZ=UTC LC_ALL=C pidstat -u -t -p PID 1 60

Check whether the guest workload itself is saturating CPU.

Step 5: Check cgroup throttling

cat /sys/fs/cgroup/cpu.stat

Look for increasing throttling counters.

Step 6: Align application metrics

Compare the same interval for:

p95
p99
throughput
error rate
request rate

Step 7: Eliminate alternative causes

Check:

I/O
garbage collection
locks
deployments
network dependencies

Step 8: Ask for host telemetry

If the symptoms still point toward host CPU scheduling, ask the platform team to inspect the corresponding host interval.

How to Write a Useful Support Ticket

Do not start with an accusation.

This is weak:

Your server is overloaded because another customer is stealing our CPU.

You cannot prove that from guest metrics.

Instead, write something measurable.

For example:

Between 10:14 and 10:22 UTC, %st increased significantly relative to the normal baseline for this VM. During the same interval, the CPU run queue increased and application p99 latency exceeded the SLO.

That gives the provider a concrete period and measurable symptoms to investigate.

Include:

VM identifier
region
number of vCPUs
kernel version
virtualization type
exact UTC interval
normal baseline
observed %st
run queue
p95/p99
throughput

Attach your raw data:

metadata.txt
vmstat.txt
mpstat.txt
application.csv
README.md

Before sending anything, remove:

passwords
API keys
access tokens
user data
sensitive request contents

Repeat the Test After Migration or a Plan Change

Suppose the provider migrates the VM to another host.

Or you change:

the VM plan
CPU limits
vCPU count

Do not simply look at the next random workload and declare the problem fixed.

Repeat a comparable test.

Keep the following as consistent as possible:

Duration
Input workload
Number of processes
Traffic profile
Measurement interval

One run is weak evidence.

Several repeated runs are much stronger.

For example:

Before:
%st elevated
p99 elevated
throughput reduced

After migration:
%st lower
p99 restored
throughput restored

If that pattern repeats across several comparable tests, the case for a platform-related effect becomes much stronger.

Monitor Steal Time with Prometheus

If you use node_exporter, CPU steal time can be observed through:

node_cpu_seconds_total{mode="steal"}

A five-minute percentage can be calculated with PromQL:

100 * avg by (instance) (
  rate(node_cpu_seconds_total{mode="steal"}[5m])
)

This converts the cumulative steal-time counter into an approximate percentage over the selected window.

Do Not Alert on a Single Spike

A simplistic alert might look conceptually like:

%st > X

But this ignores:

the VM's normal baseline
duration
user impact

A more useful alerting model is:

%st above expected baseline
AND
condition persists
AND
application impact exists

Application impact might include:

p99 > SLO

or:

run queue increased

or:

throughput decreased

or:

error rate increased

With Prometheus, the for clause can prevent a single temporary spike from creating an incident.

For example:

for: 10m

means the condition must remain true for ten minutes before the alert fires.

The exact duration should depend on the workload and SLO.

Monitor the Monitoring System Too

One subtle problem remains.

Suppose your dashboard shows:

%st = 0

That only means something if the metric is actually being collected.

If the exporter stops reporting the time series entirely, you should not interpret the missing metric as zero.

A reliable monitoring system should therefore distinguish between:

Steal time is zero

and:

Steal-time telemetry disappeared

Monitoring the monitoring pipeline is part of production observability.

The Three-Signal Model

A useful way to think about steal-time incidents is to use three layers.

1. Scheduling signal
        ↓
       %st

2. CPU pressure
        ↓
        r

3. User-visible impact
        ↓
 p95 / p99 / throughput

The strongest diagnosis appears when all three move together.

For example:

%st ↑
r ↑
p99 ↑
throughput ↓

is strong evidence that CPU scheduling pressure is having a measurable effect.

By contrast:

%st spike
r unchanged
p99 unchanged
throughput unchanged

may not justify any operational response at all.

Practical Checklist

When you notice elevated CPU steal time, work through this list.

[ ] Collect %st using interval-based measurements
[ ] Record UTC timestamps
[ ] Check the vmstat run queue
[ ] Inspect individual vCPUs with mpstat
[ ] Inspect application threads with pidstat
[ ] Check cgroup CPU throttling
[ ] Compare p95 and p99 latency
[ ] Compare throughput
[ ] Compare error rate
[ ] Match aggregation windows
[ ] Compare against the VM's baseline
[ ] Check I/O and application bottlenecks
[ ] Preserve raw data
[ ] Request host telemetry if needed
[ ] Repeat the test after migration or configuration changes

Final Takeaway

CPU steal time %st tells you that a runnable vCPU did not receive physical CPU time during part of a measurement interval.

That information is useful.

But it is not a complete diagnosis.

Do not ask only:

Is %st high?

Ask instead:

Is %st persistently abnormal for this VM, and does it coincide with CPU pressure and measurable application degradation?

A solid investigation combines:

interval-based steal-time measurements
CPU run queue
per-vCPU utilization
cgroup throttling
application latency
throughput
historical baseline
host telemetry when available

Without the run queue and application impact, %st remains an observation.

With synchronized system and application metrics, it becomes evidence you can actually use.