DEV Community: Adolfo Pedernera

Database Performance for Developers: Indexing, Query Plans, and the Queries That Don't Scale

Adolfo Pedernera — Fri, 28 Aug 2026 01:17:03 +0000

How database indexes actually work: B-trees, selectivity, composite indexes, covering indexes, and reading EXPLAIN plans so your queries scale past 10 million rows.

The Query That Worked Fine Until It Didn't

Every developer has a story about the query that worked perfectly in development and crawled to a halt in production. The table had 10,000 rows in your local database; in production it has 10 million. The query that took 5 milliseconds now takes 45 seconds. The dashboard times out. The support tickets arrive.

The culprit is almost always the same: a full table scan. The database engine reads every single row in the table, one by one, looking for the ones that match your WHERE clause. On a table with 10 million rows, that means reading millions of disk pages — work that could have been avoided with the right index.

An index is a separate data structure that the database maintains alongside your table. It works like the index at the back of a textbook: instead of reading every page to find every mention of "normalization," you look up the term in the index, which points you to pages 42, 87, and 156. Database indexes do the same thing, but with a twist that catches many developers off guard: an index is only useful if the query planner decides to use it. And the planner's decision depends on a single number: selectivity.

This guide explains how indexes actually work, when they help, when they hurt, and how to read an EXPLAIN plan to understand what your database is doing — so the next query that doesn't scale is one you catch before it reaches production.

How a B-Tree Index Works

Most database indexes are implemented as B-trees (or their variant, the B+-tree). A B-tree is a self-balancing tree structure that keeps data sorted and allows searches, insertions, and deletions in logarithmic time.

Imagine a table with 1 million rows and an index on the email column. Without an index, finding a specific email requires scanning all 1 million rows. With a B-tree index, the database starts at the root node, compares the target value to the keys in the node, and follows the appropriate branch — roughly 20 comparisons for 1 million rows (log₂ of 1,000,000 ≈ 20), each reading one disk page. The difference between reading 1 million pages and 20 pages is the difference between a query that takes seconds and one that takes milliseconds.

The B-tree structure also explains why indexes are sorted: range queries (WHERE age BETWEEN 25 AND 35) can find the starting point in the tree and then scan sequentially through the leaf nodes. A hash index, by contrast, supports only equality lookups — fast for WHERE email = 'x' but useless for ranges.

The cost of indexing is not free. Every index you add slows down INSERT, UPDATE, and DELETE operations because the database must update both the table and every affected index. A table with 5 indexes takes roughly 5× longer to insert into than a table with none. The art of indexing is balancing read speed against write speed — and knowing when an index helps so little that it isn't worth the cost.

Selectivity: The Number That Decides Everything

Selectivity is the fraction of rows a filter keeps. A filter WHERE email = 'a@b.com' on a unique column has selectivity ≈ 1/N (one row out of N). A filter WHERE status = 'active' on a column with two values has selectivity ≈ 0.5 (half the rows). The query planner uses this number to decide: is it cheaper to read the index and then fetch the matching rows, or just scan the whole table?.

The formula for selectivity is straightforward:

S = \frac{R}{N}

where R is the number of rows returned and N is the total rows in the table. A selectivity of 0.00001 (one row in 100,000) means the index is almost certainly used. A selectivity of 0.5 (half the rows) means the planner will likely ignore the index and scan the table directly — reading 500,000 rows via index lookups is more expensive than reading them sequentially.

The SQL Index Selectivity Calculator computes this from your table's row count, the number of distinct values in the indexed column, and the rows your query returns. It also reports the planner's likely choice (index scan vs sequential scan) and the estimated cost of each — useful for deciding whether an index on a given column will actually be used.

The cliff is steep. Going from 100 distinct values (1% selectivity, clearly indexed) to 10 distinct values (10% selectivity, borderline) multiplies selectivity by 10×. Below 100 distinct values on a large table, a single-column index is questionable; above 1,000 it is almost always worthwhile.

Composite Indexes: Order Matters

A composite index (also called a multi-column index) indexes two or more columns together. This is where most developers make their first indexing mistake: they assume the column order doesn't matter. It does.

A composite index on (last_name, first_name) can answer queries that filter on last_name alone, or on both last_name AND first_name. But it cannot answer a query that filters on first_name alone. The reason is the B-tree structure: the tree is sorted by last_name first, then by first_name within each last_name. Searching for first_name = 'John' without knowing the last_name is like searching for everyone named "John" in a phone book sorted by last name — you have to read the whole book.

The rule: put the most selective column first (the one that eliminates the most rows), and put columns used in equality filters before columns used in range filters. A query WHERE status = 'active' AND created_at > '2024-01-01' benefits from an index on (status, created_at) — equality first, range second. Reversing the order to (created_at, status) is less efficient because the planner can use the first column for range scanning but must filter the second column from the results.

Practical example: A users table with 5 million rows, 50,000 distinct cities, and 2 statuses (active/inactive). A query WHERE city = 'Berlin' AND status = 'active':

Index on (city, status): selectivity ≈ 1/50,000 × 0.5 = 0.00001 → index used, ~100 rows fetched
Index on (status, city): selectivity ≈ 0.5 × 1/50,000 = 0.00001 → same final result, but the planner may prefer the more selective column first

The difference becomes dramatic when the first column is not selective. An index on (status, city) forces the planner to scan all 2.5 million active users first, then filter by city. An index on (city, status) jumps directly to Berlin's ~100 users, then filters by status.

Covering Indexes: When the Index Is the Answer

A covering index is an index that contains all the columns a query needs, so the database can answer the query entirely from the index without ever touching the table. This is the fastest possible access path.

Consider a query SELECT user_id, email FROM users WHERE email = 'a@b.com'. With an index on (email), the database finds the row in the index, then fetches the corresponding table row to get user_id. With a covering index on (email, user_id), the database finds everything it needs in the index — no table access at all. On a large table, this can be 10-100× faster because the index is smaller and more cacheable than the table.

The tradeoff: covering indexes are wider (more columns = more bytes per entry), which means fewer entries per disk page and more storage. Use them for your most frequent queries, not for every query. A good candidate is a query that runs thousands of times per minute and returns few columns.

PostgreSQL's INCLUDE clause (available since PostgreSQL 11) lets you add non-key columns to an index without making them part of the sort key: CREATE INDEX ON users (email) INCLUDE (user_id). This gives you the covering benefit without the sorting overhead on the included columns.

Reading EXPLAIN: What the Planner Is Actually Doing

Every database provides an EXPLAIN command that shows the query plan — the step-by-step strategy the planner has chosen. Learning to read these plans is the single most useful skill for database performance.

A typical PostgreSQL EXPLAIN output for a query on a 10-million-row table:

Seq Scan on users (cost=0.00..183340.00 rows=1 width=36)
  Filter: (email = 'a@b.com'::text)

This tells you: the planner chose a sequential scan (reading every row), estimated cost is 183,340 (arbitrary units), expects to return 1 row, and applies the email filter to each row. The cost number is what the planner uses to compare strategies — lower is better.

After adding an index on email:

Index Scan using users_email_idx on users (cost=0.43..8.45 rows=1 width=36)
  Index Cond: (email = 'a@b.com'::text)

Now the planner uses the index, and the estimated cost dropped from 183,340 to 8.45 — a 20,000× improvement. The Index Cond shows what the index is filtering on.

Key terms in EXPLAIN output:

Term	Meaning	Action
`Seq Scan`	Full table scan	Add an index, or the table is small
`Index Scan`	Index used, then table fetch	Good for selective queries
`Index Only Scan`	Covering index, no table access	Best case for that query
`Bitmap Index Scan`	Multiple indexes combined	Common for AND/OR conditions
`Nested Loop`	Join: for each row in A, scan B	OK for small A, bad for large A
`Hash Join`	Build hash table on smaller table	Standard for equi-joins
`Sort`	Explicit sort step	Add an index on ORDER BY columns

The rows= estimate is critical. If the planner estimates 1 row but the query actually returns 100,000, it may have chosen a nested loop when a hash join would have been faster. Run ANALYZE after bulk loads to update statistics — stale statistics are the most common cause of bad plans.

When Indexes Don't Help

Indexes are not a performance panacea. They fail to help — or actively hurt — in several common scenarios:

Low cardinality columns. An index on a boolean column (2 distinct values) is almost never used. The planner would have to read half the table via index lookups, which is more expensive than a sequential scan. The same applies to any column where selectivity exceeds roughly 10-20%.

Functions on indexed columns. WHERE LOWER(email) = 'a@b.com' cannot use a regular index on email because the index stores the original values, not the lowercased versions. The fix is a functional index: CREATE INDEX ON users (LOWER(email)). Similarly, WHERE date(created_at) = '2024-01-01' cannot use an index on created_at; rewrite as WHERE created_at >= '2024-01-01' AND created_at < '2024-01-02' instead.

Implicit type casting. WHERE int_column = '42' (string literal) may prevent index use because the database must cast every row. Match types exactly: WHERE int_column = 42.

Leading wildcards. WHERE email LIKE '%@gmail.com' cannot use a B-tree index because the index is sorted from the left. WHERE email LIKE 'john%' can use the index; WHERE email LIKE '%john%' cannot. For substring search, consider full-text search indexes (GIN in PostgreSQL) instead.

Too many indexes. Each index slows writes. A table with 10 indexes takes roughly 10× longer to insert into than a table with none. If your workload is write-heavy (logging, event tracking), be stingy with indexes. If it's read-heavy (analytics, dashboards), be generous.

A Worked Example: The Slow Dashboard

A team reports that their admin dashboard takes 8 seconds to load. The query:

SELECT u.name, u.email, COUNT(o.id) as order_count
FROM users u
JOIN orders o ON o.user_id = u.id
WHERE u.status = 'active'
  AND u.created_at > '2024-01-01'
  AND o.total > 100
GROUP BY u.id
ORDER BY order_count DESC
LIMIT 20;

The users table has 5 million rows; orders has 50 million. EXPLAIN shows:

Sort (cost=452310.00..452310.50 rows=20 width=52)
  Sort Key: (count(o.id)) DESC
  -> HashAggregate (cost=452300.00..452305.00 rows=20 width=52)
        Group Key: u.id
        -> Hash Join (cost=125000.00..327300.00 rows=2500000 width=44)
              Hash Cond: (o.user_id = u.id)
              -> Seq Scan on orders (cost=0.00..150000.00 rows=10000000 width=12)
                    Filter: (total > 100)
              -> Hash (cost=100000.00..100000.00 rows=2500000 width=36)
                    -> Seq Scan on users (cost=0.00..100000.00 rows=2500000 width=36)
                          Filter: ((status = 'active') AND (created_at > '2024-01-01'))

Both tables are being fully scanned. The plan reads 2.5 million users and 10 million orders, joins them, groups, sorts, then returns 20 rows. The fix:

CREATE INDEX ON users (status, created_at) INCLUDE (name, email); — covering index for the user filter
CREATE INDEX ON orders (user_id) WHERE total > 100; — partial index for the order filter

After indexing, EXPLAIN shows:

Limit (cost=0.85..1250.50 rows=20 width=52)
  -> Sort (cost=0.85..1250.50 rows=5000 width=52)
        Sort Key: (count(o.id)) DESC
        -> GroupAggregate (cost=0.85..1000.00 rows=5000 width=52)
              Group Key: u.id
              -> Nested Loop (cost=0.85..800.00 rows=5000 width=44)
                    -> Index Only Scan using users_status_created_at_idx on users
                          Index Cond: ((status = 'active') AND (created_at > '2024-01-01'))
                    -> Index Scan on orders_user_id_idx on orders
                          Index Cond: (user_id = u.id)

Cost dropped from 452,310 to 1,250 — a 360× improvement. The dashboard loads in 20ms instead of 8 seconds. The key insight: the planner now reads only the ~5,000 active recent users (via the covering index) and joins only their orders (via the partial index), instead of scanning both tables entirely.

Practical Tips for Database Performance

Index columns in WHERE, JOIN, and ORDER BY clauses. These are the columns the planner needs to filter, join, or sort on.
Put the most selective column first in composite indexes. The column that eliminates the most rows should be leftmost.
Use covering indexes for hot queries. If a query runs thousands of times per minute, the extra storage is worth the speed.
Run ANALYZE after bulk loads. Stale statistics cause bad plans. PostgreSQL auto-analyzes, but large bulk loads may need a manual ANALYZE.
Use partial indexes for filtered queries. CREATE INDEX ... WHERE total > 100 is smaller and faster than indexing the whole table.
Don't index everything. Each index slows writes. A table with 10 indexes takes roughly 10× longer to insert into than a table with none.
Read EXPLAIN before deploying. If you see Seq Scan on a large table, investigate. The planner is telling you it has no better option.
Check selectivity before creating an index. If the column has fewer than ~100 distinct values on a large table, the index will likely be ignored. The SQL Index Selectivity Calculator tells you whether the planner will use it.

Limitations and When to Go Beyond Indexing

Indexing is necessary but not sufficient for database performance at scale. When indexes alone aren't enough:

Query rewriting. A query that uses a subquery may be slower than one that uses a join, even with identical indexes. EXPLAIN reveals the difference.
Denormalization. For read-heavy analytics, duplicating data (storing order_count on the user row instead of computing it) trades write complexity for read speed.
Partitioning. Splitting a 500-million-row table into partitions by date lets the planner skip entire partitions — a 100× reduction in data scanned for time-bounded queries.
Read replicas. For workloads with many more reads than writes, replicas distribute the read load across multiple servers.
Caching. Redis or application-level caching avoids the database entirely for frequently accessed, rarely changed data.

Indexing is the first line of defense, not the last. But it is the one that catches the most common performance problems — and the one that most developers underuse because they've never learned to read the planner's output.

Frequently Asked Questions

Q: What is a database index?

A: A separate data structure (usually a B-tree) that the database maintains alongside your table. It allows the database to find rows without scanning the entire table, similar to an index at the back of a textbook.

Q: Why isn't my index being used?

A: Most often because selectivity is too high — the filter keeps too many rows (roughly more than 10-20% of the table), so the planner decides a sequential scan is cheaper. Other causes: functions on the indexed column (WHERE LOWER(col) = ...), implicit type casting, or stale statistics.

Q: What is selectivity?

A: The fraction of rows a filter keeps: rows returned divided by total rows. Low selectivity (few rows) means the index is useful. High selectivity (many rows) means the planner will likely ignore the index.

Q: What is a composite index?

A: An index on two or more columns. Order matters: a composite index on (A, B) can answer queries filtering on A alone or A and B, but not B alone. Put the most selective column first.

Q: What is a covering index?

A: An index that contains all the columns a query needs, so the database can answer the query from the index alone without fetching table rows. This is the fastest access path for a query.

Q: How do I read an EXPLAIN plan?

A: Look for Seq Scan (full table scan — bad on large tables), Index Scan (index used — good), and Index Only Scan (covering index — best). The cost number lets you compare strategies; the rows estimate tells you how accurate the planner's guess is.

Q: Does every column need an index?

A: No. Each index slows down INSERT, UPDATE, and DELETE operations. Index columns used in WHERE, JOIN, and ORDER BY clauses, and be skeptical of columns with fewer than ~100 distinct values on large tables.

Q: What is a partial index?

A: An index with a WHERE clause: CREATE INDEX ... WHERE total > 100. It indexes only the rows matching the condition, making it smaller and faster than a full-table index. Useful for queries that always filter on the same condition.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

Security Fundamentals for Web Developers: Authentication, Authorization, and the Attacks You Need to Prevent

Adolfo Pedernera — Thu, 27 Aug 2026 14:37:19 +0000

Authentication vs authorization, JWT deep dive, OAuth 2.0, password hashing, XSS/CSRF/SQL injection prevention, and security headers.

The Developer Who Stored Passwords in Plaintext

In 2012, LinkedIn suffered a breach that exposed 6.5 million password hashes. The hashes were unsalted SHA-1 — a hashing algorithm so fast that an attacker with a modern GPU could crack billions of hashes per hour. Within days, 60% of the passwords were recovered in plaintext. The attackers didn't need sophisticated exploits; they just needed a database dump and a rainbow table.

The breach was preventable. Salting (adding random data to each password before hashing) and using a slow hashing algorithm (bcrypt, scrypt, or Argon2) would have made cracking infeasible. These weren't obscure techniques — they were well-documented best practices that LinkedIn's engineering team either didn't know or didn't prioritize. The cost: a $1.25 million settlement, a decade of reputational damage, and millions of users whose credentials were compromised.

This is the uncomfortable truth about web security: most breaches aren't caused by sophisticated attacks. They're caused by developers who didn't know the basics. SQL injection, cross-site scripting (XSS), and broken authentication have been on the OWASP Top 10 for over 20 years — and they're still the most common vulnerabilities in production applications.

This guide covers the security fundamentals that every web developer should know: the difference between authentication and authorization, how JWTs actually work (and how they're misused), how to hash passwords correctly, how to prevent the OWASP Top 10 vulnerabilities, and which security headers to include in every response. The goal isn't to make you a security engineer — it's to make you a developer who doesn't ship the vulnerabilities that attackers are counting on.

Authentication vs Authorization: Knowing Who vs Knowing What They Can Do

These two terms are confused so often that many developers use them interchangeably. They are fundamentally different questions:

Authentication answers: "Who are you?" — verifying identity (username/password, biometrics, API key)
Authorization answers: "Are you allowed to do this?" — checking permissions (is this user an admin? does this token have the read:users scope?)

A user who logs in with correct credentials is authenticated. Whether they can delete other users' accounts is an authorization decision. Confusing the two leads to vulnerabilities: a user who is authenticated but not authorized to access an admin endpoint can still access it if you only check "is logged in?".

The principle of least privilege: every user, service, and token should have exactly the permissions it needs — no more. A background job that only reads from a database should not have write access. A frontend API token should not have admin scopes. A user who can view their own profile should not be able to view others'.

JWT Deep Dive: What's Inside the Token

JSON Web Tokens (JWTs) are the most common mechanism for stateless authentication in modern web applications. A JWT is a self-contained token that encodes claims (user ID, permissions, expiration) in a JSON payload, signed by the server. The client sends the token in the Authorization: Bearer <token> header, and the server verifies the signature to trust the claims.

A JWT has three parts, each Base64URL-encoded and separated by dots:

eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjo0MiwiZXhwIjoxNjg3MzA1NjAwfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

Header (eyJhbGciOiJIUzI1NiJ9): algorithm (HS256) and token type
Payload (eyJ1c2VyX2lkIjo0MiwiZXhwIjoxNjg3MzA1NjAwfQ): claims — user_id, expiration, roles
Signature (SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c): HMAC of header+payload with server secret

The critical claim is exp (expiration). A JWT without an expiration is valid forever — if leaked, it's a permanent access token. Set short expirations (15 minutes to 1 hour) and use refresh tokens for longer sessions. The JWT Decoder parses tokens client-side, showing the header, payload, and signature — useful for debugging authentication flows and verifying that your tokens contain the expected claims.

Common JWT mistakes:

Storing sensitive data in the payload. JWTs are signed, not encrypted. Anyone can read them.
Not validating the signature on the server. If you trust the payload without verifying the signature, anyone can forge tokens.
Using alg: "none". Some JWT libraries accept tokens with no algorithm, bypassing signature verification entirely. Always pin the allowed algorithms server-side.
Long expirations without refresh. A token that's valid for 30 days is a 30-day window if leaked.

Password Hashing: Why Plaintext Is a Crime

Storing passwords in plaintext is the most basic security mistake — and it still happens. When a database is breached (and databases are breached), plaintext passwords are immediately usable. The fix is hashing : a one-way function that converts a password into a fixed-length string that cannot be reversed.

Not all hashing is equal. Fast hash functions (MD5, SHA-1, SHA-256) are designed for speed — which is exactly what you don't want for passwords. An attacker with a modern GPU can compute billions of SHA-256 hashes per second. A slow hash function (bcrypt, scrypt, Argon2) is designed to be computationally expensive, making brute-force attacks infeasible.

The algorithm comparison:

Algorithm	Hashes/second (GPU)	Recommended	Why
MD5	~180 billion	❌ Never	Broken, too fast
SHA-1	~6 billion	❌ Never	Broken, too fast
SHA-256	~2 billion	❌ Never	Too fast for passwords
bcrypt	~100,000	✅ Yes	Adaptive cost factor
scrypt	~10,000	✅ Yes	Memory-hard
Argon2	~1,000	✅ Yes (best)	Memory-hard, winner of Password Hashing Competition

Salting adds random data to each password before hashing, so two users with the same password have different hashes. This defeats rainbow tables (precomputed hash databases). Modern algorithms (bcrypt, Argon2) handle salting automatically.

The Password Generator creates strong random passwords with configurable length and character sets — useful for generating initial passwords, API keys, or test data. But remember: the generator creates the password; bcrypt/Argon2 protects it in storage.

The OWASP Top 10: Attacks Every Developer Should Prevent

The OWASP Top 10 lists the most critical web application security risks. The ones every developer should know how to prevent:

Broken Object-Level Authorization (BOLA)

The most common API vulnerability. If GET /users/42 returns user 42's data, but the server doesn't check that the authenticated user is user 42 (or an admin), then any authenticated user can access any other user's data by changing the ID in the URL. Every endpoint that takes an ID must check authorization.

Fix: Always verify that the authenticated user is authorized to access the specific resource they're requesting. Don't rely on clients to only request their own data.

Cross-Site Scripting (XSS)

An attacker injects malicious JavaScript into a page that other users view. The script runs in the victim's browser, stealing cookies, session tokens, or performing actions on their behalf. Three types:

Reflected XSS: the malicious script is part of the URL (e.g., ?search=<script>...</script>)
Stored XSS: the script is stored in the database (e.g., in a comment or post)
DOM-based XSS: the vulnerability is in client-side JavaScript that unsafely manipulates the DOM

Fix: Escape all user input before rendering it in HTML. Use a framework that auto-escapes by default (React, Vue, Angular). Set the Content-Security-Policy header to restrict which scripts can execute.

Cross-Site Request Forgery (CSRF)

An attacker tricks a user's browser into making a request to a site where the user is authenticated. If a bank transfer is triggered by POST /transfer?to=attacker&amount=1000, and the user visits a malicious page while logged in, the browser sends the request with the user's cookies — and the transfer happens.

Fix: Use CSRF tokens — a unique, unpredictable value embedded in every form and verified on the server. Modern frameworks (Rails, Django, Spring) include CSRF protection by default. For APIs, require a custom header (e.g., X-Requested-With) that cross-origin requests cannot set.

SQL Injection

An attacker injects SQL code through user input. If a query is built by string concatenation — SELECT * FROM users WHERE name = ' + userInput + ' — an attacker can input ' OR '1'='1 to bypass authentication, or '; DROP TABLE users; -- to destroy data.

Fix: Use parameterized queries (prepared statements) exclusively. Never concatenate user input into SQL. ORMs (Sequelize, SQLAlchemy, Entity Framework) use parameterized queries by default — but raw SQL queries are still vulnerable if not parameterized.

Security Misconfiguration

Default credentials, exposed error messages, unnecessary features enabled, missing security headers. The 2017 Equifax breach was caused by an unpatched Apache Struts vulnerability — a known exploit for which a patch had been available for months.

Fix: Harden your deployment: disable default accounts, remove unused features, keep dependencies updated, and include security headers in every response.

Security Headers: The Headers Every Response Should Include

HTTP security headers tell the browser how to handle your content. They're free protection — one line of configuration each:

Header	What it does	Recommended value
`Strict-Transport-Security`	Force HTTPS for all future requests	`max-age=31536000; includeSubDomains`
`X-Content-Type-Options`	Prevent MIME type sniffing	`nosniff`
`X-Frame-Options`	Prevent clickjacking (your page in an iframe)	`DENY` or `SAMEORIGIN`
`Content-Security-Policy`	Restrict which resources can load	`default-src 'self'; script-src 'self'`
`X-XSS-Protection`	Enable browser XSS filter (legacy)	`1; mode=block`
`Referrer-Policy`	Control how much referrer information is sent	`strict-origin-when-cross-origin`
`Cache-Control`	Prevent caching of sensitive data	`no-store` for authenticated responses

The most important: Strict-Transport-Security (HSTS) ensures that browsers always use HTTPS for your domain, preventing SSL stripping attacks. Once set, a user's browser will refuse to connect via HTTP for the duration of max-age.

Session Management: Stateful vs Stateless

Stateful sessions store session data on the server (in memory, Redis, or a database) and send the client a session ID cookie. The server looks up the session on every request. Pros: easy to invalidate, can store large data. Cons: requires shared session storage in a multi-server setup, memory overhead.

Stateless sessions (JWTs) encode all session data in the token itself. The server doesn't store anything. Pros: no shared storage needed, works across services. Cons: cannot invalidate individual tokens before expiration, token size grows with claims.

The hybrid approach (most common in production): use a short-lived JWT (15-60 minutes) for stateless authentication, plus a long-lived refresh token (stored server-side) to issue new JWTs. This gives you the scalability of stateless tokens with the ability to revoke access via the refresh token.

Practical Tips for Secure Development

Never roll your own crypto. Use established libraries (bcrypt, Argon2, libsodium). Cryptography is easy to get wrong in ways that are invisible until you're breached.
Use HTTPS everywhere. No exceptions. Even internal APIs should use TLS. Use Let's Encrypt for free certificates.
Validate input on the server. Client-side validation is for UX; server-side validation is for security. Never trust client input.
Keep dependencies updated. Use npm audit, Snyk, or Dependabot to flag known vulnerabilities in your dependencies.
Log security events. Failed logins, permission denied errors, and unusual patterns should be logged and monitored.
Use environment variables for secrets. Never commit API keys, database passwords, or JWT secrets to version control.
Implement rate limiting on authentication endpoints. Brute-force attacks try thousands of passwords per second. Rate limit to 5-10 attempts per minute per IP.

Limitations: What This Guide Doesn't Cover

This guide covers the fundamentals — the vulnerabilities that every developer should prevent. It does not cover:

Penetration testing — actively trying to break your own application
Threat modeling — systematically identifying attack vectors
Compliance — GDPR, HIPAA, PCI-DSS requirements
Infrastructure security — network segmentation, firewalls, intrusion detection
Cryptography engineering — designing custom protocols (don't)

For production applications handling sensitive data, hire a security professional. The fundamentals prevent the most common attacks, but determined attackers with resources require defense in depth.

Frequently Asked Questions

Q: What is the difference between authentication and authorization?

A: Authentication answers 'who are you?' (verifying identity). Authorization answers 'are you allowed to do this?' (checking permissions). A user can be authenticated (logged in) but not authorized (not an admin).

Q: What is a JWT?

A: A JSON Web Token is a self-contained token that encodes claims (user ID, permissions, expiration) in a JSON payload, signed by the server. JWTs are signed, not encrypted — never put secrets in them.

Q: How should I store passwords?

A: Hash them with a slow algorithm: bcrypt, scrypt, or Argon2. Never use fast hashes (MD5, SHA-1, SHA-256) for passwords. Never store plaintext.

Q: What is the most common web vulnerability?

A: Broken Object-Level Authorization (BOLA) — every endpoint that takes an ID must check that the authenticated user is authorized to access that specific resource.

Q: What is XSS?

A: Cross-Site Scripting — an attacker injects malicious JavaScript into a page that other users view. Fix by escaping all user input and setting Content-Security-Policy headers.

Q: What is CSRF?

A: Cross-Site Request Forgery — an attacker tricks a user's browser into making a request to a site where they're authenticated. Fix with CSRF tokens and SameSite cookies.

Q: What is SQL injection?

A: An attacker injects SQL code through user input. Fix by using parameterized queries (prepared statements) exclusively — never concatenate user input into SQL.

Q: What security headers should I include?

A: At minimum: Strict-Transport-Security (force HTTPS), X-Content-Type-Options: nosniff, X-Frame-Options: DENY, and Content-Security-Policy to restrict resource loading.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

A/B Testing & Experimentation: How to Run Tests That Actually Teach You Something

Adolfo Pedernera — Thu, 27 Aug 2026 04:48:07 +0000

Hypothesis testing, p-values, confidence intervals, statistical power, and the traps that fool even experienced teams.

The Tasting Illusion

In the 1970s, Pepsi ran a marketing campaign that seemed to prove something remarkable: in blind taste tests, more people preferred Pepsi over Coke. The campaign was a sensation — millions of consumers sipped from unmarked cups and pointed to the sweeter drink. Pepsi's stock rose. Coke panicked, reformulated, and launched "New Coke" in 1985, one of the most infamous product launches in history.

The problem wasn't the taste. It was the test. A single sip from a small cup measures immediate sweetness preference, not which beverage someone drinks for decades. The blind test had statistical significance — the difference was real and repeatable — but it lacked practical significance. It measured the wrong thing.

This is the central trap of A/B testing and experimentation: statistical significance does not mean business significance. A test can prove, with 99% confidence, that a button color change increases clicks by 0.3% — a finding that is simultaneously true and useless. This guide explains how to design and interpret experiments that avoid that trap: how to set up hypotheses, calculate whether your sample is large enough, interpret p-values without fooling yourself, and distinguish signals that matter from noise that merely looks convincing.

The Logic of Experimentation

Every controlled experiment follows the same skeleton, whether you're testing a new cancer drug, a website headline, or a pricing page. You start with a hypothesis — a specific, testable prediction. You split your subjects randomly into two groups: the control (which sees the current version) and the treatment (which sees the change). You measure an outcome in both groups. Then you ask a single question: is the difference I see larger than what random chance would typically produce?

That last step is where statistics enters. The logic is counterintuitive on purpose: you begin by assuming the boring explanation (your change did nothing) and then check whether the data is incompatible with that assumption. This starting assumption is called the null hypothesis , written $H_{0}$ . The alternative — your actual prediction — is the alternative hypothesis , written $H_{1}$ .

The engine that drives the answer is the p-value : the probability of observing a difference at least as large as the one you measured, assuming the null hypothesis is true.

p = P (observed difference ∣ H_{0} is true)

A p-value of 0.03 means: if your change actually did nothing, you would still see a difference this large 3% of the time by pure chance. It does NOT mean there is a 97% chance your change works. This misinterpretation is so common that the American Statistical Association published a formal statement warning against it.

The Two Mistakes You Can Make

Whenever you draw conclusions from an experiment, you can go wrong in two distinct ways:

Error	What happens	Probability	Cost
Type I (false positive)	You conclude the change works when it doesn't	p-value (α)	Ship a dud, waste engineering time
Type II (false negative)	You conclude the change does nothing when it actually works	β	Miss a real improvement

The p-value threshold you choose — the significance level , denoted α — is the Type I error rate you're willing to tolerate. The conventional 0.05 means "I accept a 5% chance of shipping something that does nothing." In high-stakes domains (medicine, aviation), researchers demand α = 0.001 or lower. In fast-moving consumer apps, teams sometimes accept 0.10 to move faster.

The Type II error rate, β, is the flip side. If β = 0.20, you have a 20% chance of missing a real effect. The complement, 1 − β, is called statistical power : the probability that your test will detect an effect if one truly exists. A test with 80% power and 5% significance is the standard minimum in most fields — and many published studies fall short of even that.

Power is the probability you will detect a real effect. Most teams ignore it — and run tests that were doomed to be inconclusive before the first user arrived.

Sample Size: The Number That Decides Everything

Here is the uncomfortable truth about A/B testing: a test with too few users cannot detect anything useful, no matter how good the change is. If you run a test on 50 users and see no significant result, you have learned nothing — the test may simply have been too small to detect the effect. This is an underpowered test, and underpowered tests produce more noise than signal.

The required sample size depends on four numbers:

Baseline conversion rate (e.g., 5% of visitors sign up)
Minimum detectable effect (e.g., you want to catch a 10% relative lift → from 5% to 5.5%)
Significance level α (typically 0.05)
Power 1 − β (typically 0.80)

The formula for a two-proportion z-test (the standard for conversion rate comparisons) is approximately:

n \approx \frac{( z _{α /2} + z _{β} ) ^{2} \cdot 2 p ( 1 - p )}{( p _{1} - p _{2} ) ^{2}}

where p is the pooled proportion, $p_{1} - p_{2}$ is the absolute effect size, and $z_{α /2}$ , $z_{β}$ are the critical z-values (1.96 and 0.84 for α=0.05, power=0.80).

For a baseline of 5% and a minimum detectable effect of 10% relative (0.5 percentage points absolute), this gives n ≈ 15,700 users per variation. For a 5% relative lift — a more realistic goal — you need roughly 10× more: ~157,000 users per variation. This is why small teams with modest traffic often conclude that A/B testing "doesn't work" — they were running experiments that needed ten times their monthly traffic to detect anything. The Sample Size Calculator computes this directly for your baseline and desired power, so you can plan your test duration before launching.

Users per variation needed for 80% power, α=0.05, two-sided test. Smaller effects and lower baselines explode the sample size.

Practical rule of thumb: If your site gets 1,000 visitors per day and you need 15,700 per variation, the test runs for ~31 days. If your test takes longer than 4 weeks, reconsider whether the minimum detectable effect is realistic — or whether A/B testing is the right tool for your traffic level.

Confidence Intervals: The Number That Tells the Whole Story

A p-value tells you whether the effect is "significant." A confidence interval tells you the range of plausible effect sizes — which is usually what you actually want to know.

A 95% confidence interval of [+0.2%, +1.8%] for a conversion rate lift means: we are 95% confident the true effect lies between 0.2 and 1.8 percentage points. Crucially, this interval does NOT include zero — which is why the p-value is below 0.05. But it also tells you something the p-value hides: the effect could be as small as 0.2%, which may not justify the engineering cost.

The formula for a confidence interval on the difference between two proportions:

(p_{1} - p_{2}) \pm z_{α /2} \frac{p _{1} ( 1 - p _{1} )}{n _{1}} + \frac{p _{2} ( 1 - p _{2} )}{n _{2}}

When the confidence interval includes zero, the result is not statistically significant. When it does not include zero, you have a significant result — and the interval's width tells you how precisely you've estimated the effect. A wide interval ([−0.1%, +2.1%]) means "something is happening, but we're not sure exactly what." A narrow interval [+0.8%, +1.2%] means "the effect is almost certainly in this tight range". If you have the observed counts from a finished test, the Confidence Interval Calculator returns the interval directly — useful for reporting results to stakeholders who want to know the range, not just the p-value.

Use the confidence interval as your primary result, not the p-value. It answers the business question ("how much lift can I expect?") rather than the statistical one ("can I reject the null?").

Chi-Square and t-Tests: Choosing the Right Tool

Not all experiments compare conversion rates. Sometimes you're comparing averages (revenue per user, time on page, order value), and sometimes you're comparing distributions across categories (did the change shift users between plan tiers?). The right test depends on the data type.

For conversion rates and proportions (click/no click, signup/no signup): use a chi-square test or a two-proportion z-test. The chi-square test checks whether the observed frequencies in your 2×2 table (control vs treatment × converted vs not) differ from what the null hypothesis predicts.

χ^{2} = \sum \frac{( O _{i} - E _{i} ) ^{2}}{E _{i}}

where $O_{i}$ is the observed count and $E_{i}$ is the expected count under the null. The result is compared to a chi-square distribution with 1 degree of freedom to obtain a p-value. The Chi-Square Calculator does this computation from your observed counts, returning both the statistic and the p-value — useful when you have the raw numbers from a finished test and want to verify significance without building the table by hand.

For continuous outcomes (revenue, time, score): use a t-test. The t-test compares the means of two groups while accounting for the spread (standard deviation) within each groups:

t = \frac{x ˉ _{1} - x ˉ _{2}}{\frac{s _{12}}{n _{1}} + \frac{s _{22}}{n _{2}}}

where $x ˉ$ is the sample mean, $s$ is the standard deviation, and $n$ is the sample size. The denominator is the standard error of the difference — a quantity that combines the uncertainty from both groups. The t-Test Calculator computes this from your two samples, handling both the equal-variance and unequal-variance (Welch's) cases.

The standard deviation matters enormously here. If revenue per user swings wildly (high variance), you need far more users to detect the same mean difference than if revenue is consistent. This is why the confidence interval on the mean difference is so useful: it shows whether the signal (the mean difference) is large relative to the noise (the standard deviation).

Practical decision rule:

Binary outcome (clicked/didn't, signed up/didn't) → chi-square or z-test for proportions
Continuous outcome (revenue, time, score) → t-test
Comparing more than two variants → ANOVA (not covered here)

The Traps That Fool Experienced Teams

Peeking at Results

The p-value formula assumes you look at the data exactly once, at a predetermined sample size. If you peek at your test on day 3, day 7, day 10 — and stop as soon as p < 0.05 — you are dramatically inflating your false positive rate. With 5 peeks, the true false positive rate is not 5% but roughly 14%.

Fix: Decide your sample size in advance, run the test to that sample size, then analyze once. If you must peek, use sequential testing methods (always-valid p-values, alpha spending) that adjust the threshold for each peek — but these require roughly 30% more users for the same power.

P-Hacking (Multiple Comparisons)

If you test 20 different metrics (conversion rate, revenue, time on page, scroll depth, bounce rate...) and report the one that came back significant at p < 0.05, you are p-hacking. With 20 independent tests at α = 0.05, the probability of at least one false positive is 1 − (1−0.05)^20 ≈ 64%.

Fix: Pre-specify a single primary metric before the test begins. Treat everything else as exploratory. If you must test multiple metrics, apply a Bonferroni correction: divide α by the number of tests (so 0.05 / 20 = 0.0025 for 20 metrics).

Simpson's Paradox

A treatment can win in every subgroup (mobile users, desktop users, new visitors, returning visitors) and lose in aggregate — or vice versa. This happens when the subgroups have very different sizes and very different baseline rates.

Fix: Always segment your results. If the treatment effect reverses between mobile and desktop, the aggregate number is misleading. Report the per-segment results alongside the overall.

Novelty and Primacy Effects

A new feature may initially perform worse (users are confused) or better (users are curious) than it will long-term. A 2-week test captures the novelty effect, not the steady-state effect.

Fix: Run tests for at least 2 full weeks (to capture day-of-week effects) and ideally 4 weeks for features with learning curves. Be skeptical of results that appear in week 1 and vanish in week 2.

A Worked Example: Button Color

A team tests whether a green signup button outperforms the current blue one. Baseline conversion rate: 4.0%. They want to detect a 10% relative lift (to 4.4%) with 80% power and α = 0.05.

Step 1 — sample size. Using the formula above with p₁ = 0.04, p₂ = 0.044, α = 0.05, power = 0.80: n ≈ 39,000 users per variation. With 20,000 daily visitors split 50/50, the test runs for ~4 days.

Step 2 — run the test. Control (blue): 780 conversions out of 39,000 (2.00%). Treatment (green): 936 conversions out of 39,000 (2.40%).

Step 3 — chi-square test. The observed counts form a 2×2 table. The chi-square statistic is 15.2, which gives p ≈ 0.0001 — well below 0.05. The result is statistically significant.

Step 4 — confidence interval. The difference is 0.40 percentage points, with a 95% CI of [+0.22%, +0.58%]. This interval excludes zero (confirming significance) and tells the business: "the lift is somewhere between 0.22 and 0.58 percentage points, or 5.5% to 14.5% relative."

Step 5 — decision. Is a 0.4 percentage point lift worth the engineering cost and the design inconsistency? That is a business question, not a statistical one. The statistics only say: the lift is real and almost certainly between 0.22% and 0.58%.

When A/B Testing Is the Wrong Tool

A/B testing is powerful, but it is not always the right method. Consider alternatives when:

Your traffic is too low. If you need 100,000 users per variation and get 5,000 per month, a properly powered test would take years. Use qualitative research, user interviews, or heuristic evaluation instead.
You're testing a fundamental redesign. A/B testing measures incremental changes. A complete redesign has too many variables changing at once to attribute causality. Use before/after analysis with caution.
Network effects exist. If users in the treatment group interact with users in the control group (social networks, marketplaces), the groups are no longer independent. The test is invalid.
You need to understand WHY. A/B testing tells you whether something works, not why. Pair experiments with qualitative research (session recordings, user interviews) to understand the mechanism.

Practical Tips for Running Experiments

Pre-register your test. Write down the hypothesis, primary metric, sample size, and analysis plan before collecting data. This prevents post-hoc rationalization.
Run an A/A test occasionally. Show both groups the same experience. If you get a "significant" result, your testing infrastructure is broken.
Segment after, not during. Pre-specify one primary metric. Segment (by device, by region, by new/returning) only as exploratory analysis.
Track guardrail metrics. A treatment might increase clicks but hurt revenue. Define guardrail metrics (revenue per user, unsubscribe rate, page load time) that must not degrade.
Account for multiple variants. If you run an A/B/C/D test, apply a Bonferroni correction (α / number of comparisons) or use a false discovery rate method.
Report confidence intervals, not just p-values."Lift of 0.4 pp, 95% CI [+0.22, +0.58]" is more actionable than "p = 0.0001."

Limitations and Edge Cases

A/B testing has honest limitations. It is not good at detecting small effects unless you have enormous traffic. It assumes random assignment — if your randomization is broken (e.g., more mobile users end up in treatment by chance), the results are biased. It measures short-term effects — a headline that clicks well today may fatigue users after a month. And it cannot detect interactions unless you specifically design for them (a treatment may help new users but hurt returning users, which cancels out in aggregate).

Finally, A/B testing is a tool for optimization, not for strategy. It can tell you which of two headlines works better, but it cannot tell you what your content strategy should be. Use it to refine, not to think.

Frequently Asked Questions

Q: What is a p-value?

A: The probability of observing a result at least as extreme as yours, assuming the null hypothesis (that your change did nothing) is true. A p-value of 0.05 means: if your change did nothing, you would still see a result this large 5% of the time by chance. It is NOT the probability that your change works.

Q: What is the difference between statistical significance and practical significance?

A: Statistical significance means the observed difference is unlikely due to chance (low p-value). Practical significance means the difference is large enough to matter for your business. A result can be statistically significant (real) but practically insignificant (too small to care about).

Q: How many users do I need for an A/B test?

A: It depends on your baseline rate and the minimum effect you want to detect. For a 5% baseline and a 10% relative lift (to 5.5%), you need roughly 15,700 users per variation for 80% power at α=0.05. Smaller effects require dramatically more users.

Q: What is statistical power?

A: The probability that your test will detect a real effect if one exists. Typically set to 80% (β=0.20). An underpowered test — one with too few users — will often conclude 'no significant difference' even when a real effect exists.

Q: Why shouldn't I peek at my test results every day?

A: Because each peek inflates your false positive rate. With 5 peeks at α=0.05, your true false positive rate is roughly 14%. Decide your sample size in advance and analyze once, or use sequential testing methods that adjust for peeking.

Q: What is the difference between a t-test and a chi-square test?

A: Use a t-test when comparing means of continuous outcomes (revenue, time, score). Use a chi-square test when comparing proportions or frequencies (conversion rates, click/no-click). The t-test assumes continuous data; the chi-square test works with counts in categories.

Q: What is a confidence interval?

A: A range of plausible values for the true effect. A 95% confidence interval of [+0.2%, +1.8%] means we are 95% confident the true lift is between 0.2 and 1.8 percentage points. If the interval excludes zero, the result is statistically significant.

Q: What is Simpson's Paradox?

A: A phenomenon where a trend appears in every subgroup but reverses in aggregate (or vice versa). It happens when subgroups have very different sizes and baselines. Always segment your results — the aggregate can hide or reverse real effects.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

Statistics for Data-Driven Developers: Making Decisions With Numbers

Adolfo Pedernera — Wed, 26 Aug 2026 16:37:47 +0000

Descriptive stats, probability distributions, confidence intervals, hypothesis testing, and the mistakes that fool experienced engineers.

The Alert That Cried Wolf

A team at a mid-size SaaS company set up monitoring for their API response times. They configured an alert: "Trigger if average response time exceeds 500 ms over a 5-minute window." The alert fired constantly. Engineers sprinted to investigate, found nothing wrong, and started ignoring it. Two weeks later, a genuine outage went undetected for 20 minutes because everyone had learned to dismiss the alert.

The problem was the word "average." The average response time was 350 ms, but the distribution was heavily skewed: 95% of requests completed in under 200 ms, while 5% took over 2 seconds (timeouts and retries). The average hid the tail — the 5% of users having a terrible experience. When the team switched to alerting on the 95th percentile (p95) of response time instead of the mean, the false alarms stopped. The p95 was 450 ms — still healthy — and the tail latency that had been dragging up the average became visible in a separate dashboard.

This is the fundamental lesson of statistics for developers: the summary statistic you choose determines the story you see. Mean, median, and percentile each reveal different truths. Standard deviation tells you whether your system is consistent or erratic. Confidence intervals tell you whether a change is real or noise. Hypothesis testing tells you whether your "improvement" actually improved anything or whether you're just seeing random variation.

This guide explains the statistics that matter for developers who work with data — whether that data is response times, conversion rates, error rates, or business metrics. It covers descriptive statistics (how to summarize data), probability (how to model uncertainty), and inferential statistics (how to draw conclusions from samples) — with the goal of making better decisions and avoiding the traps that fool even experienced engineers.

Descriptive Statistics: Summarizing Without Lying

Descriptive statistics compress a dataset into a few numbers. The art is choosing numbers that tell the truth — and not choosing numbers that hide it.

Central Tendency: Mean, Median, Mode

The mean (average) is the most common summary — and the most misleading when data is skewed. If 9 users pay $10/month and 1 user pays $1,000/month, the mean revenue per user is $109, which describes no one's actual experience.

The median (middle value) is robust to outliers. In the same dataset, the median is $10 — a more honest summary of the typical user. Use median when data is skewed (revenue, response times, house prices) and mean when data is symmetric (heights, test scores, measurement errors).

The mode (most frequent value) matters for categorical data: the most common error code, the most popular feature, the most frequent user country.

Spread: Variance and Standard Deviation

Central tendency tells you where the middle is. Spread tells you how far the data ranges around that middle. Two APIs can have the same mean response time (350 ms) but wildly different consistency: one with σ = 50 ms (predictable) and one with σ = 400 ms (erratic).

Variance is the average squared deviation from the mean:

s^{2} = \frac{\sum ( x _{i} - x ˉ ) ^{2}}{n - 1}

Standard deviation is the square root of variance, bringing the units back to the original data:

s = s^{2}

For the normal distribution, roughly 68% of values fall within ±1 standard deviation of the mean, 95% within ±2, and 99.7% within ±3 — the empirical rule. The Standard Deviation Calculator and Variance Calculator compute these from your raw data, including the crucial sample-vs-population distinction (divide by n-1 for samples, n for populations).

Percentiles: The Tail Tells the Story

For skewed data (response times, revenue, file sizes), percentiles are more honest than mean + standard deviation. The p50 is the median. The p95 is the value below which 95% of observations fall. The p99 captures the worst 1%.

Why percentiles matter for monitoring:

Mean hides the tail (the alert that cried wolf)
p95 captures the experience of the slowest 5% without being dominated by outliers
p99 captures the worst 1% — the users who are most likely to churn

Percentiles are not additive. The p95 of a sum is not the sum of the p95s. If API A has p95 = 200 ms and API B has p95 = 300 ms, the end-to-end p95 of calling both is not 500 ms — it's typically less, because the slowest 5% of A's calls don't always coincide with the slowest 5% of B's calls. This is why distributed tracing matters: you need to measure the end-to-end percentile, not sum individual ones.

Response time percentiles for a typical API. The median (180ms) looks healthy, but p99 reveals 1% of users wait over 1 second. The mean (350ms) is pulled up by the tail.

Probability: Modeling Uncertainty

Probability is the language of uncertainty. For developers, it appears in reliability engineering (what's the chance of outage?), capacity planning (what's the chance we exceed capacity?), and A/B testing (what's the chance this result is real?).

Distributions: The Shapes of Data

Every dataset has a shape — a distribution. The most important distributions for developers:

Normal (Gaussian) distribution is the bell curve: symmetric, mean = median = mode. Heights, measurement errors, and sums of many independent variables tend to be normal (thanks to the Central Limit Theorem, discussed below).

Log-normal distribution is skewed right: the log of the values is normal. Response times, file sizes, and revenue per user tend to be log-normal. The mean is much higher than the median. Use geometric mean or percentiles, not arithmetic mean.

Exponential distribution models time between events: time between requests, time between failures. It has a "memoryless" property: the probability of a failure in the next minute is the same regardless of how long it's been since the last failure.

Poisson distribution models counts of events in a fixed interval: requests per second, errors per hour, support tickets per day. If events arrive independently at rate λ, the count in any interval is Poisson-distributed with mean λ.

The Central Limit Theorem

The Central Limit Theorem (CLT) is one of the most powerful results in statistics: if you take sufficiently large random samples from ANY distribution (normal, skewed, it doesn't matter), the distribution of the sample means will be approximately normal.

This is why the normal distribution appears everywhere: any metric that's the sum or average of many independent contributions (response times averaged over a window, conversion rates averaged over days) tends toward normal, even if the underlying data is wildly skewed. The CLT is also why you can use normal-based methods (z-tests, t-tests) even when your data isn't normal — as long as your sample size is large enough (typically n > 30).

Correlation: The Number That Gets Misused

Correlation measures the linear relationship between two variables, ranging from -1 (perfect inverse) to +1 (perfect direct). A correlation of 0 means no linear relationship.

The Pearson correlation coefficient:

r = \frac{\sum ( x _{i} - x ˉ ) ( y _{i} - y ˉ )}{\sum ( x _{i} - x ˉ ) ^{2} \sum ( y _{i} - y ˉ ) ^{2}}

Correlation ≠ causation is the most violated principle in data analysis. Ice cream sales and drowning deaths are correlated — not because ice cream causes drowning, but because both increase in summer (a confounding variable ). In software: server load and error rates are correlated, but the cause is a third factor (a deployment that introduced both a memory leak and increased CPU usage).

Spurious correlations appear by chance when you test enough pairs. If you check 100 unrelated metrics, about 5 will show "significant" correlation at p < 0.05 purely by chance. This is why pre-specifying hypotheses matters — and why data mining without theory produces nonsense.

Inferential Statistics: Drawing Conclusions from Samples

Descriptive statistics summarize what you have. Inferential statistics let you draw conclusions about a population from a sample — which is essential when you can't measure everything.

Confidence Intervals: The Range of Plausible Values

A confidence interval is a range that likely contains the true population parameter. A 95% confidence interval of [2.1%, 2.5%] for a conversion rate means: "if we repeated this experiment many times, 95% of the intervals we'd compute would contain the true conversion rate".

The formula for a confidence interval on a mean:

x ˉ \pm z_{α /2} \frac{s}{n}

The key insight: the width shrinks with $\frac{1}{n}$ — to halve the margin of error, you need 4× more data. The Confidence Interval Calculator computes this from your sample mean, standard deviation, and sample size, for any confidence level.

Why confidence intervals beat point estimates. Reporting "conversion rate is 2.3%" hides the uncertainty. Reporting "conversion rate is 2.3% ± 0.2% (95% CI)" tells you the precision of your estimate. If the CI for the difference between two variants includes zero, the difference is not statistically significant.

Hypothesis Testing: Is This Real or Noise?

Hypothesis testing is the formal framework for deciding whether an observed effect is real or due to chance. The logic: assume the boring explanation (null hypothesis), compute the probability of seeing your data under that assumption (p-value), and reject the null if that probability is low enough.

The framework:

Null hypothesis (H₀): the default assumption (e.g., "the new feature does nothing")
Alternative hypothesis (H₁): what you want to prove (e.g., "the new feature increases conversion")
Test statistic: a number computed from your data (t-statistic, chi-square statistic, z-score)
p-value: the probability of seeing a test statistic at least as extreme as yours, assuming H₀ is true
Decision: if p < α (typically 0.05), reject H₀

The p-value is not the probability that H₀ is true. It is the probability of the data given H₀ — not the probability of H₀ given the data. This confusion is so pervasive that the American Statistical Association published a formal statement clarifying it.

The t-Test Calculator and Chi-Square Calculator compute test statistics and p-values for the two most common tests: t-test for comparing means (revenue, time, score) and chi-square for comparing proportions (conversion rates, click/no-click).

Statistical Power: The Probability You'll Detect a Real Effect

Statistical power is the probability that your test will detect an effect if one truly exists. A test with 80% power has a 20% chance of missing a real effect (Type II error).

Power depends on three things:

Effect size: larger effects are easier to detect
Sample size: more data → more power
Significance level: lower α → less power (stricter threshold)

The Sample Size Calculator computes how many observations you need for a desired power, given your expected effect size and α. Use it before running any experiment — an underpowered test is worse than no test at all, because it produces false confidence in a "no difference" result.

Practical Applications for Developers

Monitoring and Alerting

The most common statistical mistake in monitoring is alerting on the mean. For skewed distributions (response times, queue lengths, file sizes), the mean hides the tail. Alert on percentiles instead:

p95 for "most users are having a good experience"
p99 for "the worst 1% are having a bad experience"
Mean only for symmetric metrics (CPU utilization, memory usage).

Set thresholds using baselines, not absolutes. A response time of 500 ms might be normal for a database query but catastrophic for a cache lookup. Establish per-endpoint baselines using historical percentiles, and alert on deviations from the baseline (e.g., "p95 is 3× higher than the 7-day rolling average").

A/B Testing and Experimentation

A/B testing is hypothesis testing applied to product decisions. The same principles apply: pre-specify your primary metric, compute sample size in advance, run the test to completion, and report confidence intervals — not just p-values.

Common mistakes:

Peeking: checking results daily and stopping when p < 0.05 inflates false positives
Multiple comparisons: testing 20 metrics and reporting the one that's significant is p-hacking
Underpowered tests: running on too few users produces inconclusive results
Ignoring practical significance: a 0.1% lift may be statistically significant with 1M users but not worth shipping

Data-Driven Development

"Data-driven" means letting evidence guide decisions — but evidence can be misleading if you don't understand the statistics. Before trusting any metric:

Check the distribution. Is it normal or skewed? Use mean for normal, median/percentiles for skewed.
Look at the spread. A mean of 350 ms with σ = 50 ms is very different from a mean of 350 ms with σ = 400 ms.
Compute confidence intervals. A conversion rate of "2.3%" is less useful than "2.3% ± 0.2%."
Beware of confounding. Correlation is not causation. A metric may move because of a third factor you're not measuring.
Pre-specify hypotheses. Decide what you're testing before you look at the data. Post-hoc rationalization produces false discoveries.

Capacity Planning and Forecasting

Statistics also powers capacity planning: predicting how much infrastructure you'll need. If your traffic grows 10% per month, you can extrapolate when you'll need to scale. But extrapolation requires understanding variance — if traffic is highly variable (high standard deviation), you need more headroom than the average suggests.

The rule of thumb: plan for the p95 of expected traffic, not the mean. If your mean projected traffic is 10,000 requests per second, but the 95th percentile is 15,000, provision for 15,000. Otherwise, you'll be overloaded 5% of the time — which is roughly 3.6 hours per day.

Regression for forecasting. Simple linear regression fits a line to historical data (traffic over time) and extrapolates. The regression calculator on this site fits a line to your data and reports the slope (growth rate) and R² (how well the line fits). Use it to answer: "At current growth rate, when will we hit our limit?" But be cautious — regression assumes the future resembles the past. A viral post, a product launch, or a pandemic breaks that assumption.

Error Budgets and Reliability

Site Reliability Engineering (SRE) uses statistics to define error budgets. If your SLO (Service Level Objective) is 99.9% uptime, your error budget is 0.1% — roughly 43 minutes of downtime per month. Statistics tells you whether you're on track to meet that budget or whether you've already blown it.

The math: if you've had 30 minutes of downtime in the first 15 days of a 30-day month, you're on pace for 60 minutes — double your budget. But if your downtime events are random (Poisson-distributed), the variance matters. A month with 30 minutes of downtime could be normal variation or the start of a trend. Statistical process control charts (which plot metrics with control limits at ±3σ) distinguish between normal variation and a genuine shift.

Common Mistakes That Fool Experienced Engineers

Confusing correlation with causation. Two metrics moving together doesn't mean one causes the other. Look for confounding variables.
Ignoring sample size. A 5% conversion rate from 20 users (1 conversion) is not a reliable estimate. Compute confidence intervals.
Using mean for skewed data. Response times, revenue, and file sizes are almost always skewed. Use percentiles.
Not checking assumptions. t-tests assume approximately normal data (or large samples). Chi-square tests assume sufficient expected counts. Violating assumptions produces unreliable p-values.
Treating p < 0.05 as truth. A p-value of 0.04 means there's a 4% chance of this result under the null — not that the alternative is true. Replicate findings before acting on them.
Forgetting about practical significance. With enough data, any tiny effect becomes statistically significant. Ask: "Is this effect large enough to matter?"

Frequently Asked Questions

Q: What is the difference between mean and median?

A: Mean is the average (sum divided by count). Median is the middle value when data is sorted. Mean is sensitive to outliers; median is robust. Use mean for symmetric data, median for skewed data (response times, revenue).

Q: What is standard deviation?

A: A measure of spread: how far data typically sits from the mean. A small standard deviation means data is tightly clustered; a large one means data is widely scattered. For normal data, 68% falls within ±1σ, 95% within ±2σ.

Q: What is a percentile?

A: The value below which a given percentage of observations fall. p50 is the median. p95 is the value below which 95% of observations fall. Percentiles are robust to outliers and ideal for skewed data like response times.

Q: What is a confidence interval?

A: A range that likely contains the true population parameter. A 95% CI of [2.1%, 2.5%] means: if we repeated this experiment many times, 95% of the intervals would contain the true value. If the CI for a difference includes zero, the result is not statistically significant.

Q: What is a p-value?

A: The probability of observing a result at least as extreme as yours, assuming the null hypothesis (no effect) is true. A p-value of 0.05 means: if there were no real effect, you'd see a result this large 5% of the time by chance. It is NOT the probability that your hypothesis is true.

Q: What is statistical power?

Q: What is the Central Limit Theorem?

A: The principle that the distribution of sample means approaches a normal distribution as sample size increases, regardless of the underlying data distribution. This is why normal-based methods work even with non-normal data — as long as your sample is large enough (typically n > 30).

Q: What is the difference between correlation and causation?

A: Correlation means two variables move together. Causation means one causes the other. Correlation does not imply causation — both variables may be influenced by a third confounding factor. Ice cream sales and drowning are correlated (both increase in summer) but not causally related.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

API Design & Rate Limiting: Building APIs That Scale Without Breaking

Adolfo Pedernera — Wed, 26 Aug 2026 14:34:20 +0000

How to design REST APIs that scale: HTTP methods, status codes, pagination, rate limiting (token bucket, sliding window), JWT auth, and security best practices.

The API That Became a Weapon

In 2021, a developer at a major cloud provider wrote a blog post about an outage that cost the company an estimated $10 million in lost revenue. The root cause wasn't a hardware failure or a DDoS attack. It was a single internal API that was designed without rate limits. A misconfigured internal service began calling the API in a retry loop — 10,000 requests per second, each timing out and retrying immediately. Within minutes, the API was down. Cascading failures took out three dependent services. By the time engineers identified the source, customers had been unable to access their data for 4 hours.

The API had no rate limiting, no circuit breaker, and no Retry-After header to tell the client to back off. It was, in retrospect, an accident waiting to happen — and it happened.

This guide explains how to design APIs that scale gracefully: how to use HTTP methods and status codes correctly, how to implement rate limiting that protects your infrastructure without punishing legitimate users, how to handle authentication with JWTs, and how to avoid the security mistakes that turn APIs into attack surfaces. Whether you're building your first REST API or hardening a production service that handles millions of requests, the same principles apply.

REST API Design: Resources, Not Actions

The most common mistake in API design is treating endpoints like function calls. A poorly designed API looks like a list of actions: /getUser, /createOrder, /deleteProduct. A well-designed API looks like a map of resources: /users, /orders, /products. The difference matters because HTTP already provides the verbs.

REST (Representational State Transfer) models your application as a set of resources identified by URLs, and uses HTTP methods to perform operations on those resources. The four methods you'll use 95% of the time:

Method	Purpose	Idempotent	Example
`GET`	Retrieve a resource	Yes	`GET /users/42` returns user 42
`POST`	Create a new resource	No	`POST /users` creates a new user
`PUT`	Replace a resource entirely	Yes	`PUT /users/42` replaces user 42
`PATCH`	Update part of a resource	No	`PATCH /users/42` updates some fields
`DELETE`	Remove a resource	Yes	`DELETE /users/42` removes user 42

Idempotency means making the same request multiple times produces the same result as making it once. GET, PUT, and DELETE are idempotent: deleting user 42 five times leaves the database in the same state as deleting it once. POST is not: five POST /users requests create five different users.

This matters for reliability. Networks fail. Clients timeout. If a POST request times out, the client doesn't know whether the server processed it — retrying might create a duplicate. The fix is an idempotency key : a unique client-generated token (a UUID works well) sent in a header like Idempotency-Key: <uuid>. The server stores the key with the result and returns the cached result for duplicate keys.

URL design guidelines:

Use nouns, not verbs: /users not /getUsers
Use plurals for collections: /orders not /order
Nest for relationships: /users/42/orders for user 42's orders
Use query parameters for filtering: /orders?status=active&limit=20
Keep URLs lowercase with hyphens: /order-items not /orderItems

HTTP Status Codes: The Language Clients Speak

HTTP status codes are how your API communicates what happened. Using them correctly means clients can respond appropriately without parsing response bodies. The categories:

Range	Category	When to use
2xx	Success	The request succeeded
3xx	Redirection	The resource moved (rare in APIs)
4xx	Client error	The request was malformed or unauthorized
5xx	Server error	Something went wrong on your end

The codes you'll use most often:

200 OK — standard success for GET, PUT, PATCH, DELETE
201 Created — POST that created a new resource (include a Location header pointing to the new resource)
204 No Content — success with no response body (common for DELETE)
400 Bad Request — the request was malformed (missing required field, invalid type)
401 Unauthorized — authentication failed (missing or invalid credentials)
403 Forbidden — authenticated but not allowed to access this resource
404 Not Found — the resource doesn't exist
409 Conflict — the request conflicts with the current state (duplicate email, version mismatch)
422 Unprocessable Entity — the request was well-formed but semantically invalid (validation errors)
429 Too Many Requests — rate limit exceeded (include Retry-After header)
500 Internal Server Error — something went wrong on your end (don't expose details)

The 401 vs 403 distinction trips up many developers. 401 means "I don't know who you are" (authentication). 403 means "I know who you are, but you can't do this" (authorization). A user without a valid token gets 401. A regular user trying to access an admin endpoint gets 403.

429 Too Many Requests deserves its own section — it's the rate limit response, and how you implement it determines whether your API survives a traffic spike or a misbehaving client.

Rate Limiting: Protecting Your Infrastructure

Rate limiting is the practice of controlling how many requests a client can make in a given time window. It protects your API from abuse, prevents a single tenant from monopolizing resources, and ensures fair access across all users.

The challenge is doing this without frustrating legitimate users. A developer integrating with your API will hit rate limits during testing. A batch job running at midnight will hit rate limits. A user with a flaky network that retries failed requests will hit rate limits. Good rate limiting distinguishes between a client that's slightly over its limit and one that's attacking your API.

The Three Algorithms

Fixed window is the simplest approach: allow N requests per window (e.g., 1,000 per hour), reset the counter at the start of each window. The implementation is trivial — a counter per client that resets on a timer:

counter[client_id] += 1
if counter[client_id] > LIMIT:
    return 429

The weakness is the edge effect : a client that sends 1,000 requests at 10:59 and another 1,000 at 11:01 has sent 2,000 requests in 2 minutes, despite the "1,000 per hour" limit. The fixed window doesn't see that the two bursts overlap.

Sliding window log stores every request timestamp for every client and counts how many fall within the current window. It's precise but expensive: storing millions of timestamps per client is memory-intensive. A common optimization is the sliding window counter , which combines the current window's count with the previous window's count, weighted by how far into the current window you are:

co u n t = c u rre n t_{c} o u n t \times (1 - \frac{t im e _{i} n t o _{w} in d o w}{w in d o w _{s} i ze}) + p re v i o u s_{c} o u n t

This approximates the sliding window without storing every timestamp, at the cost of allowing slightly more than the limit at window boundaries.

Token bucket is the most flexible algorithm and the one used by most production APIs (including GitHub and Stripe). Imagine a bucket that holds a maximum of B tokens. Tokens are added at a constant rate r per second (e.g., 10 tokens per second). Each request consumes one token. If the bucket is empty, the request is rejected with 429.

The token bucket handles bursts naturally: a client that's been idle for a minute has accumulated 60 tokens and can burst to 60 requests immediately. But sustained traffic cannot exceed the refill rate. The algorithm is:

t o k e n s = min (B, t o k e n s + r \times Δ t) - 1

If tokens >= 0, the request is allowed. If tokens < 0, it's rejected.

The API Rate Limit & Cost Calculator computes utilization, throttled requests, and monthly cost for any of these algorithms given your limit, load, and price per request. Use it to estimate whether your current limits can handle a 10× traffic spike, or to plan the cost of a rate-limited batch job.

Communicating Limits to Clients

A rate limit without communication is just an error. Good APIs tell clients three things in every response:

What the limit is : X-RateLimit-Limit: 1000
How many requests remain : X-RateLimit-Remaining: 847
When the limit resets : X-RateLimit-Reset: 1687305600 (Unix timestamp)

When the limit is exceeded, return 429 Too Many Requests with a Retry-After header that tells the client how long to wait before retrying:

HTTP/1.1 429 Too Many Requests
Retry-After: 60
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1687305600

{
  "error": "rate_limit_exceeded",
  "message": "You have exceeded the rate limit of 1000 requests per hour.",
  "retry_after": 60
}

The Retry-After header is in seconds (or an HTTP date). A well-behaved client reads this header and waits before retrying — instead of retrying immediately and making the problem worse.

Practical implementation: Most production APIs implement rate limiting at the API gateway or load balancer level (Kong, NGINX, AWS API Gateway, Cloudflare) rather than in application code. This offloads the work and applies limits before requests reach your application servers.

Authentication: Knowing Who's Calling

Every API needs to answer two questions: who is this? (authentication) and are they allowed to do this? (authorization). The most common approaches:

API Keys

The simplest approach: give each client a unique key, and require it in every request (usually via header: Authorization: Bearer <key> or X-API-Key: <key>). The server looks up the key to identify the client and check permissions.

API keys are easy to implement but have a weakness: they don't expire. If a key is leaked (committed to a GitHub repo, logged in an error message, intercepted over an unencrypted connection), anyone who has it can impersonate that client. Mitigations: require HTTPS, allow clients to rotate keys, and set expiration dates on keys.

JWT (JSON Web Tokens)

A JWT is a self-contained token that encodes claims (user ID, permissions, expiration) in a JSON payload, signed by the server. The client sends the token in the Authorization: Bearer <token> header. The server verifies the signature and extracts the claims — no database lookup required.

A JWT has three parts, each Base64URL-encoded and separated by dots:

eyJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjo0MiwiZXhwIjoxNjg3MzA1NjAwfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

Header (eyJhbGciOiJIUzI1NiJ9): algorithm and token type
Payload (eyJ1c2VyX2lkIjo0MiwiZXhwIjoxNjg3MzA1NjAwfQ): claims — user_id, expiration, roles
Signature (SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c): HMAC of header+payload with server secret

The critical claim is exp (expiration). A JWT without an expiration is valid forever — if leaked, it's a permanent access token. Set short expirations (15 minutes to 1 hour) and use refresh tokens for longer sessions.

JWT ≠ encryption. The payload is signed, not encrypted — anyone who intercepts the token can read the claims (that's why you should never put secrets in a JWT). The signature only guarantees that the token was issued by someone with the server secret and hasn't been tampered with. The JWT Decoder parses tokens client-side, showing the header, payload, and signature — useful for debugging authentication flows.

OAuth 2.0

OAuth 2.0 is a delegation protocol: it lets a user grant a third-party application limited access to their resources without sharing their password. The flow: the user clicks "Log in with Google," Google asks for permission, and issues an access token that the third-party app can use on the user's behalf.

For machine-to-machine APIs (where there's no user to click "Log in"), use the client credentials flow: the client exchanges its client ID and secret for an access token. This is what backend services use to communicate with each other.

Security: The Mistakes That Get You Hacked

APIs are the most common attack surface for web applications. The OWASP API Security Top 10 lists the most critical risks. The ones every developer should know:

Broken Object-Level Authorization (BOLA)

Excessive Data Exposure

APIs often return more data than the client needs. If GET /users/42 returns the user's email, phone, address, and hashed password, the client may only display the name — but the sensitive data is now in the browser's network tab, in logs, and in any proxy between client and server. Return only the fields the client needs, and use different response schemas for different roles.

Lack of Resources & Rate Limiting

Without rate limits, an attacker can brute-force passwords, scrape data, or overwhelm your API with requests. This is the failure mode that caused the $10 million outage described earlier. Rate limit authentication endpoints aggressively (5 attempts per minute per IP) and API endpoints per-client.

Mass Assignment

If your API accepts a JSON body and binds it directly to a database model without filtering, an attacker can set fields they shouldn't control. A PATCH /users/42 endpoint that accepts { "role": "admin" } because the request body is bound directly to the model is a mass assignment vulnerability. Whitelist the fields that can be updated.

Security Headers

Every API response should include:

Strict-Transport-Security: max-age=31536000 — force HTTPS
X-Content-Type-Options: nosniff — prevent MIME sniffing
Cache-Control: no-store — don't cache responses with sensitive data

Practical Tips for API Development

Always version your API. Use URL versioning (/v1/users) or header versioning (Accept: application/vnd.api.v1+json). Never release a breaking change without a version bump.
Use pagination for collections. Returning 100,000 records in one response is a memory bomb. Use cursor-based pagination (?cursor=<token>&limit=20) for large datasets, or offset-based (?offset=0&limit=20) for smaller ones.
Return consistent error formats. Every error response should have the same structure: { "error": { "code": "...", "message": "...", "details": [...] } }.
Log requests, not bodies. Log the method, URL, status code, and response time. Never log request bodies — they may contain passwords or tokens.
Use HTTPS everywhere. No exceptions. Even internal APIs should use TLS.
Set request size limits. A POST endpoint without a body size limit is vulnerable to memory exhaustion attacks. Limit to what you actually need (1 MB for most APIs).
Implement circuit breakers. If a dependent service is failing, stop calling it after N consecutive failures and return a fallback response. This prevents cascading failures.

Frequently Asked Questions

Q: What is the difference between REST and GraphQL?

A: REST models your API as resources identified by URLs, with HTTP methods as verbs. GraphQL uses a single endpoint and lets clients specify exactly what data they need in the query. REST is simpler and cacheable; GraphQL is more flexible for complex data requirements.

Q: What is idempotency and why does it matter?

A: Idempotency means making the same request multiple times produces the same result as making it once. GET, PUT, and DELETE are idempotent; POST is not. For non-idempotent operations, use an idempotency key (a UUID) so retries don't create duplicates.

Q: What is the difference between 401 and 403?

A: 401 Unauthorized means authentication failed (missing or invalid credentials) — I don't know who you are. 403 Forbidden means you're authenticated but not allowed to access this resource — I know who you are, but you can't do this.

Q: What is a JWT?

A: A JSON Web Token is a self-contained token that encodes claims (user ID, permissions, expiration) in a JSON payload, signed by the server. The server verifies the signature and extracts the claims without a database lookup. JWTs are signed, not encrypted — never put secrets in them.

Q: What is the best rate limiting algorithm?

A: Token bucket is the most common in production (used by GitHub and Stripe) because it handles bursts naturally while capping sustained traffic. Fixed window is simpler but allows 2× bursts at window boundaries. Sliding window is more precise but more expensive to implement.

Q: What headers should a rate-limited API return?

A: X-RateLimit-Limit (the limit), X-RateLimit-Remaining (requests left), and X-RateLimit-Reset (when the limit resets). On 429 responses, include Retry-After (seconds to wait before retrying).

Q: How do I protect against BOLA?

A: Broken Object-Level Authorization — the most common API vulnerability. Every endpoint that takes an ID must check that the authenticated user is authorized to access that specific resource. Don't rely on clients to only request their own data.

Q: Should API keys expire?

A: Yes. Keys that never expire are a permanent liability if leaked. Set expiration dates (90 days is common) and allow clients to rotate keys. Require HTTPS to prevent interception.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

Web Performance on a Budget: Bundle Size, API Limits, and Database Indexes

Adolfo Pedernera — Tue, 25 Aug 2026 05:09:22 +0000

The three budgets every web app hits: bundle size, API rate limits, and database index selectivity. Learn the formulas, thresholds, and PR checklist.

Introduction

Every web app has three budgets, and only one of them lives in your code. The bundle budget lives in the browser — 244 kB gzip is where Lighthouse starts penalizing Time to Interactive. The API budget lives at the provider — 5,000 requests per hour on GitHub, 500 per minute on OpenAI Tier 1, 100 writes per second on Stripe. The database budget lives in the planner — an index with 0.5 selectivity will be ignored and the query will seq scan 50,000 rows. Exceed any of the three and the user pays: slower paint, 429 Too Many Requests, or a query that times out.

The budgets are linked by the same habit that breaks them: averaging. A bundle that is 71 kB lodash full on average is 7.2 kB lodash/get when imported correctly — the average hides the choice. An API that is 20 requests per minute on average is 800 per minute during a burst deploy — the average hides the spike. A table where status has two values has selectivity 0.5 on average but 0.95 for active and 0.05 for pending — the average hides the skew.

This guide makes the three budgets explicit, with the formulas, the thresholds, and a single PR checklist that covers frontend, backend, and database. Every number is reproducible with the three calculators that accompany it — the Bundle Size Impact Calculator, the API Rate Limit & Cost Calculator, and the SQL Index Selectivity Calculator. For the network that connects them, the Bandwidth Calculator sizes the pipe.

1. Bundle Size — The Frontend Budget

The smallest budget is the one the user downloads. A parsed package of 71.0 kB lodash full is 24.4 kB gzip and 20.1 kB brotli. On a 1.6 Mbps 3G link that is 122 ms of transfer before first-paint; on 9 Mbps 4G it is 22 ms. The same utility as lodash/get is 7.2 kB parsed, 2.9 kB gzip, 15 ms on 3G — an 8× saving from a one-line import change.

The formula is linear:

S_{gzip} = S_{parsed} \times r_{gzip}

T = \frac{S _{gzip} \times 8}{B \times 1000} + RTT

where B is Mbps and RTT is round-trip in seconds. The budget share is S_gzip / 244 × 100. The Bundle Size Impact Calculator evaluates all three plus brotli.

Code — the 8× mistake and the fix:

import _ from 'lodash'; // 71.0 kB parsed
import get from 'lodash/get'; // 7.2 kB parsed
import { get } from 'lodash-es'; // tree-shaken, 7.2 kB

Bundlers like webpack only tree-shake lodash-es and named ESM imports — the default lodash import is not shaken.

Reference — popular packages (minified+gzipped):

Package	Parsed (kB)	Gzip (kB)	% of 244 kB
lodash (full)	71.0	24.4	10.0%
lodash/get	7.2	2.9	1.2%
moment	66.5	19.8	8.1%
date-fns	19.0	5.8	2.4%
react-dom	130.0	42.0	17.2%

Gzip sizes — single import choice moves 21.5 kB (107 ms on 3G). Replace moment with date-fns and save another 14 kB (70 ms).

Tip: set a CI budget of 244 kB gzip total, 130 kB per route, and paste the calculator's Budget% into the PR description — future reviewers see the cost without rebuilding.

2. API Rate Limits — The Backend Budget

If the bundle budget is about bytes, the API budget is about tokens per minute. GitHub's 5,000 per hour is 83.3 per minute; OpenAI's Tier 1 is 500 per minute; Stripe's writes are 100 per second. All are token buckets: refill at a fixed rate up to a burst capacity. You can burst to capacity instantly, but sustained throughput cannot exceed refill.

Let L = limit per minute, D = demand per minute, r = retry fraction, p = price per request:

U = D / L

T = max (0, D - L)

R = T \times r

Monthly cost (30-day month, 43,200 minutes):

C_{month} = (D + R) \times 43200 \times p

At $2 per 1K ($0.002 per request), 500 per minute limit with 800 per minute burst and 30% retry gives U = 160%, T = 300 throttled/min, R = 90 extra/min, and Deff = 890/min during the burst — $3.56 for two minutes, or $76k per month if sustained. The fix is not more retries but a queue that smooths the burst.

Code — queue instead of retry:

import pLimit from 'p-limit'; const limit = pLimit(10); // 10/min
await Promise.all(urls.map(url > limit(() => fetch(url))));

Batch where the API allows it — GitHub GraphQL and Stripe batches turn N requests into 1.

The API Rate Limit & Cost Calculator evaluates U, T, R, and C_month for any per-second/minute/hour limit.

3. SQL Index Selectivity — The Database Budget

The database budget is the most misread. Selectivity S = R / N (rows returned / total rows) decides if the planner uses the index. At S = 0.5 (50% of rows) the index is ignored; at S = 0.001% (1 row) it is always used. The threshold is around 5–20% with default costs.

S = R / N

C_{seq} = N \times 1.0, C_{idx} = R \times 4.0

If C_idx < C_seq the planner chooses index scan. For status = 'active' on 100k rows where 95k are active, S = 0.95, C_idx = 380k vs C_seq = 100k — seq scan wins. For email = 'a@b.com' with one row, S = 0.00001, C_idx = 4 vs 100k — index wins 25,000×.

Code — check before indexing:

SELECT COUNT(DISTINCT status) FROM users; -- C
EXPLAIN SELECT * FROM users WHERE status='active'; -- rows, cost

If C < 100 on 100k rows, a single-column index on that column will be ignored for most values — use a composite (status, created_at) instead.

Distinct (C)	Rows/value	Selectivity	Planner
100,000 (unique)	1	0.001%	Index
1,000	100	0.10%	Index
10	10,000	10%	Borderline
2	50,000	50%	Seq Scan

Selectivity % (log) — unique is 0.001% (always indexed), 2 distinct 50% (never alone).

The SQL Index Selectivity Calculator reports S, efficiency 1−S, and the verdict with cost.

4. Putting It Together — The PR Checklist

A single PR that adds a dependency, a new API call, and a migration can bust all three budgets at once. Check them together:

Budget	Check	Tool	Gate
Bundle	`S_gzip / 244 < 5%` per new dep	Bundle Size Impact	CI 244 kB
API	`U = D/L < 80%` sustained	API Rate Limit	Alert at 80%
Database	`S = R/N < 10%` for indexed filter	SQL Index Selectivity	EXPLAIN rows

Begginer: paste the three calculator outputs into the PR description — reviewers see the numbers without pulling the branch. Senior: add the three gates to CI — bundle via Lighthouse CI, API via X-RateLimit-Remaining header check, index via EXPLAIN in migration tests. Sensei: make the checklist a required GitHub PR template so every service that ships JS, calls an API, and migrates a table is measured the same way.

For the network that connects them, the Bandwidth Calculator converts S_gzip to wall-clock time on 3G/4G.

5. Case Study — One PR, Three Budgets

A real pull request that adds a dashboard with a chart, an analytics API call, and a new filter illustrates how the budgets interact.

The PR: Add recharts for a new RevenueChart component, fetch /api/revenue?range=30d on page load, and add WHERE status = 'active' AND region = 'EU' to the revenue query.

Bundle check: recharts is 130 kB parsed, 42 kB gzip — 17.2% of the 244 kB budget. The existing bundle is 180 kB gzip, so the new total is 222 kB (91%). The Bundle Size Impact Calculator shows 42 kB + 3G 210 ms. The fix is code-splitting: move RevenueChart to React.lazy so the initial route stays at 180 kB and the chart chunk loads on demand.

API check: The dashboard polls every 30 seconds, so 2 requests per minute per user. With 500 concurrent users, D = 1,000 per minute. The API limit is 500 per minute, so U = 200%, T = 500 throttled/min. With 30% retry, R = 150, Deff = 1,150. The API Rate Limit & Cost Calculator shows the cost and the need for a queue. The fix is caching the GET for 60 seconds and batching the poll to 1 per minute via SWR.

Database check: The new filter WHERE status='active' on 500k rows where 80% are active has S = 0.80, efficiency 20% — the planner will seq scan. Adding AND region='EU' where EU is 10% of rows makes the composite S = 0.80 × 0.10 = 0.08 (8%) — now selective enough for a composite index (region, status) or (status, region) depending on which column is more selective. The SQL Index Selectivity Calculator confirms S = 8% → Index Scan, cost 32k vs seq 500k.

The PR ships when all three checks are green: bundle under 244 kB initial, API U < 80%, and EXPLAIN shows Index Scan. One budget in the red is a revert.

6. Monitoring After Ship

Budgets are not one-time checks; they drift. Bundle size grows with every npm install, API load grows with every new replica, and selectivity shifts as the table fills. Track the three on the same dashboard: Lighthouse CI for bundle (S_gzip trend), X-RateLimit-Remaining header sampling for API (U over time), and pg_stat_user_indexes scans for database (S per index). When any of the three crosses 80% of its budget, the next feature PR should include a budget-reducing change — a lazy chunk, a cache, or a composite index — before new functionality.

7. Practical Tips

Measure with the same tool you ship with. esbuild --metafile vs Bundlephobia differ by minifier — pick one.
Import the file, not the barrel. lodash/get vs lodash is 8× — the same holds for date-fns/format vs date-fns.
Queue bursts, don't retry bursts. A p-limit queue at L eliminates throttling without retries.
Check cardinality before indexing. COUNT(DISTINCT col) < 100 on 100k rows → composite, not single-column.
Read EXPLAIN rows, not just the index list. rows = S×N tells you the selectivity the planner actually used.
Version your budgets today. Keep the three thresholds (244 kB, 80% API, 10% selectivity) in a budgets.json or lighthouse-budget.json so CI and humans share the same numbers — drift happens when the budget lives only in a comment.

Frequently Asked Questions

Q: What is the 244 kB bundle budget?

A: A common Lighthouse threshold where total gzip over ~244 kB starts penalizing Time to Interactive. Use it as a reference; your product's budget may be 170 or 500 kB.

Q: Gzip or brotli for budgeting?

A: Gzip for the conservative gate (all CDNs support it), brotli for the optimistic check (most modern CDNs serve brotli, ~15 percent smaller).

Q: When should I retry a 429?

A: Retry 20–30 percent once with exponential backoff starting from the Retry-After header. Retrying 100 percent on an overloaded bucket just re-throttles.

Q: When does an index get ignored?

A: When selectivity is high — roughly above 10 percent with default costs. On 100k rows, keeping 10k rows (10 percent) is borderline; keeping 50k (50 percent) is always a seq scan.

Q: Can I fix low selectivity with an index?

A: Alone, no — a single-column index on a 2-value column is 50 percent selective. As part of a composite (status, created_at) that raises cardinality, yes.

Q: How do I know my API's limit?

A: Check the docs (GitHub 5,000/hour, OpenAI 500/minute, Stripe 100 writes/second) and the response headers X-RateLimit-Limit/Remaining/Reset. The calculator handles per second/minute/hour.

Q: Do I need all three calculators for every PR?

A: No — check the budget you touch. New dep → bundle, new API call → rate limit, new WHERE → selectivity. If a PR touches all three, check all three.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

Design Tools Guide - Contrast, Golden Ratio and Aspect Ratio

Adolfo Pedernera — Sun, 23 Aug 2026 21:24:27 +0000

WCAG color contrast, the golden ratio, and aspect ratios — how to combine readability and proportion tools for better layouts.

Introduction

Design decisions that look subjective usually reduce to two measurable questions: can people read it, and does the layout feel balanced? The first is answered by color contrast — the numerical relationship between text and background that determines legibility for everyone, and especially for people with low vision. The second is answered by proportion — how the dimensions of a frame, a block of type, or a page relate to one another. Both questions have precise answers, and both are what make the difference between a component that merely exists and one that communicates.

This guide brings the two questions together because working designers solve them together. You do not choose a contrast ratio and a frame size in isolation: a button's color must pass WCAG against the panel behind it while the panel itself sits at a proportion that fits the canvas. The Contrast Checker measures readability, the Golden Ratio Calculator measures balance, the Aspect Ratio Calculator measures shape, and the Color Converter and Ratio Calculator fill in the conversion work those three rely on.

By the end you will know what each tool computes, when it matters, where the numbers come from, and — just as important — where each number stops being useful. The goal is not to replace design judgment with formulas, but to give judgment a numerical backbone.

The Two Design Decisions: Readability and Proportion

Every visual decision can be filed under one of two headings: contrast or proportion. Readability is a contrast problem. Typography, color, and luminance all converge on one question — is there enough difference between the text and what is behind it for the eye to separate them? Proportion is a shape problem. It asks how the parts of a composition relate: how wide a block of type should be relative to its height, how a poster's panels divide, how an image frames a subject.

Contrast is enforced by a standard. The Web Content Accessibility Guidelines (WCAG) set numeric thresholds, tested by browsers, tools, and increasingly by legal requirement, so a designer can compute pass or fail with certainty. Proportion is guided, not enforced. There is no WCAG for a golden rectangle, because balance is cultural and contextual — but there are geometric relationships, like the golden ratio, that recur across centuries of design because they are consistently pleasant to look at.

The distinction matters because it changes how you use the numbers. A contrast ratio below the WCAG threshold is objectively wrong for body text — no amount of aesthetic judgment justifies it. A proportion that ignores the golden ratio is not wrong, merely one choice among many. Designers who treat every numeric tool as a hard rule end up with rigid work; designers who treat every tool as optional end up with unreadable work. The skill is knowing which number is a gate and which is a guide.

Color Contrast and WCAG Accessibility

Color contrast is the luminance difference between two colors, computed by comparing their relative luminances and expressed as a ratio from 1:1 (identical colors) to 21:1 (black on white). The ratio is standardized by WCAG and is the same ratio every contrast checker on the web reports, so a value computed here matches the value your testing tool reports.

The relative luminance of a color is a weighted sum of its red, green, and blue channels after each channel is linearized:

L = 0.2126 \cdot R^{'} + 0.7152 \cdot G^{'} + 0.0722 \cdot B^{'}

where each channel value is converted from the sRGB 8-bit range (0–255) using the piecewise curve defined in the WCAG specification. Two colors are compared by dividing the lighter luminance plus 0.05 by the darker luminance plus 0.05:

contrast ratio = \frac{L _{lighter} + 0.05}{L _{darker} + 0.05}

WCAG sets the following pass thresholds for normal text, large text, and user interface components:

Requirement	Minimum ratio	Applies to
AA normal text	4.5:1	All body copy, buttons, form labels
AA large text	3:1	Text at least 24px, or 19px bold
AAA normal text	7:1	Extended reading, long-form content
AAA large text	4.5:1	Large text under the strictest standard
UI components	3:1	Focus indicators, icons, input borders

The thresholds are not arbitrary. Four and a half to one is calibrated so that people with typical low-vision contrast sensitivity — roughly the level of someone in their eighties or with moderate visual impairment — can still read body text. AAA pushes that further to cover people with more severe loss. Horton and Quesenbery make the accessibility argument concrete in A Web for Everyone: contrast problems are the single most common reason users with low vision abandon a site, and fixing them costs nothing but a choice of color.

Practical steps when a color pair fails: darken the text, lighten the background, or widen the type. The Color Converter helps here — converting the failing color to HSL lets you drop the lightness channel by small increments and convert back, which keeps the hue intact while moving the ratio. A brand color that fails at 12px can often pass at large heading sizes, because large text only needs 3:1. That is why the ratio, not the color, is the design constraint.

The Golden Ratio and Proportion

The golden ratio, φ (phi), is the number 1.6180339887…, and it has a distinctive property: the ratio of the whole to the larger part equals the ratio of the larger part to the smaller part. In a line divided at the golden point, the larger segment is about 61.8% of the whole and the smaller about 38.2%. The value satisfies φ = 1 + 1/φ, and it is the positive root of the equation:

φ = \frac{1 + 5}{2} \approx 1.618

The Fibonacci sequence produces the same number as a limit: each term is the sum of the previous two, and the ratio of successive terms oscillates around φ, converging on it exactly. This convergence is why designers reach for Fibonacci numbers (1, 1, 2, 3, 5, 8, 13…) as a spacing scale — 8, 13, 21, 34 is close enough to a golden scale that a spacing system built from it inherits the balance without using decimals.

Famously, φ is used to build balanced rectangles in layout: a golden rectangle's sides are in the ratio 1:φ, and a range of other classic proportions sit nearby.

Proportions	Ratio	Typical use
Golden (1:φ)	1.618	Editorial grids, poster panels, image crops
Square (1:1)	1.000	Avatars, galleries, product tiles
√2 paper (1:1.414)	1.414	ISO A-series print, book margins
Fibonacci 3:5	1.667	Magazine column pairs
Fibonacci 5:8	1.600	Spacing and type scales

Classic layout proportions plotted by ratio. The golden ratio sits between the square and the √2 paper ratio, near the Fibonacci-derived 5:8 and 3:5 pairs that form practical integer spacing scales.

The golden ratio is best treated as a starting point for composition rather than a requirement. Livio's history of the number cautions against the mythologizing that claims every great work of art uses it — the evidence that the Parthenon or Leonardo's paintings deliberately follow φ is weak, and the number's power is that it appears wherever people have sought balanced proportions, not that it was universally applied. Use it to generate a width from a height (or vice versa) when you need a pleasant rectangle fast; the Golden Ratio Calculator splits a line or scales a value by φ in either direction.

Aspect Ratios and Delivery Formats

An aspect ratio is the proportional relationship between an image's width and height, written as two numbers reduced by their greatest common divisor: 1920 × 1080 reduces to 16:9. It answers a delivery question the golden ratio does not — not "does this feel balanced" but "what shape is this frame, and what else will it fit in."

Delivery formats are strict. A 16:9 video shown on a 21:9 monitor is letterboxed; a 21:9 film on a 16:9 screen is pillarboxed. Photographers crop for print sizes (4:3, 3:2), video and displays standardized on 16:9, and ultrawide monitors push 21:9. Each one is a hard constraint on the canvas, and knowing the target ratio before you start composing saves the most work. RED's filmmaking reference is blunt about this: choose the frame first, then light and compose for it — a ratio is a contract with the viewer.

The Aspect Ratio Calculator handles both directions of the problem: it reduces any width and height to its simplest ratio, and it finds the missing dimension when you know the target ratio and one side. It shares DNA with the Ratio Calculator, which expresses the same relationship as a fraction for pure math work — the aspect calculator adds the display framing (common names, decimals, percentages) that layout work needs.

Design Tools at a Glance

Tool	What it computes	Typical question
Contrast Checker	WCAG contrast ratio, AA/AAA pass/fail	"Is this text readable on this background?"
Golden Ratio Calculator	φ split and scale of any length	"What width feels balanced for this height?"
Aspect Ratio Calculator	Reduced W:H, missing dimension, common name	"What height fits 1920 px wide at 16:9?"
Color Converter	Hex, RGB, and HSL conversions	"What lightness keeps this hue accessible?"

The four tools form a small workflow. Convert a brand color with the Color Converter, check the pair with the Contrast Checker, size the frame with the Aspect Ratio Calculator, and refine the balance with the Golden Ratio Calculator. Each tool hands its output to the next, and none of them alone decides whether a layout works.

A Worked Example: A Call-to-Action Card

A concrete walkthrough shows how the four tools cooperate. Suppose you are designing a call-to-action card for a landing page: a headline, a body paragraph, and a button, on a solid background.

Start with the shape. The card will sit beside a 16:9 hero image, but the card itself should feel calmer and more editorial — a 3:2 frame is a classic choice. You want the card to be 600 px wide. The Aspect Ratio Calculator gives the matching height: 600 ÷ 3 × 2 = 400 px. That frame now composes cleanly with the hero, sharing its 16:9 language through the common factor of the 3:2 crop.

Next, the colors. Your brand gray, #767676, on the white card background is a common "muted" choice. The Contrast Checker reports a ratio of 4.54:1 — just above the 4.5:1 AA threshold for normal text, but below the 7:1 AAA threshold. That is a defensible result for a landing page: AA passes for body copy, and the requirement is met. If this were a long-form article where AAA is the target, you would darken the gray toward #555555 (7.46:1) to clear the stricter bar.

Then the spacing. You set the padding and the gap between headline, paragraph, and button using a Fibonacci scale: 8, 13, 21, 34. The Golden Ratio Calculator can refine the choice — split the 600 px width in the golden ratio and you get 371 px for the larger part and 229 px for the smaller. A two-column card with the text block at 371 px and the visual at 229 px inherits the golden proportion, while the integer Fibonacci gaps keep the vertical rhythm mechanical and consistent.

Finally, the button. Its background must clear the 3:1 threshold against the white card because it is a UI component. The Color Converter converts your brand color into HSL so you can raise its lightness without drifting hue, and the Contrast Checker re-verifies the new pair. The card is now measurable at every step: a 3:2 frame, a 4.54:1 text ratio, a golden horizontal split, and a 3:1+ component ratio. None of these numbers replaced design judgment — they made the judgment auditable.

Common Mistakes

Treating a pass/fail contrast check as the whole accessibility story. Contrast is the measurable core, but readability also depends on type size, line length, spacing, and target size for touch. A 4.5:1 pass with 10px gray text is still hard to read.
Chasing AAA everywhere. AAA on body text is a strong goal for long-form reading, but it is not required everywhere. A badge or a disabled button at 2.5:1 is acceptable in many designs; spending hours forcing every element to 7:1 usually produces flat, low-contrast-looking pages.
Applying the golden ratio as a law. Compositional rules are heuristics. If a 1.618 ratio makes a panel too wide for its content, a 4:3 or 5:8 frame is a legitimate choice. The ratio suggests, it does not mandate.
Checking contrast on the design canvas, not the delivery frame. A hero image cropped to 4:5 on social media may show the text over a different background region than the 16:9 desktop mockup. Verify the pair against the actual export.
Assuming all ratios are alike. 21:9 ultrawide panels are often physically 43:18 or 64:27. Enter the real resolution, not the marketing label, when you compute the ratio.
Skipping the browser check. Every contrast tool computes the same ratio, but rendering differences, fonts, and anti-aliasing affect perceived contrast. Spot-check pass/fail pairs in a real browser before shipping.

Frequently Asked Questions

Q: What is the difference between color contrast and an aspect ratio?

A: Color contrast is a luminance relationship between two colors, expressed as a ratio from 1:1 to 21:1, that determines text readability. An aspect ratio is a width-to-height relationship of a frame, like 16:9. Contrast answers whether something can be read; aspect ratio answers what shape the frame is.

Q: What does WCAG contrast ratio 4.5:1 mean?

A: It is the minimum contrast ratio WCAG AA requires for normal-size text. It means the lighter color is at least 4.5 times brighter (luminance plus 0.05) than the darker color. Text meeting 4.5:1 is readable for the majority of people with low vision.

Q: How do I compute the contrast ratio between two colors?

A: Convert each color to its relative luminance L by linearizing the sRGB channels and weighting them 0.2126R + 0.7152G + 0.0722B. Then divide the lighter luminance plus 0.05 by the darker luminance plus 0.05. The Contrast Checker does all of this automatically.

Q: Is the golden ratio the same as the Fibonacci sequence?

A: No, but they are related. The ratio of consecutive Fibonacci numbers (1, 1, 2, 3, 5, 8, 13…) approaches the golden ratio 1.618 as the sequence grows. The sequence gives an integer approximation of φ that designers use for spacing scales.

Q: How is the golden ratio used in layout design?

A: Most often to generate a balanced rectangle: multiply one side of a frame by 1.618 (or by its reciprocal 0.618) to get the other. It is also the basis of Fibonacci spacing scales and classic editorial grids. It is a guide for balance, not a compliance requirement.

Q: What is the most common aspect ratio for displays?

A: 16:9 is the global standard for HDTV, most online video platforms, smartphones, and computer monitors. Other common ratios are 4:3 (older monitors and photography), 3:2 (camera sensors), 21:9 (ultrawide monitors), and 1:1 (square social media images).

Q: How do I find the height given a width and a ratio?

A: Divide the width by the ratio's width component, then multiply by the ratio's height component. For example, 1920 px wide at 16:9 is 1920 ÷ 16 × 9 = 1080 px. The Aspect Ratio Calculator does this for any ratio.

Q: Do WCAG contrast rules apply to images and logos?

A: Large logos and incidental imagery are exempt from most WCAG contrast requirements, but text overlays on images are not. Any text that carries information — including on hero images, badges, and video captions — must meet the ratio for its size.

Q: Can a design pass contrast but still be inaccessible?

A: Yes. Contrast is necessary but not sufficient. Type size, line height, spacing, color-blindness-safe palettes, focus indicators, and motion reduction all affect accessibility. A 4.5:1 pass with a cramped line height and tiny type is still a poor experience.

Q: What is the fastest way to fix a failing contrast ratio?

A: Convert the color to HSL and lower the lightness of the background or raise the lightness of the text — or simply darken text and lighten background a few steps at a time, rechecking after each change. Moving to a bolder or larger type also lowers the required threshold from 4.5:1 to 3:1 for large text.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

Tokens per Second Benchmarks Explained: What You're Actually Measuring

Adolfo Pedernera — Wed, 19 Aug 2026 03:51:59 +0000

What tok/s really measures, how concurrency changes it, and why a single-user benchmark is not the whole story for local LLM performance.

A Few Moments Later… How Fast Is "Fast"?

Every interface in the world of local AI eventually shows you that dreaded spinner, and on the wrong setup it sits there long enough that your brain supplies the meme: "A few moments later…" That pause is a number wearing a disguise. Somewhere inside your machine, the model is grinding out tokens — fragments of words — and the only question that matters is how many of them it produces per second.

Tokens per second (tok/s) is the universal speedometer of local LLMs, quoted in every benchmark and every GPU review. But it is also one of the most misleading numbers in the field, because the same model can measure 45 tok/s or 793 tok/s depending on how you test it. This guide explains what the number actually means, why it moves so dramatically, and how to read a benchmark without fooling yourself.

What a Token Actually Is

Before speed makes sense, the unit has to. Models do not read words; they read tokens, which are chunks of text roughly three-quarters of a character on average in English. The word "calculator" might be one token or three, depending on the tokenizer, and this is not idle trivia — it is the reason the same prompt can cost a different amount across providers, as the Token Counter Calculator shows in practice.

Because tokens are the unit of both billing and speed, "tokens per second" is the single number that connects all three corners of the local AI decision: how fast the model answers (tok/s), how big the model is (parameters), and what it costs to run (hardware amortized over time). A model doing 50 tok/s reads roughly 100-150 words per second — comfortably faster than you can read. A model stuck at 5 tok/s feels like a slow internet connection in 1998.

The Single-User Number Is Not the Whole Story

Here is the trap: most consumer benchmarks report tok/s at one user, one request. That number describes a private chat with your own model, and for that scenario it is the right metric. But the moment you put a model behind an API or a team, the picture changes entirely, because serving engines handle many requests at once and the interesting number becomes throughput — total tokens produced across all users per second.

The mechanism that changes everything is continuous batching. A naive server waits for one request to finish before starting the next. A batched server fills idle GPU cycles with other requests, keeping the silicon busy. This is why vLLM's key innovation — continuous batching plus PagedAttention — lets one GPU serve dramatically more total traffic than a tool that processes requests one at a time.

The Concurrency Curve: Back Street vs Highway

Think of it like driving. At one car, a quiet back street and a six-lane highway are both quick — there is no traffic, so lane count does not matter. That is the single-user case, where Ollama and vLLM are genuinely close, with Ollama even slightly ahead on some hardware.

Now imagine rush hour. The back street gridlocks at a handful of cars; the highway keeps absorbing lane after lane of traffic. That is the concurrency curve for local inference. At a single request both tools idle along, but as concurrency climbs, the batching engine pulls ahead dramatically:

Throughput vs concurrency for Llama 3.1 8B on A100 40GB (Red Hat benchmark pattern). Single user: Ollama ahead. Heavy concurrency: vLLM up to ~19x total throughput.

Under sustained load, the Red Hat benchmark on an A100 40GB with Llama 3.1 8B measured vLLM peaking around 793 tok/s of combined throughput versus about 41 tok/s for Ollama — roughly a 19x gap that emerged only as concurrency rose from 1 to 256 users. Ollama even stayed behind when told to run 32 parallel workers. The exact multiplier varies by model, GPU, and workload — it is not a universal constant.

Throughput vs Latency: Two Numbers, Two Questions

The tok/s number hides a second, equally important split: throughput versus latency. Throughput is how much work the machine does per second (useful for a server). Latency is how long one specific user waits for their answer (useful for a chat). They pull in opposite directions — serving more users at once raises throughput but can stretch how long any single request takes to start producing.

That is why real serving guides pay attention to the 99th-percentile latency (P99), the wait experienced by the slowest 1% of requests. In the same Red Hat test, vLLM delivered an 80-millisecond P99 under concurrency while Ollama's tail latency ballooned to 673 milliseconds — an 8x difference that matters enormously for interactive applications, even though average latencies looked acceptable. Average numbers flatter the tool that fails occasionally; percentiles expose it.

For a single user running a private model, latency and throughput are almost the same number, and the headline tok/s figure is honest enough. For a server, ignore the single-user benchmark entirely — read the concurrency curve and the P99 column.

There is a third metric hiding between them: time to first token (TTFT), the delay between submitting a prompt and the first word appearing. TTFT is dominated by prompt processing — the model reading and attending to your entire input before it can start generating. A long RAG context or a multi-turn history makes TTFT grow, independently of how fast the model then generates subsequent tokens. Under concurrency, TTFT is where a poorly batched server shows its worst face, because queued requests wait behind others before their prompt is even processed.

The three numbers tell different stories. Tok/s describes sustained generation, P99 describes worst-case wait under load, and TTFT describes how the experience starts. A model can look great on all three for a chat and collapse on TTFT for a long-document workload, or vice versa. When you read a benchmark, ask which of the three it is actually reporting — many reviews only quote tok/s and quietly ignore the other two, which is exactly where the misleading numbers hide.

What Realistic Speeds Look Like in 2026

Consumer hardware in 2026 lands in predictable bands, and knowing them keeps expectations sane. A 7B model in Q4 on a fast gaming card typically generates in the 120-300 tok/s range, which is more than fast enough for interactive chat. A 70B model on the same hardware crawls to single digits or low tens, because the memory bandwidth and compute demand scale with size. Unified-memory Macs sit in the middle: large models fit easily, but generation speed is capped by their bandwidth ceiling.

The practical translation: for a personal assistant or coding autocomplete, anything above roughly 30-40 tok/s feels instant, and the difference between 80 and 200 tok/s is barely noticeable to a human. The tok/s arms race only matters when you are serving many users, where the concurrency curve decides whether your hardware keeps up at all.

What Speed Feels Like: Tokens to Real Words

Numbers on a screen are abstract until you translate them into the experience of actually waiting. Because a token is roughly three-quarters of a word, you can estimate reading speed by multiplying tok/s by 0.75 to get words per second, then by 60 to get words per minute. The result is a feeling you already have a reference for.

At 50 tok/s, a model produces about 2,250 words per minute — several times faster than any human reads. Long answers materialize almost instantly, and the only real pause is the initial delay before the first token, which is dominated by prompt processing rather than generation speed. This is the experience most people mean when they say local AI "feels fast enough."

Drop to 10 tok/s and the math becomes 450 words per minute. Still readable, but now you watch the text assemble across a couple of seconds, and a 500-word answer takes nearly a minute. The "A few moments later…" feeling starts here, not because the model is slow in absolute terms, but because it is slower than your patience.

The distinction between these two experiences is almost never the headline benchmark. A review that says "50 tok/s" tells you nothing about whether the first token arrived in 200 milliseconds or 4 seconds — and that first-token latency is what you actually notice when you hit Enter. When you test a model yourself, watch the gap between pressing Enter and seeing the first word, then judge the generation rate separately. Both matter, but they are different problems with different fixes: prompt processing is about compute and prompt length, while generation speed is about memory bandwidth.

A useful mental anchor: compare the model's reading speed against your own. If a model produces text faster than you can read it, generation speed is effectively a non-issue — your attention is the bottleneck, not the hardware. That threshold sits near 15-20 tok/s for a comfortable reader, which is why budget local setups on 7B models feel so satisfying and why a slow 70B model on the wrong card feels broken even when it is "technically working." Speed, in the end, is a relationship between the machine and the person waiting.

How to Measure Your Own Tokens per Second

You do not need a lab to get your number — just ten minutes and a model file. The two most common tools, Ollama and vLLM, both make measurement straightforward, and both are open source with their own documentation.

With Ollama , the fastest path is interactive: download a GGUF model, run ollama run <model>, and ask it to produce a fixed amount of text. The terminal prints timing statistics when the response finishes, including a tokens-per-second figure for that exact prompt on your exact hardware. Keep the prompt the same across tests and you get a repeatable baseline.

With vLLM , the honest number requires a tiny bit of setup because the tool is built for serving, not single chats. Point it at your model file, start the OpenAI-compatible server, and send a request through its /v1/chat/completions endpoint with a known prompt length. The response metadata includes timing information you can turn into tok/s, and you can raise concurrency by firing several requests in parallel to build your own concurrency curve.

Whichever tool you use, standardize three things so the number is comparable: the model (same quantization), the prompt length (same token count), and the hardware (same GPU and driver). Change any one and the benchmark changes with it. Also measure a couple of times — GPU thermals and background load shift first-run numbers by 10-20%.

Once you have your real tok/s, plug your model choice and hardware into the LLM Hardware Requirements Calculator to sanity-check that the setup makes sense before committing to it.

Practical Tips for Reading Benchmarks

Ask "at what concurrency?" A tok/s number without a concurrency context is meaningless. 45 tok/s at one user and 45 tok/s at 50 users are opposite results.
Match the model to your hardware. Benchmarks on A100s do not predict RTX 5090 performance. Find numbers for your actual card and quantization.
Check the P99, not the average, for serving. Average latency hides the requests that actually time out.
Read your own tok/s before buying. Download a GGUF, run the LLM Hardware Requirements Calculator for the model+context combo, then measure the real speed on your machine.
Do not pay for more speed than you can perceive. Single-user chat above ~40 tok/s is already instant. The budget is better spent on a bigger model than a faster small one.
When the numbers disagree, trust the method. A benchmark that does not state its concurrency, GPU, quantization, and context is marketing, not measurement.

Limitations and Edge Cases

Tokens per second is a snapshot, not a law. Real numbers shift with context length (longer prompts slow generation), temperature sampling (which changes how much the model reuses cache), quantization, and the exact runtime. Batch efficiency also depends on request mix — varied prompt lengths batch better than uniform ones. And tok/s is silent about quality: a blazing-fast small model is still a small model. Use it to compare how well a given model runs on a given box, not to compare models against each other.

Frequently Asked Questions

Q: How many tokens per second is 'good' for a local LLM?

A: For interactive chat, roughly 30-40 tok/s already feels instant to most people. Above that, gains are barely perceptible. For coding autocomplete, similar thresholds apply. Only servers need the 500+ tok/s numbers.

Q: Why is my model slower than the benchmark said?

A: Benchmarks use specific hardware, quantization, and context lengths. Longer context, a bigger model, a lower-bandwidth card, or background processes all cut real speed. Measure your own setup rather than trusting a headline number.

Q: Why does vLLM beat Ollama under concurrency but not at one user?

A: Continuous batching. At a single request there is nothing to batch, so Ollama's simpler pipeline can even edge ahead. As requests pile up, vLLM fills idle GPU cycles with other work, multiplying total throughput.

Q: Is a 70B model always slower than a 7B?

A: Roughly yes on the same hardware, because every token pass touches more parameters. The gap is often 10x or more. This is why fitting a big model matters less than fitting one that generates at a usable speed.

Q: What is the difference between throughput and latency?

A: Throughput is total tokens produced per second across all users (a server metric). Latency is how long one user waits for their response (an experience metric). Serving more users raises throughput but can worsen tail latency.

Q: Can I use tok/s to compare two different models?

A: Only loosely. Tokens are tokenizer-dependent, so two models can split the same text into different token counts. Tok/s compares how efficiently a model runs on a machine, not how good the model is.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

How to Estimate Local LLM Cost per Token

Adolfo Pedernera — Mon, 17 Aug 2026 16:48:20 +0000

Hardware amortization, electricity, and tokens per second translated into a real cost per token — compared against 2026 cloud API pricing.

The Pied Piper Trap: When Your Server Costs More Than It Makes

In Silicon Valley, Richard Hendricks builds a brilliant compression algorithm that eventually brings down his own company's infrastructure — the deeper he pushes, the more the hardware bill balloons while the product fights to justify it. It is fiction, but every developer who has self-hosted an LLM has lived a smaller version: the GPU sits in the corner glowing, the electricity meter spins, and somewhere in the back of your mind is the question "am I saving money or just paying a different bill?"

The honest answer requires a number most people never compute: the cost per token of your local setup. Cloud APIs advertise prices per million tokens; local hardware comes with a price tag but no per-token figure. Until you translate one into the other, "local is cheaper" and "local is expensive" are both vibes, not facts. This guide builds the math from first principles so you can answer the question for your own machine.

The Local Cost per Token Formula

The cost of a local token has three components, all of which you can estimate before buying anything:

cost/token = \frac{hardware amortization + electricity + operational overhead}{tokens generated per period}

Each piece matters differently depending on your setup. Hardware amortization dominates for a new GPU. Electricity dominates for a machine that runs 24/7. Operational overhead — your time, maintenance, cooling, storage — is the piece everyone forgets and the one that quietly makes self-hosting expensive for a small team.

The crucial insight is that the denominator grows with utilization. A GPU that generates a million tokens a month spreads its fixed cost across a million tokens; the same GPU generating ten thousand spreads the same fixed cost across ten thousand. This is why "local is cheap" and "local is expensive" are both true — the answer depends entirely on how much you actually use the hardware.

Amortizing the Hardware

Hardware is a fixed cost, so the way to price it is amortization: spread the purchase price over the useful life of the machine. The standard assumption for consumer GPU hardware is 36 months — long enough that the cost-per-month is meaningful, short enough that you are not pretending a GPU lasts forever.

A real example makes this concrete. An RTX 5090 lists at $1,999 MSRP, though street prices run higher. Amortized over 36 months at the MSRP, that is about $56 per month before electricity. If that card generates, say, 40 million tokens per month (a heavy but realistic single-user workload on a fast model), the hardware alone contributes roughly $1.40 per million tokens.

The formula for hardware per-token cost is simple:

hardware cost/token = \frac{price / months}{tokens per month}

If your usage is a tenth of that — 4 million tokens a month — the same card costs $14 per million tokens, suddenly more than many cloud APIs. The fixed cost does not care how much you use it; that is the whole trap.

Two refinements make the amortization honest. First, subtract a resale value : a 36-month-old consumer GPU still sells for a meaningful fraction of its original price, so the true cost is the purchase price minus what you recover at the end, spread over the months you actually own it. Second, remember that scaling means buying hardware again : moving from a 7B to a 70B model may mean a second card or a workstation, which resets the amortization clock and roughly doubles the fixed cost overnight. Amortization is not a one-time decision; it is a commitment that renews every time your requirements grow.

The deeper point is that hardware cost is a step function while cloud cost is a curve. Your local setup has discrete cost jumps every time you add a GPU or a bigger machine, and between those jumps the per-token cost keeps falling as you generate more. The cloud has no steps — the price per token is flat whether you send one request or a million. That difference in shape is why local favors heavy, growing usage and cloud favors light, variable usage, and why the two never converge into one clean answer.

A useful discipline is to re-run the estimate on a fixed schedule — monthly, or every time your usage visibly changes. Hardware prices, electricity rates, cloud pricing, and your own workload all drift over time, and the answer to "is local worth it?" drifts with them. A setup that was clearly worth it in March can quietly become a money pit by September, or vice versa, with no single event marking the flip. Treating the estimate as a living number, not a one-time calculation, is the difference between making a decision and pretending a decision stays made.

The Electricity Bill Nobody Mentions

Electricity is the variable cost that scales with actual runtime, and it is the piece most cost estimates skip entirely. The formula is straightforward:

electricity/month = watts \times hours \times rate

An RTX 5090 has a 575W thermal design power. Add the rest of the system — CPU, motherboard, fans — and a loaded inference box commonly draws 700-900W at the wall. At a 15 cents per kWh rate, running that box at full load for 12 hours a day lands near $40-55 a month. That is often more than the hardware amortization, and it is pure recurring cost that never goes away.

This is why the electricity component punishes always-on setups. A GPU that idles between requests still draws a baseline; a server that must stay responsive 24/7 pays the full tab even if you use it an hour a day. Cloud APIs, by contrast, charge only for the tokens you actually send — the idle time is free because it is someone else's electricity.

Putting It Together: Your Real Cost per Token

Combining the pieces for the same RTX 5090 at heavy usage gives a realistic picture. Hardware amortization contributes about $1.40 per million tokens, electricity another $1.00-1.40 depending on utilization, and operational overhead (your time, setup, maintenance) adds whatever you value your hours at. The honest all-in number for a well-utilized single-GPU setup lands in the range of $2.50-4 per million tokens — before you count your own labor.

Now compare that to cloud pricing in 2026. DeepSeek V4 Flash costs $0.14 per million input tokens and $0.28 per million output. Claude Sonnet 5 runs $2 per million input and $10 per million output, and GPT-5.6 tiers span a wide range depending on the model.

The pattern is the key takeaway: a well-utilized local box lands near the mid-tier cloud pricing , competitive with Sonnet 5 and far below the flagship tiers, but dramatically more expensive than the cheapest cloud models like DeepSeek Flash. If your workload can tolerate a smaller model, the cloud at $0.14/M is simply unbeatable — local hardware cannot touch that price per token no matter how hard you amortize it.

Cloud Pricing as the Benchmark You Cannot Ignore

Local cost only means something against a reference, and that reference is what the same tokens cost as an API call. The 2026 cloud market spans two orders of magnitude, which is exactly why a single "is local cheaper?" answer does not exist. The range, with verified August 2026 pricing, looks like this:

Provider	Model	Input $/1M	Output $/1M
DeepSeek	V4 Flash	$0.14	$0.28
DeepSeek	V4 Pro	$0.435	$0.87
OpenAI	GPT-5.6 Luna	$0.20	$1.20
OpenAI	GPT-5.6 Terra	$2.00	$12.00
Anthropic	Claude Sonnet 5	$2.00	$10.00
Anthropic	Claude Fable 5	$10.00	$50.00

Input price per million tokens for representative 2026 models. The gap between budget and flagship is an order of magnitude — and local hardware must compete against the whole range.

The way to use this table is to pick the cloud model closest to what you would run locally. If you would self-host a compact 7B model, the honest comparison is against DeepSeek Flash at $0.14/M input — and local loses, hard. If you would self-host a top-tier 70B, the comparison is against Claude Sonnet 5 at $2/M, where a well-utilized local box can genuinely compete.

This is why "local is cheaper" debates are always underspecified. The real question is cheaper than which model, at what utilization? Until you pin down both, you are comparing a number you have not computed against a range you have not scoped.

When Local Actually Wins the Cost Game

The numbers above reveal that local is rarely the cheapest option on pure cost-per-token. Where it wins is in the situations where the price per token is not the whole equation. Three cases justify self-hosting despite the math:

Privacy and data control. If your prompts contain sensitive data you do not want to send to an API provider, the cost of local is the price of keeping the data in the building. That is a compliance and risk decision, not a per-token optimization.

Predictable heavy usage. If you genuinely generate tens of millions of tokens a month on a mid-tier model, the fixed hardware cost gets spread thin enough to undercut the equivalent cloud model — the break-even analysis in the Local LLM Break-Even Calculator finds your exact crossover point.

Latency and availability control. A local model has no network hop, no rate limits, and no provider outages. For applications where a few hundred milliseconds matter or where reliability is contractual, that has real value even if the token price is higher.

In every other case — light usage, bursty demand, budget-conscious projects, or workloads that only need a small model — the cloud wins on cost. The discipline is to compute your own numbers instead of assuming.

The Step-by-Step Cost Estimate

Here is the process you can run today, without buying anything. First, decide your realistic monthly token volume — count your actual usage with the Token Counter Calculator rather than guessing. Second, price the hardware you are considering (card, PSU, RAM, storage) and divide by your amortization period in months. Third, estimate electricity from the card's power draw and your expected runtime. Fourth, add a line for operational overhead — your hours have a value even if you do not bill them. Finally, divide the monthly total by your monthly token volume to get cost per token, and compare it against the cloud price for the same class of model using the LLM API Cost Calculator.

The single most common mistake is skipping step one and assuming a usage level. Run the numbers for both a light month and a heavy month — the spread will tell you whether local is a rounding error or a liability. Most people discover that local is either obviously worth it or obviously not, and the grey zone is narrower than the marketing suggests.

Practical Tips for Cost Estimation

Amortize over 36 months, not "forever." GPUs become obsolete faster than they physically fail. A 36-month window is the honest middle ground.
Count electricity at the wall, not the TDP. Add 20-40% to the card's rated power to cover the system around it.
Include your time. Setup, updates, and debugging are real costs. If your hours are worth $50, a week of tinkering is $400 of operational overhead.
Recompute at your real volume. The break-even point lives at the intersection of usage and cost. Guess the volume wrong and every downstream number is wrong.
Remember the cloud has no idle cost. For bursty or unpredictable demand, paying per token beats paying per month for hardware you do not saturate.
Do not forget resale. A 36-month amortization can subtract a partial resale value at the end, which many people ignore and which makes local a bit cheaper in hindsight.

Limitations and Edge Cases

The per-token estimate is only as good as its assumptions, and several assumptions are easy to get wrong. Electricity rates vary wildly by region and time-of-day; a card running at idle draws far less than its peak TDP, so always-on setups may cost less — or more — than the naive formula. Token volume is tokenizer-dependent: the same workload produces different token counts across models, which silently changes the per-token comparison. And comparing local against cloud output-only pricing misses the reality that real API bills mix cheap input and expensive output. Finally, the largest unquantified term is usually your own labor — it does not appear in the formula unless you add it, but it is often the difference between local being a bargain and local being a hobby.

Frequently Asked Questions

Q: Is running a local LLM cheaper than cloud APIs?

A: Only under heavy, predictable usage. A well-utilized box lands near mid-tier cloud pricing, but the cheapest cloud models (like DeepSeek at $0.14/M input) beat local hardware on pure price per token. Local wins on privacy, control, and latency, not usually on raw cost.

Q: How much does it cost to run an RTX 5090 locally per month?

A: Roughly $56 in amortization plus $40-55 in electricity at 12h/day, before your labor. That translates to roughly $2.50-4 per million tokens at heavy utilization — competitive with mid-tier cloud, far above the cheapest cloud models.

Q: What is the break-even point for local vs cloud?

A: It depends on your model choice, hardware, and volume. Against Claude Sonnet 5 the crossover is around 135M tokens a month; against cheap DeepSeek pricing it may be unreachable on a single consumer GPU. Use the Local LLM Break-Even Calculator for your specific numbers.

Q: Why is local LLM cost per token never quoted by vendors?

A: Because it depends entirely on your utilization, electricity rate, and hardware. There is no fixed per-token price for hardware, so nobody can quote one honestly. You have to compute it from your own numbers.

Q: Does local cost less if I only use it occasionally?

A: Almost never. Occasional use leaves the fixed hardware and electricity costs spread over very few tokens, which pushes cost per token far above cloud rates. Light users should strongly prefer pay-per-token cloud APIs.

Q: How do I count my own labor in the cost?

A: Estimate the hours spent on setup, updates, and maintenance, value them at whatever your time is worth, and divide by your monthly token volume. It is often the largest hidden term and the one that turns 'free' self-hosting into a real expense.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

How Much VRAM Do You Really Need for Local LLMs?

Adolfo Pedernera — Sun, 16 Aug 2026 15:11:56 +0000

The VRAM equation explained: quantized model sizes, real 2026 GPU options, and the 'can I run it?' answer for any local model.

Can You Run It? The Question Every Gamer Already Knows

Every PC gamer knows the feeling: you open a store page, scroll to the system requirements, and hold your breath. "Can I run it?" The GPU shader count, the RAM floor, the dreaded minimum versus recommended specs. Local AI has inherited that exact anxiety, repackaged as a single metric: VRAM. When you read about a 70B model running "locally," the unspoken question is whether your graphics card actually has the memory to hold it.

The good news is that the math is refreshingly simple — more predictable than game performance, which depends on drivers, resolution, and scene complexity. An LLM either fits in your VRAM or it does not, and the calculation is a straightforward equation you can do on a napkin. This guide teaches you that equation, walks through the real numbers for 2026 hardware, and helps you answer the "can I run it?" question for any model.

The VRAM Equation in Plain Terms

Every model consumes memory proportional to its parameters, and that number depends on how you store the weights. The core rule of thumb:

VRAM \approx \frac{parameters \times bits per weight}{8}

This gives you gigabytes from a parameter count. At 16-bit precision, a 7B model needs about 14GB of weights alone. That is why almost everyone who runs models locally uses quantization — storing weights in fewer bits (8, 4, or even 3 bits) at a modest quality cost, which is how a 7B model fits into a mainstream 8GB card.

Quantization is not a lossy afterthought; it is the entire reason local AI is possible on consumer hardware. The 4-bit GGUF format is the de facto standard for local inference, and the numbers below use it as the baseline.

The precision ladder shows why the bit choice dominates everything else. The same 7B model goes from unfittable on most consumer cards at FP16 to comfortable on an 8GB card at Q4:

Precision	Bits/weight	7B model VRAM	Quality
FP16	16	~14GB	Reference
Q8	8	~7GB	Near-lossless
Q6	6	~5.5GB	Very good
Q4	4	~4GB	Good (sweet spot)
Q3	3	~3GB	Noticeable loss

VRAM needed for a 7B model at each precision level. Quantization is the lever that makes local LLMs practical on consumer GPUs.

Each step down roughly halves memory while adding a little quantization error. That is why the community standard for local use is Q4: it fits mainstream cards while staying close enough to the original quality that most users cannot tell the difference on everyday tasks. Going below Q4 is only worth it when the alternative is not running the model at all.

Quantized Sizes: The Numbers That Matter

Here is the practical reality for popular model sizes, in 4-bit quantization (the default you will encounter in GGUF files):

Model	Parameters	4-bit VRAM (weights)	Realistic total with context
Llama 3	7B	~4GB	~5-6GB
Mistral	7B	~4.8GB	~6GB
Qwen 3	14B	~8GB	~9-10GB
Llama 3	70B	~40GB	~44-48GB
Llama 3	405B	~200GB+	Needs multi-GPU or server

The "realistic total" column matters more than the raw weights. Your context window, KV cache, and the running application all live in the same memory. A 70B model at Q4 needs roughly 40GB of weights, plus several more gigabytes for a reasonable context window — which is why you will rarely see a 70B running comfortably on a single 24GB card without aggressive quantization or a short context.

The 2026 Hardware Landscape

The current consumer flagship is the NVIDIA RTX 5090 with 32GB of GDDR7 memory and 1,792 GB/s of bandwidth. That 32GB is the dividing line: it comfortably handles 7B, 14B, and even 32B models at Q4, but a 70B model only fits at Q3 or smaller — for Q4 70B, you are looking at either two cards or a workstation part.

The other popular route is unified memory. Apple's Mac Studio M5 Ultra packs up to 192GB of unified memory shared between CPU and GPU, which can run 120B+ parameter models natively that a discrete GPU could not touch. The trade-off is bandwidth: 819 GB/s on the M5 Ultra versus 1,792 GB/s on the 5090, which directly caps tokens-per-second generation speed.

Hardware	VRAM	Bandwidth	Best fit
RTX 4060 Ti	8GB	288 GB/s	7B Q4
RTX 4090	24GB	1,008 GB/s	14B-32B Q4, 70B Q3
RTX 5090	32GB	1,792 GB/s	32B Q4, 70B Q3
Mac Studio M5 Ultra	192GB unified	819 GB/s	70B-120B+ native

Beyond these consumer options there are two practical routes for the 70B-plus tier. The first is multi-GPU: two RTX 4090s give 48GB of combined VRAM, which fits a 70B at Q4 — but you need motherboard support, a high-wattage power supply, and a runtime like vLLM that shards the model across cards. The second is the new class of desktop AI appliances such as the NVIDIA DGX Spark with 128GB of unified memory, which puts a serious fraction of a datacenter node on your desk at $4,699. Each route trades money, complexity, and speed differently; the right choice depends on whether you need one model at scale or many models at convenience.

Why Context Eats VRAM (the Part Nobody Explains)

Almost every guide tells you the weight size and stops there. But the reason your "40GB for a 70B" plan falls apart in practice is the KV cache — memory the model uses to remember what has been said so far while generating. Every token in your prompt and every token the model has produced gets stored in this cache, scaled by the number of attention layers and heads.

The KV cache is why your usable VRAM shrinks as your conversation grows. A short prompt on a 7B model might use 1-2GB of cache; a long RAG document or a multi-turn chat can eat 4-8GB or more. The formula for cache size is roughly proportional to context length × model size, which is why the same model feels "fine" at a 2K context and suddenly OOMs at 32K.

This creates the practical rule that most people learn the hard way: your model fits, but your conversation does not. When you plan VRAM, decide your context budget first, then add it to the weight size. The Context Window Calculator helps you size realistic context needs for your workload before you buy hardware around the wrong assumption.

How to Check Your Real VRAM Usage

Theory gets you in the ballpark; measurement gets you the answer. Once you have a model running, you can confirm exactly how much memory it really uses instead of trusting the marketing numbers, and the tools to do it are free and built into your operating system.

On Windows, open the Task Manager and go to the Performance tab, then GPU. You will see "Dedicated GPU memory" — that is your VRAM. Run your model, start a conversation, and watch the number rise as the prompt grows. The difference between the idle reading and the in-use reading is the model's true footprint, and it will climb as your context window fills, which is the KV-cache effect in action.

On Linux, the command-line tools do the same job. nvidia-smi shows a live per-process memory breakdown for NVIDIA cards, while rocm-smi and radeontop cover AMD hardware. A simple loop that samples nvidia-smi --query-gpu=memory.used every few seconds while you chat with the model gives you a clear picture of both the peak and the steady-state usage.

The practical reason to measure is that the advertised figures and the real numbers rarely match. A "4-bit 7B" model ships weights of roughly 4GB, but the running process regularly holds 5-6GB once the tokenizer, runtime buffers, and a normal conversation are included. Knowing your real ceiling — not the theoretical one — is what turns "should fit" into "does fit." If you find yourself right at the edge, the levers are the same ones from earlier: drop to a smaller quantization, shorten the context budget, or offload layers to system RAM.

The VRAM Calculation Walked Through

Let us do the napkin math for a 70B model at Q4, because that is the case that confuses most people. First, the weights:

weights = \frac{70 \times 4}{8} = 35 GB

That is 35GB just for the weights, before anything else. Add a few gigabytes for the KV cache at a moderate context window, plus the model runtime overhead, and you are at roughly 40-44GB. Against an RTX 5090's 32GB, it does not fit — so the realistic options are Q3 quantization (smaller, some quality loss) or CPU offloading, where part of the model lives in system RAM and only active layers stream through the GPU.

For a 7B model at Q4, the same math gives ~4GB of weights. Add context and overhead, and an 8GB card (RTX 4060 Ti) handles it comfortably, which is why 7B models are the sweet spot for budget local setups.

Step-by-Step: Can You Run It?

Let us apply the process to a real decision, the kind you will actually face. Imagine you want to run a 32B parameter model for local coding assistance, with a 16K context window, on a budget.

Step 1 — compute the weights. At Q4, 32B parameters means 32 × 4 / 8 = 16GB of weights. That alone rules out any 8GB card and most 12GB ones.

Step 2 — budget the context. A 16K context on a 32B model needs roughly 3-5GB of KV cache plus runtime overhead. Realistic total: 20-24GB.

Step 3 — compare against hardware. An RTX 4090 (24GB) fits this at Q4 with a moderate context. An RTX 4060 Ti (8GB) does not even come close. The Mac Studio M5 Ultra (192GB) fits it trivially — but at 819 GB/s its generation speed is capped below what the 4090 achieves with its 1,008 GB/s, so your real choice is between the 24GB card for speed or the Mac for sheer capacity.

Step 4 — sanity-check the economics. Before spending, run your real workload through the LLM Hardware Requirements Calculator to confirm the model+context combo, and the Local LLM Break-Even Calculator to confirm you actually use enough tokens to justify the hardware over cloud API calls. A 32B model you use twice a month is cheaper as cloud tokens than as a $1,500 GPU.

That four-step flow — weights, context, hardware, economics — is the entire skill of answering "can I run it?" for any model you will ever see.

Practical Tips for Buying or Repurposing a GPU

Check your real constraint before buying. If you want 7B models, an 8GB card is enough; if you want 32B+, you are on the 24-32GB tier. Know the target before the budget.
Watch out for "weights only" marketing. Advertised VRAM figures are often weights-only. Add 2-6GB for context and overhead when planning.
Quantization is your friend, not a cheat. Q4 (4-bit) is the quality/size sweet spot for most local use. Q3 fits bigger models but the quality loss shows on complex reasoning.
Bandwidth determines speed, VRAM determines fit. A 70B model on a huge unified-memory Mac runs slower than a 7B on a fast gaming card. The model you can fit is not the same as the model you can enjoy.
Test before you commit. Download a GGUF file and measure your actual tokens-per-second with your real context before spending thousands on a bigger card.
When NOT to buy a GPU: if your workload is bursty and small, renting cloud tokens is cheaper — use the Local LLM Break-Even Calculator to check before you spend.

Limitations and Edge Cases

The VRAM equation is a strong heuristic, not a precise oracle. Real usage varies with context length, batch size, KV-cache behavior, and the exact runtime (llama.cpp, MLX, vLLM all manage memory slightly differently). The "realistic total" column assumes a moderate context window; heavy RAG workloads with large contexts can push 5-10GB beyond the table. CPU offloading blurs the line further — a model can "run" while partially streaming through system RAM, at a steep speed penalty. And unified-memory systems (Apple Silicon, DGX Spark) have a different memory model where the entire system RAM is the pool, so the math changes entirely.

Frequently Asked Questions

Q: How much VRAM do I need for a 7B model?

A: About 4GB of VRAM for the weights at Q4 quantization, plus 1-2GB for context and overhead. An 8GB card like the RTX 4060 Ti is the comfortable entry point for 7B models.

Q: How much VRAM does a 70B model need?

A: Roughly 35GB for Q4 weights, and 40-44GB with context and overhead. That exceeds a single RTX 5090's 32GB, so 70B at Q4 realistically needs two cards, a workstation GPU, or a high-VRAM unified-memory Mac.

Q: Can I run a 70B model on 24GB VRAM?

A: Only at Q3 or smaller quantization, or with CPU offloading. Q4 70B needs more than 24GB; you would sacrifice quality or speed to squeeze it in.

Q: What is quantization and why does it matter?

A: Quantization stores model weights in fewer bits (8, 4, or 3 instead of 16), shrinking memory use roughly proportionally at a small quality cost. It is the reason local AI is feasible on consumer GPUs.

Q: Does more VRAM always mean faster output?

A: No. VRAM determines whether a model fits; bandwidth determines how fast it generates. A huge-model Mac can be slower than a small-model gaming card because of the bandwidth gap.

Q: Is CPU offloading a good idea for a 70B model?

A: It lets a model run on insufficient VRAM by streaming layers through system RAM, but at a significant speed penalty. It is a stopgap, not a comfortable long-term setup.

Q: Can I run an LLM on a laptop with integrated graphics?

A: You can, but only small models at low speed. A 1-3B model in Q4 fits in shared system memory and will crawl along at a few tokens per second. For anything interactive, a dedicated GPU or an Apple Silicon Mac with unified memory is the practical entry point.

Q: What happens when a model does not fit in VRAM?

A: Two things: either the runtime crashes with an out-of-memory error, or it silently falls back to CPU offloading and runs drastically slower. The fix is smaller quantization, a shorter context window, or a bigger card.

Q: Is a 32GB card enough for everything in 2026?

A: It handles the realistic consumer sweet spot: up to 32B models at Q4 and 70B at Q3, with room for context. It does not run 70B at Q4 or 120B-class models comfortably — for those you need dual-GPU, a workstation, or unified memory in the 96-192GB range.

Q: Does batch size change VRAM needs?

A: Yes, dramatically. Processing multiple requests at once multiplies the KV-cache footprint per active sequence. Serving frameworks like vLLM handle batching efficiently, but a single-user setup with llama.cpp typically processes one request at a time, keeping cache usage lower.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

vLLM vs Ollama: Production Serving 2026

Adolfo Pedernera — Fri, 14 Aug 2026 12:16:51 +0000

Compare vLLM and Ollama for LLM serving in 2026 — architecture, verified performance under concurrency, and a decision framework for choosing or combining them.

Two Tools for Two Very Different Jobs

If you have run a large language model locally in the last two years, you have almost certainly touched Ollama. It is the tool that made local LLMs approachable: install it, pull a model, and run a chat in minutes. If you have served a model to hundreds of concurrent users in production, you have almost certainly touched vLLM. It is the workhorse behind many hosted inference platforms, built from the ground up for throughput at scale.

The mistake most people make is treating them as interchangeable. They are not. They are built for different workloads, different concurrency profiles, and different priorities. This guide explains the architecture that makes them different, shows the verified performance gap under real conditions, and gives you a decision framework for choosing — or combining — them in 2026.

The short version: at a single concurrent user, Ollama is simpler and can even be slightly faster. The moment you add concurrency — multiple users, parallel requests, a front-end app — vLLM pulls ahead, and the gap grows with the number of simultaneous requests.

What Each Tool Actually Is

vLLM: a production serving engine

vLLM, developed at UC Berkeley's Sky Computing Lab, is a high-throughput inference and serving library written in Python. It is not a "runner" that wraps a backend; it is a full serving stack with its own scheduler, memory manager, and batching engine. Its two signature ideas are PagedAttention and continuous batching.

PagedAttention manages the KV cache — the memory that stores prior tokens during generation — the way an operating system manages pages of RAM. Instead of allocating one contiguous block per request, it stores tokens in fixed-size blocks that can point to non-contiguous memory. This eliminates the memory fragmentation that wastes up to 60-80% of KV cache in naive implementations, letting far more requests share the same GPU memory.

Continuous batching goes further: instead of waiting for a whole batch of requests to finish before starting the next, it lets requests join and leave the batch as they complete. A request that finishes early frees its slot immediately, and a new request joins right away. This keeps the GPU saturated instead of idling while stragglers finish, and is the single biggest driver of vLLM's throughput advantage. vLLM supports 200+ model architectures and scales across multiple GPUs with tensor, pipeline, data, expert, and context parallelism.

Ollama: a local runner built on llama.cpp

Ollama is a Go-based application that runs open-source models locally, built on the llama.cpp engine (with MLX support for Apple Silicon). Its entire design philosophy is simplicity: a clean CLI, a small API, and models distributed through its registry. It was built for the developer sitting at a terminal, running a model on one machine, chatting with it directly.

That simplicity has a cost. Ollama is not designed for high concurrency. Its parallelism is capped by the OLLAMA_NUM_PARALLEL environment variable, which defaults to 4, and its scheduling model does not aggressively batch work the way vLLM does. For a single interactive user this rarely matters. For a service behind a load balancer it becomes the bottleneck.

The Verified Performance Gap

The most reliable independent benchmark available compares Ollama and vLLM serving Llama 3.1 8B on an NVIDIA A100 40GB across a concurrency range from 1 to 256 simultaneous requests. The results are unambiguous.

At a single request, the two tools are close. As concurrency rises, vLLM's continuous batching and PagedAttention take over, and the gap widens to nearly 20x at high concurrency: vLLM peaked around 793 tokens per second against Ollama's 41 tokens per second, with 99th-percentile latency of 80ms versus 673ms. Notably, Ollama remained behind even when its parallel limit was raised to 32.

Illustrative throughput vs concurrency for Llama 3.1 8B on A100 40GB. At single user they are close; under concurrency vLLM pulls ahead dramatically. Values approximate the Red Hat benchmark pattern.

The exact multiplier depends on the model, GPU, and workload — it is not a universal constant. A 2026 benchmark on an A100 80GB with Llama 3 8B found vLLM about 2.3x faster at 8 concurrent users (187 vs 82 tok/s) after Ollama actually edged ahead at a single user (45 vs 38 tok/s). Another test with a dual-GPU Qwen3 14B setup measured vLLM up to 3.2x ahead at 128 concurrent requests.

Two things are consistent across every source: at one user the tools are comparable, and under real concurrency vLLM wins by a large margin. Anyone repeating the claim that "vLLM is 20-29x faster" should treat that as a high-concurrency figure on a specific workload, not a general rule.

Feature Comparison

Feature	vLLM	Ollama
Primary use	High-throughput production serving	Local dev / single-user chat
Continuous batching	Yes (default)	No (capped parallelism)
PagedAttention	Yes (core design)	No
OpenAI-compatible API	Yes (+ Anthropic, gRPC)	Yes (via /v1, no stateful)
Multi-GPU / distributed	Yes (tensor/pipeline/data/expert)	Limited (single node)
Quantization support	GGUF, GPTQ, AWQ, FP8, MXFP8	GGUF (Q4_K_M, etc.)
Concurrency control	Dynamic scheduler	OLLAMA_NUM_PARALLEL (default 4)
Setup complexity	Higher	Very low
Language	Python	Go (llama.cpp backend)

Approximate relative throughput advantage of vLLM over Ollama as concurrency grows (1x at single user to ~19x at high concurrency). Illustrative, based on the Red Hat benchmark.

The Key Calculations

Token throughput

Throughput is the number of tokens generated per second across all requests combined. It is the metric that matters for cost and capacity:

t h ro ug h p u t = \frac{t o k e n s _{g e n er a t e d}}{t im e}

vLLM maximizes throughput under load by keeping the GPU saturated via continuous batching. Ollama's throughput plateaus because its parallelism cap and less aggressive batching leave GPU capacity idle between requests.

Latency percentiles

For interactive applications, the 99th-percentile latency (P99) matters more than the average, because it reflects the worst experience a user actually gets. Under concurrency, vLLM's P99 stayed at ~80ms while Ollama's degraded to ~673ms — the tail latency of Ollama under load is an order of magnitude worse.

Cost per token at scale

The practical consequence is cost. At 100 concurrent users, the number of GPU-seconds required to serve a fixed request volume is far lower on vLLM because it packs more work into each GPU. This connects directly to the Local LLM Break-Even Calculator: the throughput a framework extracts from your hardware determines how much self-hosting actually saves you.

The KV cache: where concurrency lives or dies

The fundamental resource that limits concurrent inference is not raw compute — it is the KV cache , the memory that holds the key-value representations of every token processed so far in a generation. Each active request holds its own KV cache for the duration of its generation, and with many concurrent requests the total can dwarf the model weights themselves.

Naive serving allocates one contiguous block of KV memory per request up front. Because requests generate different numbers of tokens at different rates, these blocks are mostly wasted: a request that finishes early leaves its reserved memory idle, and fragmentation prevents the freed space from being reused efficiently. This is where vLLM's PagedAttention is decisive. By splitting the KV cache into fixed-size blocks that can be scattered across memory and shared between requests, it eliminates most of that fragmentation, letting a far larger number of requests coexist on the same GPU.

The practical formula for sizing a server is therefore:

V R A M_{t o t a l} \approx W + K V_{p e r_{r} e q u es t} \times C

Where $W$ is the model weights, $K V_{p e r_{r} e q u es t}$ is the KV cache per active request (proportional to context length and model size), and $C$ is the concurrency. The ability to pack more requests per gigabyte of KV memory is precisely what lets vLLM serve higher concurrency on the same hardware — and why an Ollama default cap of four parallel requests underutilizes a data-center GPU that could hold dozens.

This is also why context length matters so much. A model serving very long contexts consumes KV memory far faster, so the trade-off between concurrency and context length is real. A server tuned for 200K-token contexts supports far fewer concurrent requests than one serving short prompts, regardless of framework. Understanding this relationship — rather than treating throughput as a single number — is the difference between a configuration that serves your users and one that OOMs at the worst moment.

When to Use Ollama

Ollama is the right tool when simplicity and iteration speed matter more than concurrency:

Local development — prototyping a prompt or testing a model on your laptop.
Single-user tooling — a CLI assistant, a personal notebook, an experiment.
Model exploration — pulling and comparing models from its registry.
Education and learning — the lowest-friction way to get started with local LLMs.
On-device / edge scenarios — a small model on a workstation or Apple Silicon.

If your workload is one person talking to one model on one machine, Ollama is not just easier — it can be slightly faster, since it avoids vLLM's serving overhead. Its cold start is also faster (about 3 seconds vs 9 seconds in one 2026 test), which matters for interactive-first use.

When to Use vLLM

vLLM is the right tool when you are serving a model to other people or machines:

Production API — multiple concurrent users hitting a model behind a load balancer.
Agents and automation — many parallel agents making inference calls.
High request volume — throughput and P99 latency are business requirements.
Multi-GPU models — models too large for one GPU need vLLM's distributed inference.
Quantization variety — AWQ, GPTQ, and FP8 support beyond GGUF.

The trade-off is operational complexity. vLLM requires more setup: it is Python-based, needs a scheduling and serving configuration, and rewards someone who understands GPU memory and batching. That complexity buys you the ability to serve dozens or hundreds of users from one node.

Common Mistakes

Assuming vLLM is always faster. At a single concurrent user it is comparable to, sometimes slower than, Ollama. The advantage only appears under concurrency.
Quoting a single throughput number without concurrency context."vLLM is 20x faster" is meaningless without the concurrency and workload it was measured at. Always ask: at what concurrency, on what model and GPU?
Using Ollama as a production server without tuning. Its default parallel limit of 4 becomes a hard ceiling for any real service. Raising OLLAMA_NUM_PARALLEL helps only marginally.
Ignoring tail latency. Average latency can look fine while P99 is unusable. Under concurrency, measure percentiles, not just averages.
Treating them as mutually exclusive. Many teams use Ollama for development and vLLM for the same model in production — the same GGUF or model files feed both.

A Practical Decision Framework

Ask three questions in order:

How many concurrent users? One, or a handful on the same machine → Ollama. Tens or hundreds → vLLM.
Is this a service with an SLA? If yes, you need vLLM's P99 control and throughput. If it is an interactive single-user tool, Ollama.
Do you need multi-GPU or specific quantizations? vLLM for distributed inference and AWQ/FP8; Ollama's GGUF simplicity otherwise.

In many real 2026 deployments the answer is a hybrid: develop and prototype with Ollama, then serve the same model to production traffic with vLLM. The model files are largely interchangeable, so you get the best of both — Ollama's low-friction iteration and vLLM's production throughput.

Hardware Considerations (2026)

Production concurrency was benchmarked on NVIDIA data-center GPUs — the A100 40GB and 80GB — which remain the reference for multi-user serving. Consumer cards like the RTX 5090 (32GB GDDR7) handle single-user and light concurrency well, but for sustained high-concurrency throughput, a data-center GPU with more memory bandwidth and larger VRAM, or multiple GPUs behind vLLM's distributed inference, is the practical choice.

There is no universal "minimum GPU" — it depends on model size, quantization, and context length. The reliable approach is to benchmark your specific model and concurrency target, exactly as the sources behind this guide did. Use the Hardware Requirements Calculator to estimate VRAM for your model and then measure real throughput on your target hardware.

Quantization as the multiplier

Quantization is the cheapest way to shift the trade-off in your favor, and it is where the two tools differ in flexibility. Ollama's strength is the GGUF format and its Q4_K_M and similar quantizations from the llama.cpp ecosystem — enough to run a model on consumer hardware that would otherwise not fit. vLLM supports GGUF too, but adds AWQ, GPTQ, and FP8/MXFP8 quantization that preserve accuracy better at scale and integrate with its batching scheduler.

A quantized model is not only smaller in VRAM; it directly reduces the KV cache per token and often improves throughput, because the GPU processes fewer bytes per operation. The catch is accuracy: more aggressive quantization trades output quality for speed and memory. In production, teams typically benchmark a Q4 versus a Q8 or FP8 version of the same model on their real workload to find the smallest quantization that still passes their quality bar. The token counter and context tools on this site help you estimate the memory and throughput impact of that choice before you commit hardware to it.

Frequently Asked Questions

Q: Is vLLM always faster than Ollama?

A: No. At a single concurrent user the two are comparable, and Ollama can even edge ahead due to lower overhead. vLLM's advantage appears and grows with concurrency: on an A100 40GB running Llama 3.1 8B, vLLM reached roughly 793 tok/s vs Ollama's 41 tok/s at high concurrency, but at one user they were close.

Q: Can Ollama serve a production API?

A: Technically yes, but it is not designed for it. Its parallelism is capped (OLLAMA_NUM_PARALLEL, default 4) and it lacks vLLM's continuous batching, so throughput and tail latency degrade under real concurrency. For a single user or light internal tool it is fine; for a public service use vLLM.

Q: Do Ollama and vLLM use the same model files?

A: Largely yes. Ollama uses GGUF (from llama.cpp), and vLLM also supports GGUF among many formats (GPTQ, AWQ, FP8, MXFP8). This makes it practical to prototype with Ollama and serve the same model with vLLM in production.

Q: What is continuous batching and why does it matter?

A: It lets requests join and leave a GPU batch as they complete, instead of waiting for the whole batch to finish. This keeps the GPU saturated and is the main reason vLLM's throughput scales with concurrency while Ollama's plateaus.

Q: Is the 20-29x claim about vLLM accurate?

A: It is a high-concurrency figure on a specific workload, not a general rule. A widely cited 23x number was measured against naive static batching, not Ollama. A Red Hat benchmark put vLLM around 19x ahead of Ollama at high concurrency on an A100. Always ask about concurrency and hardware before trusting a multiplier.

Q: Which is better for a single developer building an app?

A: For development and iteration, Ollama is the low-friction choice. When you deploy that app so many users hit the model concurrently, move the serving layer to vLLM. Many teams use both: Ollama to build, vLLM to serve.

Q: Do I need a data-center GPU for vLLM?

A: Not for small models or light concurrency — a consumer card like the RTX 5090 works. For sustained high-concurrency serving, a data-center GPU (like the A100 used in benchmarks) or multiple GPUs behind vLLM's distributed inference is more practical. Benchmark your model and concurrency target to decide.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide

Local vs Hosted LLMs: The Decision Framework

Adolfo Pedernera — Thu, 13 Aug 2026 19:03:21 +0000

A comprehensive framework for deciding between local LLMs and cloud APIs. Covers cost, privacy, latency, control, and the hybrid approach.

The Cathedral and the Bazaar, Revisited

In 1997, Eric Raymond published an essay that framed a fundamental tension in software: the cathedral (centralized, carefully crafted, top-down) versus the bazaar (distributed, chaotic, bottom-up). Three decades later, that tension defines one of the most consequential decisions in modern software architecture: should you run large language models on your own hardware, or send your data to someone else's cloud?

The question is not abstract. In 2026, a startup shipping AI-powered features faces it directly. So does a hospital wanting to analyze patient records with LLMs. So does a solo developer building a coding agent. The answer shapes cost structure, latency profile, privacy posture, and strategic flexibility for years to come.

This guide provides a framework for that decision — not a one-size-fits-all answer, but a way to think about the tradeoffs with clarity. We will examine cost (using the Local LLM Break-Even Calculator), performance, privacy, latency, vendor risk, and the increasingly popular hybrid architectures that combine both approaches.

A Brief History: From Mainframes to APIs

Centralized computing is not new. In the 1960s, organizations rented time on mainframes they could not afford to own. The personal computer revolution of the 1980s shifted power to the edge — every desktop became a computing platform. The cloud era of the 2000s recentralized, offering unlimited scale without capital expenditure. And now, in the 2020s, generative AI has created a new pendulum swing: the most capable models live in the cloud, but open-weight alternatives are powerful enough to run locally.

The economics of each era followed the same pattern: centralized solutions win on convenience and time-to-market; distributed solutions win on control, long-term cost, and independence. LLMs are no different.

The Cost Dimension

Cost is where most analyses start, and often where they end — incorrectly. The naive comparison is seductive: a cloud API charges $2-30 per million tokens, while a GPU costs a fixed amount amortized over years. But this framing misses the full picture. Use the Local LLM Break-Even Calculator to find your specific crossover point, and the LLM Hardware Requirements Calculator to determine which hardware can actually run your target model.

Beyond Per-Token Pricing

Cloud API pricing in 2026 spans two orders of magnitude. GPT-5.6 Luna costs $0.20 per million input tokens; Claude Fable 5 costs $10.00. DeepSeek V4 Flash undercuts everyone at $0.14. The right comparison depends entirely on which model tier your use case requires.

Self-hosting costs include:

Hardware amortization : A $3,500 RTX 5090 (current street price, well above $1,999 MSRP) amortized over 36 months is ~$97/month
Electricity : 575W × 16hr/day × 30 days × $0.17/kWh = ~$47/month
Engineering labor : 2-20 hours/month depending on scale, at $75-150/hour fully loaded
Model updates : Manual evaluation and redeployment (4-6 cycles per year)
Opportunity cost : Time spent on infrastructure instead of product development

The Break-Even Calculation

Using the Local LLM Break-Even Calculator, you can find your specific crossover point. As a rule of thumb with August 2026 pricing:

Against Claude Sonnet 5 ($2/$10 intro pricing): ~135M tokens/month with 2× RTX 5090
Against GPT-5.6 Sol ($5/$30): ~40M tokens/month with single RTX 5090
Against DeepSeek V4 Flash ($0.14/$0.28): essentially unreachable for most teams

The Utilization Trap

A GPU at 80% utilization has roughly half the effective per-token cost of the same GPU at 40%. At 10% utilization — common for bursty, unpredictable workloads — the cost is 8x higher than at 80%. This is why self-hosting rewards steady, predictable demand and penalizes sporadic usage.

The Privacy Dimension

Privacy is the non-cost reason that often drives the decision. When you send data to a cloud API, it traverses the internet, touches another organization's infrastructure, and may be logged, cached, or used for model training depending on the provider's terms. Even with strong encryption in transit and at rest, you are entrusting sensitive information to a third party's operational security.

Regulatory Landscape

In 2026, data protection regulations continue to tighten globally. GDPR in Europe, HIPAA in US healthcare, and emerging frameworks in Asia impose strict requirements on where data can be processed and stored. For organizations in regulated industries, local processing is often not a preference but a legal requirement. Cross-border data transfers face increasing scrutiny, and the Schrems II ruling has made US cloud providers' compliance more complex for European organizations.

The Trust Spectrum

Different providers offer different privacy postures:

Standard cloud APIs : Data may be used for training (opt-out available), retained for 30 days for abuse monitoring
Enterprise tiers : Data not used for training, shorter retention, available in select regions
Local deployment : Data never leaves your infrastructure

For use cases involving trade secrets, patient data, classified information, or simply user conversations you would not want exposed, local deployment eliminates an entire category of risk. The question is not whether cloud providers are trustworthy — it is whether you want to depend on their trustworthiness for your most sensitive data.

The Latency Dimension

Cloud APIs add network round-trip time — typically 50-200ms before the first token, depending on your location and the provider's infrastructure. For interactive use cases where a human waits on the response, this matters. A voice assistant that takes 200ms to begin speaking feels sluggish; a coding suggestion that appears after a perceptible delay disrupts flow.

Local inference on an RTX 5090 produces the first token in 50-100ms with no network dependency. For applications like voice assistants, real-time coding suggestions, or industrial control systems, this difference is decisive. The variance is also lower — you are not competing with other users for GPU time or subject to provider-side queuing.

However, for batch processing (summarization, classification, overnight report generation), latency is irrelevant — cloud APIs work fine and scale automatically. The latency dimension only matters when a human or real-time system waits on the response.

The Control Dimension

Cloud APIs introduce dependency on an external vendor. This manifests in several ways:

Pricing changes : API prices have fallen roughly 80% over 2025-2026, benefiting users. But providers can raise prices, and switching costs are real. Teams that built products around GPT-4.1's $2/$8 pricing faced margin pressure when competitors launched cheaper alternatives. Building on local infrastructure eliminates this variability.

Model deprecation : When a provider retires a model, you must migrate. Cloud providers have retired capable models with limited notice, forcing rushed migrations. Local deployment means you control when and whether to upgrade. You can run a model indefinitely, even if the provider moves on.

Rate limits and availability : Cloud APIs enforce rate limits that require engineering workarounds (retry logic, request queuing). Outages are outside your control — when a major provider goes down, your product goes down with it. Local deployment provides predictable availability limited only by your own infrastructure.

Feature availability : New capabilities (tool use, vision, longer context) arrive on cloud APIs first. Local deployment lags by weeks to months depending on the open-source ecosystem. If your product requires the absolute latest capabilities, cloud APIs provide earlier access.

The Hybrid Architecture

For many teams in 2026, the optimal answer is not "local OR cloud" but "local AND cloud" — a deliberate hybrid architecture that routes each request to the most appropriate backend.

Routing Logic

A well-designed hybrid system routes based on:

Request Type	Recommended Backend	Rationale
High-volume, simple tasks	Local model	Free after hardware amortization; keeps GPUs utilized
Privacy-sensitive data	Local model	Data never leaves infrastructure
Latency-critical interactions	Local model	No network round-trip
Complex reasoning requiring frontier capabilities	Cloud API (GPT-5.6 Sol, Claude Fable 5)	Capabilities not available locally
Burst overflow beyond local capacity	Cloud API	Scale without provisioning
New model evaluation	Cloud API	Access before open weights available

Implementation Patterns

The simplest hybrid implementation uses a proxy layer that inspects each request and routes accordingly. Local inference typically runs on vLLM or Ollama depending on concurrency needs — vLLM delivers 20-29x higher throughput under concurrent load thanks to continuous batching and PagedAttention, while Ollama prioritizes simplicity for single-user scenarios. More sophisticated implementations use a local model for initial classification, then escalate to cloud only when confidence is low.

The key metric to track is GPU utilization. Move steady, high-volume traffic onto local models and the per-token figures start working in your favor; reserve cloud APIs for spikes and frontier-model calls.

Decision Framework: A Checklist

When evaluating local versus hosted LLMs for your specific use case, work through these questions:

Volume and utilization

◻️What is your projected monthly token volume?
◻️Is usage steady or bursty?
◻️Would local GPUs run at >40% average utilization?

Quality requirements

◻️Does your use case require frontier model capabilities (GPT-5.6 Sol, Claude Fable 5)?
◻️Can a 7B-30B open-weight model handle >80% of your requests?
◻️Have you benchmarked open-weight models on your actual workload?

Privacy and compliance

◻️Does your data fall under GDPR, HIPAA, or similar regulations?
◻️Are you willing to send data to a third-party API?
◻️Do you need data residency in specific geographic regions?

Latency and reliability

◻️Is sub-100ms time-to-first-token required?
◻️Can you tolerate cloud API outages?
◻️Do you need guaranteed availability SLAs?

Engineering capacity

◻️Does your team have GPU infrastructure expertise?
◻️Can you budget 5-20 hours/month for ops and model updates?
◻️Do you have hardware procurement and maintenance capacity?

If you answer "yes" to most volume, privacy, and latency questions, and "no" to frontier-quality requirements, local deployment deserves serious consideration. If you need frontier capabilities, have bursty usage, or lack infrastructure expertise, cloud APIs remain the rational starting point.

Practical Recommendations by Team Type

Solo Developers and Small Teams (fewer than 5 engineers)

Start with cloud APIs. The engineering overhead of self-hosting is a distraction from product development. Use the cheapest model tier that handles each task (DeepSeek V4 Flash for classification, Claude Sonnet 5 for complex work). Revisit when monthly API spend exceeds $500.

Growth-Stage Startups (5-20 engineers)

Run a hybrid stack. Self-host a 7B-14B model for high-volume, simple tasks (classification, extraction, routing). Route complex requests to frontier cloud APIs. This typically reduces cloud spend by 40-60% while maintaining capability.

Enterprise and Regulated Industries

Deploy local-first for any workload touching sensitive data. Use cloud APIs only for non-sensitive workloads and frontier capabilities. Budget for dedicated MLOps support (0.2-0.5 FTE per model in production).

AI-Native Products

If your product's core value is AI inference, self-hosting at scale becomes a competitive advantage. At 500M+ tokens/month on premium models, local deployment can save $50K+ annually versus cloud APIs — savings that drop directly to your bottom line. Some AI-native companies have made local inference a core part of their infrastructure strategy, offering it as a privacy guarantee to customers.

The Future: Convergence or Divergence?

The local-cloud tension will not resolve soon. Cloud models continue to improve faster than open-weight alternatives, but the gap is narrowing. Models like Qwen 3.6 and Gemma 4 deliver frontier-adjacent quality on consumer hardware. Inference tooling (vLLM, llama.cpp, MLX) matures monthly, making local deployment increasingly accessible.

What is converging is the tooling: deploying a local model in 2026 is dramatically simpler than in 2024. One-command setup, automatic quantization, and better GPU utilization have lowered the barrier significantly. What remains divergent is the capability ceiling: the most capable models (Claude Fable 5, GPT-5.6 Sol) are available only via API, and the gap between frontier and open-weight models persists for complex reasoning tasks.

The hardware trajectory also favors local deployment. Each generation of consumer GPUs brings more VRAM and bandwidth — the RTX 5090's 32GB fits models that required enterprise hardware two years ago. Apple's M5 Ultra with 192GB unified memory can run 120B+ parameter models natively.

The pragmatic path for most teams is to start with cloud, instrument actual usage, and migrate workloads to local hardware as volume grows and open-weight models improve. The Local LLM Break-Even Calculator tells you when that crossover point arrives for your specific numbers.

Frequently Asked Questions

Q: Should I run an LLM locally or use a cloud API in 2026?

A: It depends on your volume, quality requirements, and privacy needs. For most individuals and small teams, cloud APIs are cheaper and simpler. Self-hosting wins at high, steady volume (typically 50M+ tokens/month on premium models), when privacy regulations require local processing, or when sub-100ms latency is critical. A hybrid approach — local for baseline load, cloud for frontier capabilities and overflow — is optimal for many teams.

Q: What is the cheapest way to use LLMs in 2026?

A: For cloud APIs, DeepSeek V4 Flash at $0.14/$0.28 per million tokens is the cheapest frontier-class model. For local deployment, a used RTX 4090 ($1,600-2,000) running a 7B model at Q4 quantization delivers the lowest per-token cost. But remember: at low utilization, even 'free' local inference costs more than cloud APIs when you factor in hardware amortization and engineering time.

Q: How much does it cost to self-host an LLM?

A: Hardware: $1,800-5,000 for a consumer GPU (RTX 4090/5090) or $3,000-8,000 for Mac Studio. Monthly operating costs: $30-100 electricity, $150-3,000 engineering labor depending on scale. At full GPU utilization, raw inference costs $0.05-0.10 per million tokens. At realistic 10-30% utilization, effective cost is $0.50-5.00 per million tokens.

Q: Is local AI more private than cloud APIs?

A: Yes. Local deployment means data never leaves your infrastructure. Cloud APIs transit data over the internet and may log, cache, or (depending on terms) use it for training. For regulated data (healthcare, finance, classified) or sensitive business information, local processing eliminates an entire category of privacy and compliance risk.

Q: What GPU do I need to run LLMs locally?

A: For 7B-14B models: RTX 4090 (24GB) or RTX 5090 (32GB) at $1,600-5,000. For 70B models: 2× RTX 5090 or Mac Studio M5 Max/Ultra with 128-192GB unified memory ($3,000-8,000). The RTX 5090 fits 70B only at Q3 quantization or smaller; for Q4 70B, partial CPU offloading is required, dropping throughput significantly.

Q: How do vLLM and Ollama compare for production use?

A: Ollama (built on llama.cpp) is designed for simplicity — single-user, low-concurrency workloads. vLLM is optimized for production serving with continuous batching and PagedAttention, delivering 20-29x higher throughput under concurrent load. For personal use, choose Ollama. For production with multiple users, vLLM or similar production frameworks are the appropriate choice.

Q: What is prompt caching and how does it affect cost?

A: Prompt caching discounts repeated input tokens. OpenAI applies it automatically (~90% off cached input for prompts over ~1K tokens). Anthropic requires explicit cache breakpoints and charges ~25% extra on cache writes. For chatbots with repeated system prompts, caching reduces input costs by 30-50%.

Q: How do I estimate my monthly token usage?

A: Log token counts from API responses with a session ID and timestamp. Alternatively, use the Token Counter Calculator with your typical prompts to estimate average tokens per request, then multiply by request volume. Most workloads follow a 3:1 input-to-output ratio; coding assistants skew 8:1 input-heavy.

notAcalculator provides free online calculators and educational guides covering finance, fitness, mathematics, and everyday calculations.

🔗 Original guide