DEV Community: Muhammad Azlaan Zubair

AI Coding Tools and Productivity: What the Research Actually Found

Muhammad Azlaan Zubair — Thu, 12 Nov 2026 19:00:00 +0000

AI coding tools can make a developer feel faster while making a particular task take longer. They can increase the amount of code produced while also increasing review work, duplication, or security findings. Those statements are not necessarily contradictory. They describe different outcomes, measured at different levels.

That distinction is the most useful way to read the research on AI coding tools and productivity. Controlled experiments measure task completion under defined conditions. Repository analyses observe patterns in code changes. Security evaluations test whether generated snippets choose safe implementations. Surveys capture adoption, trust, and perceived impact. None of those measurements is a substitute for all the others.

The evidence supports neither a universal productivity boost nor a universal slowdown. It supports a narrower conclusion: AI changes the distribution of work. It can reduce effort in some tasks and contexts, while shifting cost into verification, integration, review, and maintenance. Teams need to measure those stages separately.

Why developer productivity is difficult to measure

“Productivity” sounds like one metric, but software development produces several outputs. A developer might finish a ticket sooner, write more code, resolve more tasks, reduce time spent searching, or improve a product outcome. Those are related, but they are not interchangeable.

A randomized task experiment can compare time or completion between AI-assisted and unassisted conditions, but a short task may not represent work in a large codebase or include downstream review costs. Field experiments are more realistic but vary by task, tool, and team. Surveys capture adoption and sentiment, not causal task speed. Repository analyses reveal code-change patterns, but trends do not establish what caused them.

A useful reading rule is to ask four questions before accepting a headline:

Who or what was measured? A developer, a task, a commit, a repository, a generated function, or a team?
What counted as success? Less elapsed time, more completed work, accepted code, fewer defects, or a positive self-report?
What was the comparison? No AI, a previous year, another team, a different model, or a vendor-defined baseline?
What remains outside the measurement? Review, debugging, security, maintainability, onboarding, or the value of the shipped product?

Without those details, “AI productivity” is too broad to be a useful conclusion.

Controlled studies: faster in some settings, slower in others

Controlled studies provide the clearest task-level comparisons, but their results depend heavily on the task and participants.

GitHub Copilot: a narrow task completed faster

In a 2023 controlled experiment, 95 professional developers were randomly split between implementing a JavaScript HTTP server with or without GitHub Copilot. The Copilot group completed the task 55.8% faster on average. This is evidence for acceleration on a bounded task—not a universal speedup. The study did not measure long-term maintenance or team delivery, and tested an earlier version of the tool. Read the study.

METR: experienced maintainers took longer

METR’s randomized controlled trial, published July 10, 2025, tested a different setting. Sixteen experienced open-source developers worked on 246 real issues in mature repositories they had contributed to for multiple years. The repositories averaged more than 22,000 stars and one million lines of code. Developers had moderate AI experience and an average of about five years of experience with the repositories.

Each issue was randomly assigned to an AI-allowed or AI-disallowed condition. Participants could choose their tools; in the AI condition they primarily used Cursor Pro with Claude 3.5 or 3.7 Sonnet. Tasks averaged about two hours. Participants recorded their screens and self-reported implementation time. METR reported that allowing AI increased task completion time by 19%. Before the tasks, developers expected a 24% time reduction; afterward, they still estimated that AI had made them 20% faster.

The result is striking because it contrasts perceived speed with measured time in a realistic, repository-specific setting. But METR explicitly cautioned against generalizing it to most developers or software work. The sample was small and highly experienced, and the tested tools represented the early-2025 frontier. METR also listed possible improvements in prompting, scaffolding, and repository-specific context as reasons the result might not transfer to other setups. Read METR’s study and its limitations.

A February 2026 update described a later experiment with 57 developers, 143 repositories, and 800+ tasks. METR said selection effects made its estimates unreliable: some developers declined to work without AI, and 30%–50% said they withheld tasks they did not want to attempt without it. Raw estimates suggested possible speedups, but confidence intervals included no effect. The update does not erase the 2025 result; it reinforces that it was time- and context-bounded. Read the update.

Three workplace field experiments: more tasks completed

A study by Kevin Cui and co-authors combined three randomized field experiments at Microsoft, Accenture, and an anonymous Fortune 100 company. Across 4,867 developers given access to an AI code-completion assistant or not, the combined analysis found a 26.08% increase in completed tasks (standard error: 10.3%). Less experienced developers had higher adoption and larger gains.

The individual experiments were noisy and varied. Completed task count is not the same as time per task or long-term code quality. The working paper was revised in August 2025 and published in Management Science on February 27, 2026. It counters a blanket slowdown narrative, but covers code-completion assistants in three enterprise settings—not every agentic workflow. Read the study.

These studies disagree only if we treat task time, task count, and work context as the same outcome. A bounded JavaScript task, a mature open-source issue, and ordinary enterprise work create different opportunities for assistance and different costs of review.

Large codebase analyses: quality signals, not a causal verdict

GitClear’s February 2025 report, AI Copilot Code Quality: Evaluating 2024’s Increased Defect Rate via Code Quality Metrics, analyzed 211 million meaningful changed lines from January 2020 through December 2024. Its data combined roughly two-thirds anonymized data from private corporations that opted in and one-third open-source projects, including projects associated with Google, Meta, and Microsoft.

GitClear classified code changes into operations such as added, moved, updated, copy/pasted, and churned lines. It reported that moved lines—used as a proxy for refactoring—fell from 24.8% of changed lines in 2021 to 9.5% in 2024. Copy/pasted lines rose from 8.4% to 12.3% over the same period. In 2024, copy/pasted lines exceeded moved lines for the first time in the report’s series. GitClear also found that commits containing a duplicated block of at least five contiguous lines rose from 0.45% in 2022 to 6.66% in 2024 in the commits it scanned.

These are maintainability signals, not direct measures of maintenance difficulty. Repeated code can create more places to update, but the time trend does not prove AI caused it or establish a randomized AI-versus-no-AI comparison. GitClear’s classification excludes generated files and other changes, includes comments, and does not separate test from production code. Its historical duplicate-block scan examined the largest 1,000 commits per repository for earlier years, which it says may bias those years toward finding more duplication. Treat the results as a reason to inspect local trends—not proof that AI-generated code is inherently duplicated. Read the report.

Security research: code that compiles can still be unsafe

Security studies test a narrower question than productivity studies: when a model is asked to generate code, does it choose a secure implementation?

Veracode’s 2025 GenAI Code Security Report evaluated more than 100 language models using 80 code-completion tasks across Java, JavaScript, C#, and Python. The tasks covered four vulnerability classes: SQL injection, cross-site scripting, log injection, and use of broken or risky cryptographic algorithms. Each task supplied a short function with a missing implementation and a comment describing the desired behavior. Veracode ran static analysis against the generated code.

Across the tested models and tasks, Veracode reported an average security pass rate of approximately 55%—meaning 45% of generated completions contained one of the known security flaws being tested. Results varied sharply by vulnerability type: models performed much better on SQL injection and cryptographic choices than on cross-site scripting and log injection. The report found little meaningful improvement in security performance across model sizes or release dates in its tested set.

The limitation is central to the result. These were intentionally constrained function-completion tasks with minimal context and no security-specific prompting. Veracode did not check functional correctness; it checked compilation and the selected security properties using its own static analysis engine. The report therefore does not mean 45% of production AI code is vulnerable. It means the tested models often chose insecure implementations under these task conditions. The vendor’s own report discusses the trade-off and the narrowness of the test. Read the report.

Apiiro’s September 2025 report looked at a different level: repository and developer activity inside Fortune 50 enterprises. Using its proprietary Deep Code Analysis engine, Apiiro analyzed tens of thousands of repositories and several thousand developers using a variety of coding assistants. It reported that AI-assisted developers produced three to four times more commits but fewer, larger pull requests; the company also reported ten times as many security findings, with more than 10,000 new findings per month by June 2025 compared with December 2024.

Apiiro reported a mixed risk profile: syntax errors fell 76% and logic bugs fell by more than 60%, while privilege-escalation paths rose 322% and architectural design flaws rose 153%. It also reported that AI-assisted developers exposed certain Azure credentials nearly twice as often as non-AI peers.

Those are Apiiro’s findings from its own enterprise telemetry, not an independently replicated causal experiment. The report does not provide enough public detail to independently reconstruct the full sampling, attribution, or normalization behind every comparison. Its “security findings” include a broad set of risks and should not be read as a count of confirmed exploitable vulnerabilities. The company also sells application-security products, so its commercial position belongs in the reader’s assessment. The evidence is a warning about the possible security workload created by high-volume changes, not a universal vulnerability multiplier. Read Apiiro’s report.

Surveys: adoption and trust are not productivity measurements

The 2025 Stack Overflow Developer Survey received 49,009 responses from 177 countries. It was fielded from May 29 to June 23, 2025, with respondents recruited primarily through Stack Overflow-owned channels. The survey reported that 84% of respondents used or planned to use AI tools in their development process, while 51% of professional developers said they used AI tools daily.

The same survey showed a trust gap: 46% distrusted AI output accuracy, compared with 33% who trusted it; 66% reported frustration with answers that were almost right, and 45% said debugging AI-generated code could take more time. These are self-reported sentiments, not causal productivity measures. Recruitment through Stack Overflow-owned channels also means the sample is not a probability sample of every developer worldwide. Read the survey and methodology.

DORA’s 2025 State of AI-assisted Software Development report used survey responses from nearly 5,000 technology professionals and more than 100 hours of qualitative data. The survey ran from June 13 to July 21, 2025. DORA reported that 90% of respondents used AI at work and more than 80% believed it increased their productivity. Its analysis associated greater AI adoption with higher software delivery throughput and product performance, but also with greater software delivery instability. DORA’s central interpretation was that AI amplifies the strengths and weaknesses of the surrounding organizational system.

This is organizational evidence, not a randomized trial of individual task speed. The reported relationships are associations, not proof that AI alone caused a team to improve or deteriorate. DORA moves the question beyond typing speed to whether increased output becomes a better product. Read DORA’s 2025 research overview.

Where the evidence agrees—and where it does not

Across these sources, several patterns are consistent:

Perceived productivity is not measured productivity. METR’s 2025 participants believed AI helped even when their measured task time increased. Surveys capture perception; experiments can compare observed task outcomes.
Task and context matter. A narrow implementation task, a mature repository issue, and enterprise delivery work do not test the same capability. Results should not be averaged into one universal speedup.
More code is not the same as more value. Commit volume, lines changed, and completed tasks are output measures. They do not automatically account for review cost, defects, maintainability, or product outcomes.
Verification remains part of the work. Veracode and Apiiro identify security risks under different methods, while Stack Overflow respondents report trust and debugging concerns. DORA connects AI adoption to delivery instability as well as throughput.

The disagreements are also real. GitHub’s bounded task, Microsoft’s three field experiments, and DORA’s 2025 organizational analysis report positive productivity or throughput signals. METR’s 2025 RCT reports a slowdown for experienced maintainers working on their own repositories. GitClear’s code-change trends and Apiiro’s security telemetry identify possible downstream costs. These findings should not be collapsed into a vote. They differ in period, tools, participants, tasks, outcomes, and measurement units.

What engineering teams should measure

For an engineering manager, the practical response is not to choose the most optimistic or pessimistic paper. It is to run a local evaluation that separates the stages where AI may save or add work.

Choose representative tasks. Include routine implementation, unfamiliar code, bug fixes, refactoring, tests, documentation, and cross-service changes. Do not benchmark only the tasks where the tool is strongest.
Record total cycle time. Measure from task start to accepted merge, including prompting, waiting, debugging, review, rework, and test failures. Keep active developer time separate from elapsed time when agents run asynchronously.
Measure accepted outcomes. Track completed and accepted tasks, review rounds, change size, reverts, escaped defects, and rework. Avoid treating lines of code or commits as productivity by themselves.
Inspect maintainability. Track duplication, churn, refactoring, dependency changes, and the age of code being modified. Compare these measures against your own baseline rather than assuming GitClear’s dataset describes your repository.
Add security checks. Review authentication and authorization boundaries, data handling, secrets, dependency provenance, and security-sensitive code paths. Static analysis and tests are useful controls, but neither replaces architectural review.
Segment the results. Compare by task type, developer experience, repository familiarity, tool/model, and workflow. A team-wide average can hide where assistance helps and where it creates rework.
Re-evaluate after tool changes. Model capabilities and agent workflows change quickly. A result from early 2025 is not a permanent property of AI-assisted development.

The most useful baseline is the team’s existing performance on comparable work. Run a small, time-bounded pilot, define success before starting, and keep the evaluation long enough to observe review and rework—not just initial code generation.

What I think: verification is the missing denominator

My interpretation: the central mistake in many AI productivity claims is measuring the generation step as though it were the whole engineering task. Code can appear quickly, but it still has to be understood, tested, reviewed, secured, integrated, and maintained. If the measurement ends when code is produced, it misses the work that determines whether the output is useful.

That is why verification deserves to be measured as a first-class part of the workflow. It is also the connection to two related Letinix essays: Code Generation Is Fast. Verification Is Not. and The Competence Debt of Agentic Coding. The first explores the verification bottleneck; the second examines the risk of delegating work that the developer can no longer confidently explain or assess. Their URLs were not supplied with this assignment, so they are noted for internal linking rather than linked to guessed destinations.

This is an editorial judgment, not a result directly established by any one study. The research supports the need to distinguish output, time, quality, security, and trust. It does not yet provide one agreed metric that captures the full value of AI-assisted engineering across organizations.

What to do next

If you want to use AI more deliberately, start with the AI Prompt Pack for Developers. Treat prompts as a way to improve task framing and context—not as a substitute for code review, tests, security analysis, or engineering judgment.

References

This article was originally published on Letinix.

Back-of-the-Envelope Estimation for System Design, With Worked Examples

Muhammad Azlaan Zubair — Tue, 10 Nov 2026 19:00:00 +0000

System design estimation is a way to turn an ambiguous product description into approximate traffic, storage, bandwidth, and capacity requirements. The goal is not to predict an exact production number. It is to expose the order of magnitude, make assumptions visible, and identify which part of the design deserves attention first.

A useful estimate follows a repeatable sequence: define the workload, calculate average and peak requests per second, split reads from writes, estimate data growth, calculate network throughput, and translate the results into design choices. The arithmetic should be simple enough to explain aloud and detailed enough for another engineer to challenge.

Note: Treat every number in the worked examples as an explicit hypothetical input, not a benchmark or a universal industry default. Replace assumptions with measured product analytics, load tests, payload samples, and retention requirements when sizing a real system.

The estimation method: assumptions before formulas

Start by describing what the system does. “A social feed” is not a workload; “two million daily active users each request twenty feed pages and publish half a post per day on average” is a workload you can calculate.

Write down the assumptions before doing arithmetic:

Users and activity: daily active users (DAU), actions per user, and whether actions happen evenly or in bursts.
Operation mix: reads, writes, uploads, downloads, background jobs, and internal fan-out.
Payload sizes: average request, response, record, and file sizes. Keep ingress and egress separate.
Retention: how long records, events, files, and derived data remain available.
Traffic shape: observed peak-to-average ratio, seasonal events, regional time zones, and retry behavior.
Capacity target: the utilization or headroom the system should retain during normal and degraded operation.

Ask which values are known, which are estimates, and which are deliberately conservative. If a prompt does not provide a value, choose a round assumption, say why it is plausible for the scenario, and show how the result changes if it is wrong. In an interview, this is part of the answer—not a detour before the answer.

The conversions worth keeping close

A day contains 86,400 seconds. For quick mental arithmetic, 100,000 seconds is a convenient approximation, but use 86,400 when showing a worked calculation. The approximation makes the result about 13.6% lower than the exact division, so it is useful for a first pass, not a substitute for checking a decision near a capacity boundary.

Time interval	Seconds
1 day	86,400
30-day month	2,592,000
365-day year	31,536,000

For traffic, remember the order of magnitude: one million requests per day is 1,000,000 ÷ 86,400 = 11.57 requests per second, or roughly 12 average requests per second. Ten million per day is about 116/s; one hundred million per day is about 1,157/s.

Be precise about data units. Decimal units use powers of 10: 1 MB = 1,000,000 bytes and 1 GB = 1,000,000,000 bytes. Binary units use powers of 2: 1 KiB = 1,024 bytes, 1 MiB = 1,048,576 bytes, and 1 GiB = 1,073,741,824 bytes. NIST documents the distinction between SI and IEC binary prefixes. NIST: Definitions of the SI units — binary prefixes

Use one convention throughout a calculation. In the examples below, storage totals use decimal GB and TB; file sizes use MiB where binary units are more natural. State the convention rather than switching between MB and MiB as if they were identical.

Core formulas for system design estimation

These formulas provide a starting model. They do not capture every system cost, but they expose the main quantities that influence an initial architecture.

Average and peak request rate

requests_per_day = DAU × requests_per_user_per_day
average_RPS      = requests_per_day ÷ 86,400
peak_RPS         = average_RPS × peak_to_average_factor

RPS means requests per second. QPS means queries per second; teams sometimes use the terms interchangeably, but define whether a “request” is an external API call, a database query, or an internal operation. One external request can fan out into several internal calls.

Do not assume one peak factor fits every product. Calculate it from observed time-bucketed traffic when possible: peak interval rate divided by the average rate for the same interval definition. If no telemetry exists, state a scenario assumption and test alternatives. A factor of 3 in an example means “we are modeling a peak three times the average,” not “all systems peak at 3×.”

Calculate read and write rates separately. A read-heavy product may need to serve a large volume of repeated data, while a write-heavy product may need sustained ingestion, ordering, durability, or downstream processing. A combined RPS figure can hide the real constraint.

Storage growth

new_data_per_day = writes_per_day × average_record_size
logical_storage  = new_data_per_day × retention_days
physical_storage = logical_storage × copy_factor

Add indexes, metadata, compaction space, snapshots, backups, and derived data separately. A copy factor is a simplified planning input, not a universal statement about how a managed database or object store implements durability. Compression also changes the stored size; estimate it from representative data rather than assuming a fixed saving.

Network bandwidth

bytes_per_second = requests_per_second × average_payload_bytes
bits_per_second  = bytes_per_second × 8

Calculate ingress and egress independently. Use the relevant request or response size for each path. Add protocol overhead, encryption, retries, fan-out, and retransmission where they materially affect the estimate. For large files, a small request count can still produce substantial bandwidth.

Server count

required_instances = ceil(peak_RPS ÷ sustainable_RPS_per_instance)

If the capacity target is below full utilization, use ceil(peak_RPS ÷ (sustainable_RPS_per_instance × target_utilization)). The per-instance rate must come from a representative load test or a defensible capacity measurement for the actual workload, software, instance type, and latency target. Do not copy a generic “requests per server” number into a design and present it as fact.

Worked example 1: a read-heavy feed

Assume a feed service has 2 million daily active users. Each user loads twenty feed pages per day and creates half a post per day on average. Model peak traffic at four times the daily average. Assume each post record occupies 1,000 bytes and each feed response transfers 12,000 bytes. These are illustrative inputs.

1. Calculate reads.

reads/day = 2,000,000 users × 20 reads = 40,000,000
average read RPS = 40,000,000 ÷ 86,400 = 462.96 ≈ 463
peak read RPS = 463 × 4 ≈ 1,852

2. Calculate writes.

posts/day = 2,000,000 × 0.5 = 1,000,000
average write RPS = 1,000,000 ÷ 86,400 = 11.57 ≈ 11.6
peak write RPS = 11.57 × 4 = 46.3

The workload is roughly 40 reads for every write. That ratio does not prove a cache is required, but it tells you the read path deserves more attention than the write path in the first design pass.

3. Estimate response bandwidth.

peak egress = 1,852 responses/s × 12,000 bytes
             = 22,224,000 bytes/s
             ≈ 22.2 MB/s ≈ 178 Mbps

This is application response payload only. It excludes transport overhead and any media served separately. If feed responses include images, model image delivery as a separate path rather than hiding it inside an average API response.

4. Estimate post storage.

new logical data/day = 1,000,000 posts × 1,000 bytes = 1 GB
one year              = 1 GB/day × 365 = 365 GB
three-copy scenario   = 365 GB × 3 = 1.095 TB

The 1.095 TB figure excludes indexes, metadata, backups, and media. It is not a complete database capacity plan.

Design implications: investigate caching or precomputing frequently requested feed data, and consider read replicas only if measured database read pressure justifies them. Keep writes on a simpler path initially unless ordering, fan-out, or consistency requirements change the picture. Separate media delivery from feed API responses, and validate the assumed read distribution: a highly personalized feed may not benefit from the same cache strategy as a feed of mostly shared content.

Worked example 2: a write-heavy event log

Assume 500,000 daily active devices each emit 200 events per day. Each stored event, including its basic metadata, averages 600 bytes. Model peak ingestion at five times the average and retain events for 30 days. Use a three-copy scenario for a first storage estimate.

1. Calculate event volume and ingestion rate.

events/day = 500,000 × 200 = 100,000,000
average write RPS = 100,000,000 ÷ 86,400 = 1,157.4
peak write RPS = 1,157.4 × 5 = 5,787

2. Calculate storage.

logical data/day = 100,000,000 × 600 bytes = 60,000,000,000 bytes
                 = 60 GB/day
30-day logical storage = 60 GB × 30 = 1.8 TB
three-copy scenario = 1.8 TB × 3 = 5.4 TB

The 5.4 TB estimate excludes indexes, compression effects, retention gaps, backups, and temporary compaction space. If event payloads vary substantially, calculate by event class or use a measured size distribution instead of relying on one average.

3. Check ingress bandwidth.

peak ingress = 5,787 events/s × 600 bytes
              = 3,472,200 bytes/s
              ≈ 3.47 MB/s ≈ 27.8 Mbps

The request rate is more revealing than the raw bandwidth in this example. The system must accept and durably process nearly 5,800 small events per second at the modeled peak, even though the byte rate is modest.

Design implications: consider an append-oriented ingestion path, batching, and a durable queue or log when producers and consumers need decoupling. Partitioning can distribute sustained write volume, but the partition key must avoid concentrating one hot tenant or event type on a single partition. Define ordering requirements before choosing a partition strategy: global ordering is much more restrictive than per-device or per-aggregate ordering. Use retention and compaction policies that match the actual query and audit requirements.

Worked example 3: a file upload service

Assume 100,000 daily active users upload two files each per day. The average file is 8 MiB. Model upload peaks at eight times the average. Assume each uploaded file is downloaded three times on average, downloads peak at four times their average, and files are retained for 30 days. For a simplified capacity scenario, count three stored copies.

1. Calculate upload traffic.

uploads/day = 100,000 × 2 = 200,000
average upload RPS = 200,000 ÷ 86,400 = 2.31
peak upload RPS = 2.31 × 8 = 18.52

The request rate looks small. The payload size changes the capacity picture.

2. Calculate upload bandwidth.

peak ingress = 18.52 uploads/s × 8 MiB
             ≈ 155.3 MB/s ≈ 1.24 Gbps

The conversion uses 8 MiB = 8,388,608 bytes and decimal MB for the final rate. This is a payload estimate, not a provisioned network target; protocol overhead and burst shape still matter.

3. Calculate storage.

data/day = 200,000 × 8,388,608 bytes
         = 1.678 TB/day (decimal)
30-day logical storage = 1.678 TB × 30 ≈ 50.33 TB
three-copy scenario = 50.33 TB × 3 ≈ 150.99 TB

4. Estimate download egress.

downloads/day = 200,000 uploads × 3 = 600,000
average download RPS = 600,000 ÷ 86,400 = 6.94
peak download RPS = 6.94 × 4 = 27.78
peak egress = 27.78 × 8 MiB ≈ 233 MB/s ≈ 1.86 Gbps

Design implications: this is a data-transfer and storage problem, not simply an API request-count problem. Consider direct-to-object-storage uploads using short-lived, scoped upload authorization so application servers do not proxy every file byte. Use asynchronous processing for malware scanning, metadata extraction, and thumbnail generation when those tasks do not need to block the upload response. A CDN may reduce origin egress for repeatedly downloaded files, subject to access control and cache policy. Confirm whether “three copies” is actually how the chosen storage service bills or exposes durability; do not multiply managed storage by three if its documented storage model already accounts for redundancy.

Compare the three estimates

Example	Peak primary operation	Data growth	Main pressure to investigate
Read-heavy feed	~1,852 reads/s; ~46 writes/s	1 GB/day logical posts	Read path, cacheability, feed fan-out
Write-heavy event log	~5,787 writes/s	60 GB/day logical events	Sustained ingestion, partitioning, retention
File upload service	~18.5 uploads/s; ~27.8 downloads/s	~1.68 TB/day logical files	Ingress/egress bandwidth and object storage

The examples are not directly comparable capacity benchmarks: their workloads, payloads, and assumptions differ. Their purpose is to show how the same method reveals different constraints.

Sanity checks before you trust the result

Run these checks before turning estimates into an architecture:

Recalculate from the other direction. Multiply average RPS by 86,400 and confirm that it returns the daily request count within rounding error.
Check units at every step. Requests/s × bytes/request gives bytes/s. Multiply by eight only when converting bytes to bits. Do not label MB/s as Mbps.
Separate external and internal work. If one request causes five database queries or three downstream calls, estimate those operations separately.
Check peak and average consistency. Apply a peak factor to the same time unit and traffic category. Do not multiply a peak rate by a daily count again.
Test the assumptions. Recalculate with half and twice the assumed activity, payload size, or retention. If the design changes sharply, the assumption needs measurement.
Keep logical and physical storage distinct. State whether the estimate includes copies, indexes, backups, compression, and derived data.
Look for missing workload dimensions. Concurrent connections, background jobs, retries, batch windows, and hot-key skew can matter even when average RPS looks modest.

Turn estimates into design decisions

An estimate is useful only when it changes a question or decision. Use the numbers to identify what to investigate—not to select a database or cache by formula alone.

Caching: consider it when reads repeat, data has an acceptable freshness window, and measured origin work is expensive. A high read/write ratio is a clue, not proof that every response is cacheable.
Replication: separate read scaling, availability, and durability requirements. Replicas add operational and consistency considerations; they are not a universal response to high RPS.
Partitioning: investigate it when write volume, storage growth, or data ownership exceeds a single partition's tested limits. Choose a key based on access patterns and skew, not only on even-looking IDs.
Queues and logs: use them when ingestion and processing need independent scaling, buffering, replay, or failure isolation. They add delivery semantics and operational complexity.
CDNs and object storage: consider them when large or repeated file transfers dominate bandwidth or application-server work. Access control and cache invalidation remain part of the design.
Capacity and headroom: use representative load tests to estimate sustainable per-instance throughput at the latency and error-rate targets that matter. Then calculate instance count with the formula above and add failure or deployment headroom explicitly.

For an interview, narrate the assumptions, show the arithmetic, identify the largest constraint, and explain what you would measure next. For a real design, record the inputs, source, confidence, and owner beside the estimate. A useful next step is to put the assumptions and capacity implications into a technical design document, where reviewers can challenge the numbers before they become infrastructure commitments.

Use the System Design Cheat Sheet as the companion reference while practicing these calculations. The cheat sheet is the primary resource for this topic; this article explains the reasoning behind the estimates and how to connect them to architectural choices.

References

Definitions of the SI units: The binary prefixes — National Institute of Standards and Technology

This article was originally published on Letinix.

The AI Bubble Question Is About Timing, Not Demand

Muhammad Azlaan Zubair — Tue, 10 Nov 2026 04:45:18 +0000

Whether the AI bubble is popping is the wrong question to start with. The demand side of AI is real and still growing quickly. The risk is a timing mismatch: the largest AI companies and their cloud suppliers have committed to spending that is bigger than their current revenue, much of it can't be cancelled, and the bills arrive before the revenue they are betting on. If demand keeps compounding on schedule, the build-out pays off. If it arrives a year late, the losses land on whoever is holding the fixed costs.

Note: Figures in this article are as of November 9, 2026. Anthropic's IPO pricing and Nvidia's quarterly results are both expected in mid-November and may change the picture quickly.

TL;DR

Demand is not the weak point. Google Cloud revenue grew 103% year over year in Q3 2026, and Anthropic reported $17.0 billion of revenue in the same quarter.
Commitments are. Anthropic's amended S-1 lists about $895 billion of compute and infrastructure commitments, with $69.8 billion due in the next 12 months, roughly equal to four times its latest quarterly revenue.
The strain shows up in cash flow and financing first: Alphabet's free cash flow turned negative for a second quarter, Oracle sold $10 billion of stock at a discount alongside $10 billion of new debt, and OpenAI cut its funding target by a third.
For builders, the practical response is to stop assuming today's AI API prices and providers are stable inputs.

What changed in October 2026

For most of the past two years, the AI investment story was told through rising valuations. In October and early November 2026, it started to show up in less flattering numbers.

On October 29, Alphabet's Class A shares fell 12.6%, their worst day since they began trading more than 20 years ago, according to the Associated Press. About $500 billion of market value was erased, per the Wall Street Journal's figure. Earnings and search revenue missed expectations, so AI spending was not the only cause, but negative free cash flow was a central part of the reaction.

The same week, the Financial Times, Wall Street Journal and CNBC reported that OpenAI had lowered the target for its latest private round from up to $30 billion to about $20 billion, at a $1.25 trillion valuation instead of the $1.4 trillion it had sought (summary of the reports). The round had not closed when it was reported.

On November 2, Oracle priced 86.2 million new shares at $116, about an 11% discount to the prior close, plus $10 billion of senior notes with coupons from 7.00% to 8.65% (Oracle's pricing announcement).

And on October 13, the International Monetary Fund's Global Financial Stability Report named an AI-driven market correction as one of three salient risks to global financial stability (coverage of the report).

None of these events, alone, is a crash. Together they show investors shifting from asking whether AI demand exists to asking when today's spending pays for itself.

Demand is real

The bubble framing often implies that the underlying product is hollow. The 2026 numbers don't support that.

Google Cloud revenue reached $30.71 billion in Q3 2026, up from $15.16 billion a year earlier (Alphabet's Q3 results, summarized by Techloy). Google Cloud's chief executive described "extraordinary demand" from AI labs, financial institutions and enterprises on the earnings call (SiliconANGLE).

Anthropic's amended S-1 reports Q3 2026 revenue of $16.96 billion, up 1,127% year over year, after $16.3 billion in the first half of the year (StockAnalysis summary of the filing). More than 11,900 customers pay it at least $100,000 a year (Renaissance Capital).

These are not the revenue lines of a technology nobody wants. If AI were a pure hype cycle, the argument would be simple. It isn't, which is why the more useful question is about timing.

The mismatch: commitments versus revenue

The clearest public view of the timing problem is Anthropic's IPO filing, because it is the first frontier AI lab to disclose its obligations in an SEC document.

The amended S-1 filed on November 2 lists approximately $895.4 billion of purchase commitments and lease agreements, which the company says relate substantially to compute and infrastructure (FourWeekMBA's reading of the S-1/A). About $69.8 billion falls due in the 12 months after September 30, and about $447.5 billion before the end of 2030 (The Register). The deals with Microsoft, Google, Amazon and Broadcom are largely non-cancellable, and some require payment regardless of usage.

The revenue side is shaped differently. Fortune's reading of the filing notes that much of Anthropic's revenue is consumption-based, and many contracts let customers reduce usage or terminate with limited notice (Fortune). The company contracts compute 12 to 18 months in advance.

Annualising Q3 revenue (4 × $16.96 billion) gives about $67.8 billion, slightly less than the $69.8 billion of commitments due over the same 12 months. That is a deliberately crude comparison: revenue is still growing, and commitments are not the company's only costs. It does show the shape of the bet. The obligations are sized for revenue that hasn't been booked yet.

Anthropic's chief executive, Dario Amodei, described this risk himself in February. Explaining on the Dwarkesh Podcast why the company was not buying compute even more aggressively, he said that if his growth forecast is off by a year, or growth is 5x instead of 10x, "then you go bankrupt" (Fortune). That is the timing thesis in the words of someone with every incentive to be optimistic.

Anthropic is the case with public numbers, not the only case. OpenAI has announced compute partnerships with Nvidia, Broadcom, Oracle and AMD, and the same logic applies to any company that commits to multi-year capacity against usage-based revenue.

Where the strain shows up first

Timing risk does not show up first as falling demand. It shows up in cash flow, in the cost of financing, and in projects that slip.

Cash flow. Alphabet spent $51.36 billion on capital expenditure in Q3 2026, up from $23.95 billion a year earlier, and reported free cash flow of negative $9.07 billion after negative $5.86 billion in Q2 (Techloy). It raised its 2026 capex guidance to $195 billion to $205 billion in July (Briefing.com). Across Amazon, Microsoft, Alphabet, Meta and Oracle, 2026 capex is projected at roughly $803 billion to $828 billion (StorageNewsletter). These companies can absorb negative quarters. Investors are now pricing how many more there will be.

Financing. Oracle is the case where the build-out leans most on borrowed money. Its debt stood at about $125 billion at the end of August 2026 (Political.org, citing Oracle's filings). Its stock peaked at $345.72 in September 2025 after a 36% one-day jump on its AI backlog, then fell to $114.50 in late July 2026, down about 67% (Motley Fool via Cbonds). The November share sale and new bonds at 7% to 8.65% show what capital costs once the market is less sure of the payoff.

Physical delivery. Oracle's 2.45 GW Project Jupiter campus in New Mexico depends on a gas pipeline that state regulators have denied twice. Oracle sent a force majeure notice to the developer, a Blue Owl Capital unit, preserving the right to defer payments if the campus misses its 2028 target (TechCrunch). Oracle says the project remains on schedule. The notice still shows who carries the risk when power, permits or construction run late: the parties with fixed obligations.

Circular financing makes timing errors spread

A timing miss at one company would be manageable if the industry's money flows were independent. They aren't.

The IMF report identified about $300 billion of equity transactions from hyperscalers into weaker AI firms over three years, about $650 billion of related debt, and about $1.4 trillion of future revenue commitments, and flagged these circular arrangements as a channel that could deepen a correction (coverage of the report). A Bank for International Settlements bulletin published on October 1 mapped the same overlap of investment and commercial relationships across more than 1,200 AI firms (BIS Bulletin No. 137).

The mechanism matters for the timing argument. When an investor is also the supplier, demand and investment are partly the same money. Backlog can look strong while depending on the customer's next funding round. If one lab's revenue arrives late, the effect passes to its cloud provider's backlog, that provider's debt, and its chip orders.

The depreciation assumption underneath

There is one more timing assumption buried in the accounting. Between 2020 and 2023, the large cloud providers extended the useful life of their servers from three or four years to about six, which spreads the cost of each server over more years and raises reported profit (SiliconANGLE).

Critics argue that AI accelerators turn over every two to three years, so six-year lives overstate earnings. Defenders, including Bernstein, argue that older GPUs move to inference and cheaper workloads and keep earning. In 2025, Amazon shortened the useful life of a subset of its servers from six years back to five, citing the pace of AI hardware development, while Meta lengthened some to 5.5 (Advisor Perspectives).

Neither side has settled this, and it does not need to be settled for the thesis to hold. If the useful life of today's hardware is shorter than the books assume, the window for revenue to catch up is shorter too.

The strongest counterarguments

Demand is outrunning supply, not the other way around. In July, Google said it would rely on higher-cost third-party capacity to serve very large cloud customers while its own capacity ramps up (Briefing.com). If that persists, today's capacity is not overbuilt, and the timing risk sits further out than the bears assume.

The leading lab is already generating cash. Renaissance Capital reports that Anthropic's operating cash flow turned positive at $6.6 billion, from negative $2.0 billion a year earlier, and that it held $95.6 billion of cash and $36 billion of short-term investments at September 30. About $213.8 billion of its commitments are subject to termination rights or conditions (FourWeekMBA). A company in that position has more room to absorb a slower year than the headline commitment figure suggests.

Headline losses overstate the problem. Anthropic's reported net loss for the first nine months of 2026 is about $104 billion (Idlen), but most of it is a non-cash charge from revaluing convertible notes as the company's valuation rose (TechSpot). Its Q3 operating loss was $567 million. A viral claim that the company "barely makes money" does not hold up against its own filing.

Overbuilt infrastructure has been useful before. Fiber laid during the dot-com boom carried the internet's next decade. The counterpoint is that fiber stays useful for decades, while accelerators lose competitiveness much faster, which brings the argument back to depreciation.

These points narrow the thesis rather than refute it. They suggest that a correction, if it comes, is more likely to look like financing stress, delayed projects and repricing than like collapsing demand.

What it means for builders

A financial correction in AI would not mean the technology stops working. After the dot-com crash, the infrastructure and many of the products survived, and useful companies were built on what was left. For engineers and teams shipping AI features, the practical implications are narrower:

Treat API pricing as a variable, not a constant. Prices could rise if providers need margin, or fall further under competition. Anthropic's filing explicitly flags the risk of being forced to cut token prices (Fortune). Design so your unit economics survive either direction.
Avoid a single-provider dependency for core features. A provider under financial pressure can change pricing, rate limits or model availability quickly. An abstraction over two providers, or an open-weight fallback, is cheaper to build before you need it.
Invest in cost engineering now. Caching, prompt and context reduction, routing simple requests to smaller models, and measuring cost per task are useful whether or not prices move.
Be skeptical of products that only work at today's prices. Thin wrappers whose margin depends on current token pricing are the most exposed if that pricing changes.
Measure outcomes, not adoption. MIT NANDA's 2025 report found that 95% of the organizations it studied saw no measurable return from generative AI, with the successful 5% tied to specific workflows (Campus Technology). It is a preliminary study with a small sample, but the lesson about measurable outcomes holds.

For a closer look at what the productivity evidence actually shows, see AI coding tools and productivity: what the research actually found.

Conclusion

The evidence as of November 2026 supports a narrower claim than "the AI bubble is popping." Demand for AI is real and growing. The fragile part is the calendar: multi-year, largely non-cancellable compute commitments sized for revenue that has not arrived yet, financed partly through circular deals and partly through debt, on hardware whose useful life is contested.

If growth continues at anything like 2026 rates, most of today's commitments will look reasonable in hindsight. If growth slows or slips by a year, the losses fall on whoever holds the fixed costs, and the effects travel through suppliers and lenders. The next useful data points are Anthropic's IPO pricing, Nvidia's mid-November results, and whether hyperscaler free cash flow recovers in early 2027.

References

This article was originally published on Letinix.

How to Write a Technical Design Doc That Gets a Decision

Muhammad Azlaan Zubair — Sun, 08 Nov 2026 19:00:00 +0000

A technical design doc helps a team decide what to build, why it matters, and which trade-offs to accept. It is not a place to document every implementation detail before the direction is agreed.

When a document gets no response, reviewers may not know what you need. When discussion jumps to minor implementation choices, the proposal may have described a solution before proving the problem. Write it as a decision proposal: lead with the question, support the problem with evidence, compare credible options, and state the decision required.

This guide offers a practical structure and a clearly hypothetical example. Adapt the depth to the change; this is not a universal company process.

What a technical design doc is for

A technical design document is a proposal for a technical change. It gives affected people enough context to evaluate the problem, challenge assumptions, and decide before implementation creates expensive commitments.

A useful design doc supports three outcomes:

Alignment before code: expose missing requirements and disagreements while the design can still change.
A record of reasoning: preserve why an option was selected and which constraints mattered.
A decision mechanism: state what is being approved, who decides, and when.

A detailed design can still fail if it does not explain why the change is needed, which alternatives were considered, or what reviewers must approve.

Google's documentation guidance describes design docs and PRDs as documents used to collect feedback on a proposed implementation, then retained as archives of decisions after implementation. Google's Architecture Decision Records guidance separately describes capturing options, requirements, decisions, and reasons. These are public examples of useful practices, not a claim that every team follows the same process.

Start with the decision, not the solution

Before writing the architecture section, complete this sentence:

We need a decision on whether to [specific choice] so that [intended outcome], by [date].

Then identify three things:

What is being decided? State the actual choice: approve a migration, select a storage pattern, change an API boundary, introduce a queue, or keep the current design.
Who decides? Name the decision owner or accountable group. Invite other people because they own affected systems, requirements, security, operations, or product constraints—not simply because they are senior.
By when? Give reviewers a realistic deadline and explain the dependency. A date tied to a planned implementation milestone is more useful than an unexplained “ASAP.”

If you cannot answer these questions, do more discovery first. Do not ask reviewers to infer the decision from a long document.

Hypothetical example: An engineering team sees intermittent timeouts while generating monthly customer exports. The engineer proposes moving export generation to a background worker. The decision is not “review my worker architecture.” It is: “Should we move exports to an asynchronous job flow, or first optimize the existing request path? We need the backend lead and service owner to decide by 14 May so implementation can be planned for the next cycle.” The roles and date are illustrative, not a real team process.

The sections that make a design doc useful

A design doc does not need a large template for every change. A small refactor may need a short proposal; a cross-service migration or security-sensitive change may need more context, diagrams, and operational detail. Include the sections that help reviewers evaluate the decision.

1. Context and problem: show what is happening

Describe the current state and the problem that makes a change worth considering. Be specific about who or what is affected, when the issue occurs, and what happens if the team does nothing.

Use inspectable evidence: metrics with units and time windows, incident reports, support tickets, traces or logs, system constraints, and concrete user scenarios.

Avoid vague statements such as “the architecture does not scale.” Name the observed constraint, such as requests timing out when generation exceeds a deadline. Include counts or impact estimates only when your data supports them; label incomplete evidence clearly.

For the export example, attach timeout data, incident or ticket references, and the current request flow. If unavailable, label the failure pattern an initial report, not a confirmed trend.

2. Goals and non-goals: define the boundary

Goals describe success. Prefer measurable outcomes with a known baseline: for example, a completion rate, processing-time target, latency budget, or reliability objective. Include the measurement window and source.

Do not invent targets because a template has a number-shaped blank. If the baseline is unknown, make baseline collection part of the proposal or label the target as an open decision.

Non-goals prevent scope creep. For the export example, goals might be completing long-running exports without holding an HTTP request open and exposing job status. Non-goals might include redesigning reporting or changing export formats.

3. Proposed design: explain the shape before the details

Start with a system overview a reviewer can understand quickly. Describe the main components, their responsibilities, and the path of data or control between them. A simple diagram is often more useful than several paragraphs of component names.

For the hypothetical proposal, the overview could show:

The client submits an export request.
The API validates it and creates a job.
A worker claims the job and generates the file.
The result is stored and the job status is updated.
The client checks status and retrieves the completed export.

Explain only decision-relevant details: job ownership, retries, idempotency, storage lifecycle, access control, failure handling, observability, and API changes. Move lower-level details to implementation tasks.

Use capacity estimates when scale changes the architecture. State assumptions such as request volume, payload size, processing duration, concurrency, retention, and expected growth. Show the calculation instead of dropping an unexplained number into the doc. The Letinix System Design Cheat Sheet can help organize estimation inputs and system building blocks; validate its reference figures against your own workload rather than treating them as service measurements.

4. Alternatives: include the simplest fix and doing nothing

Do not explain only the preferred design. Show what else could solve the problem and why it may be less suitable.

At minimum, compare:

Do nothing: what happens if the team accepts the current behavior? Is the cost tolerable, or does the risk continue?
The simplest fix: could a timeout adjustment, query optimization, pagination, caching, or a bounded change solve the actual problem?
The proposed approach: what does it improve, and what complexity does it introduce?
Other credible options: include alternatives that meet the requirements and are realistic for the team to implement or operate.

Option	Benefits	Costs and risks	When it may fit
Keep the synchronous flow	No new job infrastructure	Long requests can still fail; request capacity remains occupied	Impact is low or the issue is temporary
Optimize the current path	Smallest architectural change	Does not remove request-duration limits if work remains long-running	Profiling identifies a clear bottleneck
Use a background job flow	Separates request handling from long-running work	Adds job state, retries, worker operations, and status handling	Asynchronous completion is a requirement

The table is illustrative. A real proposal should compare options that fit the actual system and evidence. Avoid straw-man alternatives that exist only to make the preferred solution look inevitable.

5. Risks, rollout, and rollback: explain how change stays controlled

List specific failure modes and how the team will detect or contain them. Avoid generic labels such as “performance, security, and scalability.”

For a background export flow, ask whether duplicate job delivery can create duplicate exports, what happens when a worker crashes mid-file, whether users can access another customer's export, how stuck jobs are detected, how long completed files are retained, and what happens if the queue or storage is unavailable.

Describe a staged rollout. Identify the first cohort or traffic slice, the signals that determine whether rollout can continue, and who watches those signals. Define rollback in operational terms: which switch, deployment, or routing change returns the system to the previous behavior? What happens to jobs already in progress? A rollback plan that ignores partially processed work is incomplete.

For data migrations, include compatibility and recovery details. State whether old and new versions can run together, whether writes are reversible, and what data may need reconciliation. If rollback becomes unsafe after a particular step, identify that point explicitly.

Write for reviewers who skim

Reviewers will scan before they read closely. Make the scan useful.

Write the summary last. Once the problem, goals, design, alternatives, and risks are stable, write a short summary containing the problem, proposed option, most important trade-off, requested decision, and deadline. Do not use it to introduce claims the body does not support.

Use headings that communicate the point. “Why large exports time out” is more informative than “Background.” A diagram should show the current or proposed flow, boundaries, and important failure paths—not decorative boxes. Tables help compare alternatives, requirements, and rollout criteria.

Cut anything that does not help the reviewer understand the problem, evaluate the options, or make the decision. Move detailed API contracts, field-by-field schemas, exhaustive test cases, and implementation tickets into linked supporting material when they are not central to the architecture choice. Keep the proposal focused; link to details rather than duplicating them.

Run the review as a process, not a broadcast

A shared document link is not a review plan. Choose reviewers based on ownership and risk. Invite the decision owner, owners of affected systems and APIs, service operators, and relevant security, privacy, data, product, or support partners.

Ask reviewers to assess the areas they own: operations, authorization and data exposure, or user-facing behavior.

Use asynchronous comments for bounded proposals reviewers can assess independently. Set a deadline and ask focused questions. Meet when trade-offs remain unresolved, requirements conflict, or dependencies need joint discussion. The meeting should resolve issues, not replace the proposal.

End the review with a recorded outcome: accepted, rejected, deferred, or accepted with conditions. Capture the decision owner, date, rationale, unresolved conditions, and follow-up actions. Public engineering references show different ways to do this. Flutter's published RFC guidance describes asynchronous approval on the RFC pull request, with meetings used for discussion. GitLab's public architecture workflow recommends documenting context, benefits, trade-offs, alternatives, and the reason for the selected decision. Treat these as examples to adapt, not policies to assume at your company.

A short list of review questions

Add questions that expose the uncertainties you need reviewers to address:

Is the problem supported by evidence, and is the impact described accurately?
Are the goals measurable and the non-goals clear?
Does the proposed design satisfy the stated requirements?
Have we compared the simplest fix and the option of doing nothing?
Which assumption, dependency, or failure mode could change the recommendation?
Can the change be rolled out gradually and rolled back safely?
Who owns the decision, and what must be true before implementation begins?

Avoid asking reviewers to “share any thoughts.” A specific question gives them a place to start and helps distinguish a blocking concern from a preference.

After approval: preserve the decision and update the design

Approval is not the document's endpoint. Record the decision, rationale, and conditions. If implementation diverges, update the design or link to a follow-up record so it does not misdescribe the system.

Separate the proposal from the decision record when a choice deserves a durable, independently discoverable history. An Architecture Decision Record (ADR) can capture a specific architectural choice, its context, consequences, and alternatives. Google's public ADR overview describes recording options, requirements, the decision, and reasons, and retaining the history when a decision changes. GitLab's architecture workflow also describes using ADRs to record decisions and linking them from the main design document.

Do not silently overwrite the original reasoning when a decision changes. Record what changed, why the old decision no longer fits, and which new constraints or evidence drove the revision. Keep links between the proposal, decision record, implementation, and relevant operational documentation.

What to do next

A strong design doc does not need to be long. It needs to make the problem legible, the options comparable, the risks visible, and the requested decision unmistakable.

If you are starting from a blank page, use the free Letinix Technical Design Doc Template. It gives you a structured starting point for presenting a system change, showing supporting evidence, and asking reviewers for a clear decision. Adapt the sections to the scope of your proposal rather than filling every heading mechanically.

References

This article was originally published on Letinix.

How to Budget Retrieved Context in a RAG Pipeline

Muhammad Azlaan Zubair — Fri, 06 Nov 2026 19:00:00 +0000

A useful RAG token budget starts with the complete request, not just the retrieved passages. Reserve capacity for system instructions, the user query, conversation history, retrieval metadata and formatting, and the model’s output allowance. The remaining space is the maximum available for retrieved context—and it should be treated as a ceiling, not a target that every request must fill.

There is no universal chunk size or top-k value that works for every retrieval-augmented generation (RAG) workload. Choose chunking and retrieval settings based on your model’s limits, document structure, query patterns, relevance, answer quality, and latency or cost constraints. Then measure the assembled request with the tokenizer or request-counting API for the model you actually call.

Calculate the budget before retrieval

Start with the model’s documented context capacity and output constraints. The exact relationship varies by model and API: some limits share a total context budget, while others describe input and output separately. Confirm the model’s current documentation rather than assuming every provider exposes limits in the same way.

retrieval_budget = context_limit
                 - instructions_and_query
                 - conversation_history
                 - formatting_and_metadata
                 - reserved_output
                 - safety_margin

This is a planning model, not a substitute for the API’s actual validation rules. If your provider defines separate input and output limits, enforce both. If tools, images, or structured message boundaries add tokens, count them through the provider’s supported request-level method where possible.

Conceptual allocation only; actual limits depend on the model and API.

Measure the request you will actually send

A retrieved document chunk is not always the same string that reaches the model. The application may add a source title, URL, document ID, rank, access-control metadata, separators, instructions, or a citation wrapper. Those fields can be useful, but they also consume input capacity.

Count the final serialized context after the application has assembled these parts. Counting each chunk in isolation is useful for filtering candidates, but it cannot account for repeated wrappers, query text, chat history, or message structure.

Use the Letinix Token Calculator for a quick text-only inspection. It provides exact counts for the o200k_base and cl100k_base encodings and a character-based estimate for other selected models. For a production request, use the target provider’s request-counting API when it supports the inputs you send, and compare the count with actual usage reported by completed requests.

Treat chunk size and retrieval count as separate decisions

Chunk size determines how much source material is grouped into each retrieval unit. Retrieval count determines how many candidates are selected. They interact, but changing both at once makes it harder to identify why quality or cost changed.

Small chunks can make retrieval more specific but may separate a claim from the context needed to interpret it. Large chunks can preserve surrounding context but may include irrelevant passages and consume more of the prompt budget. Overlap may preserve information near boundaries, but it can also repeat text across retrieved chunks.

The right values depend on the corpus and the question. A policy manual, API reference, source-code repository, and conversational knowledge base have different structural boundaries and retrieval needs. Avoid treating a provider’s example configuration as a universal recommendation.

A controlled evaluation should vary one dimension at a time:

Establish a baseline with the current chunking and retrieval settings.
Change chunk size or overlap while holding the retrieval count and prompt constant.
Measure retrieval relevance and answer quality on a representative query set.
Change the retrieval count and evaluate again.
Compare token use, latency, failure modes, and answer quality—not token reduction alone.

Google’s Gemini File Search documentation, for example, exposes chunking configuration with maximum tokens per chunk and maximum overlap tokens. Those parameters are configuration options, not evidence that a particular setting is best for another corpus or model.

Select chunks under a token ceiling

A fixed top-k retrieval step can return a set whose combined serialized size exceeds the available budget. A budget-aware assembly step should count candidates and include them only while the request remains within its limit.

A simple greedy strategy can work as a first implementation:

rag-budget.ts

// Pseudocode: countWithTargetModel must use the target model's tokenizer.
const selected = [];
let used = countWithTargetModel(renderPrompt({ query, chunks: [] }));

for (const chunk of rankedChunks) {
  const candidate = renderPrompt({ query, chunks: [...selected, chunk] });
  const candidateTokens = countWithTargetModel(candidate);

  if (candidateTokens + reservedOutput + safetyMargin <= contextLimit) {
    selected.push(chunk);
    used = candidateTokens;
  }
}

This example illustrates a budget check; it is not a tested production implementation. In a real system, use the provider’s actual input/output constraint semantics, avoid repeatedly re-tokenizing the entire prompt when that becomes expensive, and preserve any required citation or permission metadata. A greedy fit check also does not guarantee that the selected chunks are the most useful combination: relevance, diversity, source coverage, recency, and redundancy may need to influence selection.

If chunks are reordered or truncated after retrieval, retain source boundaries and enough provenance for citations. Never trim text blindly in a way that leaves a number, exception, code block, or claim without the qualification needed to interpret it.

Evaluate the budget against answer quality

The goal is not to maximize the number of tokens sent. It is to include enough relevant evidence to answer the query while avoiding unnecessary or redundant context.

Retrieval quality: whether the needed source passages appear among candidates and selected chunks.
Answer quality and grounding: whether the answer uses the retrieved evidence correctly and preserves qualifications.
Context utilization: actual input tokens relative to the available input budget.
Operational behavior: latency, request failures, truncation, and cost under realistic traffic.

A smaller prompt is not necessarily a better RAG system if it drops decisive evidence. Likewise, a larger prompt is not automatically better if it adds duplicates or distractors. Token usage is one constraint in the evaluation, not the quality metric by itself.

For cost-focused planning after you have measured representative input and output usage, use the LLM API Cost Calculator. It estimates spend from request volume and token counts; it does not replace a model-specific count or an evaluation of retrieval quality.

A practical implementation checklist

Confirm the target model’s context and output limits from its current documentation.
Measure system instructions, query, history, metadata, and serialized retrieved passages—not only raw chunks.
Keep output allowance and a safety margin separate from the retrieval budget.
Tune chunk size, overlap, and retrieval count independently before testing combinations.
Enforce a budget after retrieval, using a target-model-compatible counter or request-counting API.
Evaluate relevance, grounding, quality, latency, and cost together on representative queries.
Re-run the evaluation after changing the model, prompt template, corpus, chunker, or metadata format.

References

This article was originally published on Letinix.

Why Token Counts Differ from Word Counts

Muhammad Azlaan Zubair — Wed, 04 Nov 2026 19:00:00 +0000

A word count cannot tell you exactly how many tokens an LLM will process. Token counts depend on the tokenizer or model family, the spelling and formatting of the text, and sometimes the structure of the API request around it. Rules such as “one token equals three-quarters of a word” are rough estimates for some English text—not a reliable conversion formula.

If you need a useful count, measure the text with the tokenizer used by your target model. For OpenAI requests, tiktoken can count text using supported encodings; for a complete API request, use the provider’s token-counting API when available. The Letinix Token Calculator provides exact counts for its supported OpenAI encodings and labeled estimates for other models.

A word and a token measure different things

A word count treats text as words according to a counting rule. A tokenizer divides text into units from a model’s vocabulary. Depending on the encoding, a token may represent a whole word, part of a word, punctuation, whitespace attached to text, or a short sequence of characters.

Consider this illustrative text:

Retry the request if status_code == 429.

A word counter might count the identifiers and punctuation according to its own splitting rules. A tokenizer instead applies its learned vocabulary and segmentation rules. It may represent status_code, 429, the equality operator, and surrounding spaces as several tokens or as fewer units. The exact result depends on the encoding; the example is not assigned a universal token count here.

That is why word count, character count, and token count should be treated as separate measurements.

Illustrative only: token boundaries vary by encoding and text.

Why the same text can produce different token counts

The tokenizer matters

Different model families use different token vocabularies and segmentation rules. Even within one provider, an encoding used by one model generation may differ from another.

OpenAI’s maintained tiktoken library exposes named encodings and can map supported model names to an encoding with encoding_for_model(). Its documentation includes o200k_base and cl100k_base, among others. A count produced with one encoding is exact for that encoding; it is not automatically exact for another provider’s tokenizer.

Anthropic and Google provide model-aware token-counting interfaces for their APIs. Use the provider’s own counting method when the result must match a request to that provider. A generic tokenizer from another ecosystem is not a substitute.

Spelling, punctuation, and formatting matter

Tokenization does not simply split at spaces. Capitalization, contractions, uncommon identifiers, repeated punctuation, numbers, and whitespace can change how a sequence is segmented. A familiar word may be represented compactly, while an unusual string may be divided into smaller pieces.

This matters for prompts containing paths, version numbers, hashes, log lines, stack traces, source code, or JSON. A character-based rule of thumb can be useful for rough planning, but it can miss how technical strings are actually segmented.

Language matters

The relationship between words, characters, and tokens is not constant across languages. Word boundaries differ, scripts encode information differently, and a tokenizer’s vocabulary may represent some languages more efficiently than others. Do not carry an English-text estimate into multilingual content as though it were a guarantee.

When is a token estimate good enough?

Use the method that matches the decision you need to make.

Task	Suitable method	Important limitation
Roughly size a short English draft	Character- or word-based estimate	Not an exact tokenizer count
Count plain text for a supported OpenAI encoding	Compatible tokenizer such as tiktoken	Exact for that encoding, not necessarily the full API request
Count a request for a specific provider	That provider’s token-counting API	Check which request components the endpoint accepts and counts
Verify actual production usage	Response usage fields	Available after the request; field names vary by API

Choose a counting method based on the task and required accuracy.

For a quick first pass, paste the text into the Letinix Token Calculator. The tool’s current documentation says it computes exact counts for o200k_base and cl100k_base; for other selected models it estimates tokens as characters divided by four, rounded up. It processes up to 500,000 characters in the browser. Its non-OpenAI estimates can be substantially inaccurate, especially for code and non-English text.

If your application depends on a hard limit, use a model-specific method instead of relying on a generic estimate.

Plain-text counts are not complete API request counts

A local tokenizer answers a narrow question: how does this text segment under this encoding? An API request can include additional structure, including message roles and boundaries, system instructions, tool definitions, schemas, images, files, or other multimodal content.

For example, counting only the visible user prompt does not account for a system instruction or a JSON schema passed as a tool definition. The provider’s request-level counting endpoint is the better fit when you need to know how much input a structured request will consume. OpenAI documents a Responses API input-token counting endpoint; Anthropic’s token-counting endpoint accepts structured message inputs and supported tools and media; Google documents count_tokens for Gemini inputs.

Even then, distinguish input tokens from output allowance and billed usage. A count of input tokens does not predict how many tokens the model will generate. Context limits and output limits are separate constraints, and some models also use internal reasoning tokens that affect usage. Check the documentation for the exact model and API endpoint you deploy.

A practical workflow for developers

Choose the target model first. Do not estimate against a convenient tokenizer and assume the result transfers to another model family.
Count the actual text you plan to send. Include the system prompt, retrieved passages, conversation history, and other text components you control.
Count the structured request when possible. Use the provider’s token-counting endpoint for message structure, tools, images, files, or other supported inputs.
Leave room for output and request overhead. A prompt that nearly fills a context window may leave too little space for generation or additional context.
Compare with actual usage. Log provider-reported usage in representative requests and revisit assumptions when models, prompts, tools, or content types change.

For the next step, use the Letinix Token Calculator to inspect a text sample and compare its count across supported encodings. If your primary question is financial rather than tokenization or context fit, continue with the LLM API Cost Calculator, which estimates spend from expected request volume and input/output token counts.

References

This article was originally published on Letinix.

XML Tags vs. Markdown in LLM Prompts: When to Use Each

Muhammad Azlaan Zubair — Tue, 03 Nov 2026 09:40:38 +0000

Use Markdown headings for a readable default and XML-style tags when explicit content boundaries or nested structure help. Combine them when each does a different job—for example, Markdown for top-level instructions and XML tags around a supplied document. There is no universal winner: OpenAI documents both as useful ways to mark logical boundaries, while Anthropic recommends descriptive XML tags for complex prompts. Test the format against your model and task rather than treating syntax as a performance guarantee.

What changes between Markdown and XML tags?

Markdown uses familiar headings, lists, and emphasis to organize instructions. XML-style tags wrap a section in an explicit opening and closing marker. Both can make prompt parts easier to distinguish, but they are not a security boundary and do not replace clear instructions.

Two ways to mark prompt structure; choose based on readability and boundary clarity.

Markdown example:

## Task
Summarize the supplied document in three bullets.

## Context
The audience is a software engineering manager.

## Input
[DOCUMENT TEXT]

XML-style example:

<task>
Summarize the supplied document in three bullets.
</task>

<context>
The audience is a software engineering manager.
</context>

<input>
[DOCUMENT TEXT]
</input>

These examples express the same intended organization. The choice is about how you want to express boundaries and maintain the prompt—not about choosing a magic syntax.

When Markdown is the better default

Choose Markdown when the prompt is primarily a set of instructions that people need to read, review, and maintain.

Human-readable sections: headings like ## Task and ## Constraints are quick to scan in a code review.
Lists of requirements: bullets make independent rules easy to inspect and edit.
Simple prompt templates: a short task with a few sections may not benefit from additional opening and closing tags.
Documentation and collaboration: Markdown is widely used in repositories, issue trackers, and internal docs.

For example, a classification prompt with a clear task, a few decision rules, and a short output contract may be easiest to maintain as Markdown. Keep the structure proportional to the prompt; do not add headings simply to make a small instruction look more formal.

When XML-style tags are useful

Use XML-style tags when the prompt contains multiple kinds of content that need explicit start and end boundaries, especially when some sections contain long or nested material.

<instructions>
Summarize the source. Use only facts supported by it.
</instructions>

<source_document>
[UNTRUSTED DOCUMENT CONTENT]
</source_document>

<output_requirements>
Return three bullet points and identify any missing information.
</output_requirements>

This makes it easier to see where the source begins and ends. Anthropic recommends descriptive, consistent tag names and nesting tags when the content has a natural hierarchy. OpenAI's guidance also describes XML tags as a way to delineate supporting documents and context data. The benefits are organizational; tags alone do not neutralize malicious instructions inside a document or guarantee that the model will follow the intended boundaries.

XML-style tags are particularly useful when:

The prompt includes a long source document or several documents.
You need to distinguish instructions from quoted or user-provided content.
A section has a natural hierarchy, such as a collection of documents and each document's metadata.
You need clear boundaries around examples or variable input.

Use descriptive tag names that explain the content, such as <instructions>, <context>, <source_document>, and <output_requirements>. Consistency matters more than inventing a complicated tag vocabulary.

When to combine Markdown and XML

You do not have to choose one format for every part of a prompt. A practical hybrid uses Markdown for the top-level outline and XML-style tags for data boundaries:

## Task
Summarize the supplied source in three bullets. Do not add facts that are absent from the source.

## Source
<source_document>
[PASTE SOURCE TEXT HERE]
</source_document>

## Output format
Return exactly three bullets. If the source does not support a requested detail, say so.

The Markdown headings make the overall task easy to scan. The XML tags mark the beginning and end of the source content. Use a hybrid only when it adds clarity; duplicated layers of headings and tags can make a prompt harder to maintain.

A decision guide

Prompt situation	Practical starting point	Why
Short task with a few rules	Markdown	Readable without extra syntax
Long source document	XML tags around the source, with Markdown instructions if useful	Makes the source boundary explicit
Multiple related documents	Nested XML-style tags plus concise top-level instructions	Represents the hierarchy of inputs
Prompt maintained by a team	Whichever format is easiest to review consistently	Reviewability is a maintenance requirement
Model-specific workflow	Follow the provider's current guidance, then evaluate	Prompt behavior can differ across models and versions
Strict machine-readable output	Use the provider's supported structured-output feature when available	Prompt formatting alone is not schema enforcement

Practical starting points, not benchmark results.

This is a starting guide, not a benchmark result. The table summarizes the cited providers' guidance and a practical maintainability recommendation; it does not claim that one syntax is more accurate across all models.

Formatting does not replace application safeguards

A tagged input section can make a source boundary visible, but untrusted text may still contain instruction-like content. Do not rely on Markdown headings or XML tags as the only protection against prompt injection. Limit tool permissions, separate trusted instructions from untrusted data in the application design, validate outputs, and test realistic adversarial cases.

Likewise, if your application needs JSON or another strict output shape, defining it in Markdown or XML is not the same as enforcing a schema. Use a supported structured-output feature where available and validate the result in your application. OpenAI distinguishes JSON mode from Structured Outputs in its Structured Outputs guide.

Test the choice with your actual task

Do not evaluate syntax in the abstract. Keep the task, input, model, and success criteria stable while you compare versions.

Write one baseline prompt using the format your team finds easiest to maintain.

Create an alternative that changes only the boundaries or section markup—not the underlying instructions.

Run both against representative normal, ambiguous, and edge-case inputs.

Score outputs against explicit criteria, such as required fields, factual support, correct refusal or fallback behavior, and downstream parseability.

Repeat on the model and deployment configuration you actually intend to use.

Keep the clearer or better-performing version, record the decision, and rerun the checks after meaningful changes.

If you already have prompt wording and only need to arrange it into Markdown headings, XML tags, or plain labels, use the Letinix Prompt Formatter. It organizes existing sections and can clean whitespace or quotation marks; it does not rewrite wording or prove that the resulting prompt is better.

Bottom line

Start with Markdown for a readable outline. Add XML-style tags where long or nested content needs explicit boundaries. Combine them when the two formats have distinct jobs, and keep the result as simple as the task allows. The only reliable way to choose for a production workflow is to evaluate representative inputs against criteria that matter to your application.

References

OpenAI, Prompt engineering. Guidance on message formatting with Markdown and XML tags.

Anthropic, Claude prompting best practices. Guidance on XML tags, descriptive names, and nested content.

OpenAI, Structured Outputs. Guidance on schema-constrained output and related limitations.

This article was originally published on Letinix.

The Hardest Part of Starting a New Developer Job Is Understanding the Codebase

Muhammad Azlaan Zubair — Sat, 31 Oct 2026 14:39:28 +0000

You clear the interviews. You accept the offer. You finish the paperwork, get your laptop, meet the team, and finally receive access to the codebase.

You open the repository expecting to get started.

Then you see it: hundreds of files, folders inside folders, components importing other components, services calling other services, and names that seem familiar but don't tell you much about what the code actually does.

You know the language. You know the framework. You have built applications before.

But suddenly, even a small task feels harder than it should.

This is one of the uncomfortable parts of switching jobs as a software developer: you are experienced enough to write the code, but you don't yet understand the system you're expected to change.

And that gap can make the first few weeks feel much longer than they need to.

Knowing the stack isn't the same as knowing the codebase

A new job often comes with the assumption that your technical experience will transfer immediately. In some ways, it does. Your knowledge of JavaScript, React, Node.js, databases, testing, or deployment still matters.

What doesn't transfer automatically is the context around those technologies.

Every codebase has its own history and conventions:

Where the team puts business logic.
How components communicate with services.
How API requests are structured.
Where validation and error handling happen.
Which abstractions are intentional and which are historical.
How data moves through the application.
What needs to be tested when a particular part changes.

Two teams can use the same stack and organize their applications very differently.

You might be comfortable building a React application from scratch, but that doesn't mean you'll immediately know where a particular team handles authentication, permissions, caching, or form state.

The challenge isn't that you've forgotten how to code. You're missing the context that makes the code understandable.

The temptation to understand everything

When developers feel lost in a repository, a common reaction is to start reading files.

You open the application entry point. Then a router. Then a layout. A component imports a hook, the hook calls a service, and the service imports an API client. You follow another import and end up somewhere completely different.

An hour later, you've read a lot of code but still can't confidently explain how one feature works.

This happens because a repository is not a book that needs to be read from beginning to end. Its file structure is only one clue about the system. Reading files without a question gives you details without necessarily giving you a useful mental model.

A better starting point is smaller: find one behavior you need to understand.

Start with one feature you can follow

Imagine you're asked to make a small change to the login experience.

Instead of exploring every authentication-related file at once, begin with the interaction itself.

What happens when someone submits the login form?

In a typical web application, the path might look something like this:

A user enters credentials and submits a form.
A component handles the submit event.
A function validates or prepares the input.
A client sends a request to an API endpoint.
A backend route passes the request to the relevant service.
The server returns a response.
The frontend updates the interface based on the result.

This is an illustrative flow, not a claim about every application's architecture. Your project may use different boundaries, naming conventions, or authentication mechanisms.

The point is to follow one complete path from the action a user takes to the result they see.

As you trace it, note the files that actually participate in the behavior. You may discover that a feature is spread across a page, a custom hook, an API client, a backend route, and a service. You now have a concrete starting map instead of a vague impression of the entire repository.

Find the places where execution begins

Once you have a behavior in mind, look for the places where the application receives work.

In a frontend application, that might mean locating:

The application entry file.
The router or route definitions.
The layout surrounding the relevant page.
The page or component that owns the interaction.

In a backend application, relevant starting points might include:

The server startup file.
HTTP route definitions.
Middleware.
Queue consumers or background jobs.
Scheduled tasks or event handlers.

These are examples, not a universal folder structure. A monolith, a monorepo, a serverless application, and a distributed system can have very different entry points.

The useful question is not “What is the most important file in this repository?” It is “Where does the behavior I'm investigating enter this part of the system?”

That question keeps exploration tied to the task.

Use the running application to check your assumptions

Reading code gives you a theory of what should happen. Running the application helps you see what actually happens.

If you can reproduce the behavior locally or in an approved development environment, use the tools available to inspect it:

Put a breakpoint where the interaction begins.
Step through the relevant function calls.
Inspect the values passed between layers.
Check the network request and response.
Add a temporary, targeted log when the existing observability is insufficient.
Run the relevant tests to understand expected behavior.

For example, if a form behaves unexpectedly, inspect the value at the event handler, then at the API boundary, rather than guessing which component is responsible.

Be careful with logs: they can expose tokens, personal data, or other sensitive values. Avoid logging secrets and remove temporary debugging statements before committing changes.

Tests are useful evidence too, but they are not automatically a complete description of the system. A test may cover only one case, or it may reflect behavior that has since changed. Read what it asserts and consider whether it covers the path you're investigating.

The goal is to replace assumptions with observations wherever you can.

Learn the team's patterns before introducing your own

At some point, you'll need to change something. Before creating a new abstraction or writing a new implementation, look for a similar feature that already works.

If you're adding an API endpoint, inspect another endpoint with comparable responsibilities.

If you're changing a React component, find one that handles a similar interaction.

If you're adding validation, look at how the project handles validation and errors elsewhere.

Pay attention to more than naming. Look at where logic lives, how dependencies are passed, how loading and error states are represented, how tests are organized, and what conventions the team follows.

Existing code is not automatically good code, and copying a pattern without understanding its context can carry forward a mistake. Treat it as evidence of how the project currently works—not as proof that every implementation decision is ideal.

When a pattern seems inconsistent, ask a teammate before assuming it is safe to standardize.

Keep a small map as you explore

You don't need a complete architecture document to become productive. A short set of notes is often enough to stop repeating the same investigations.

For the feature you're working on, record:

The route, page, or entry point.
The main components or handlers involved.
The service or API boundary.
The backend route or job, if relevant.
The tests that cover the behavior.
Any important dependencies or unresolved questions.

A simple flow written in plain language is useful:

Login form → submit handler → auth client → API route → authentication service → response → UI state

Adjust it to match what you actually found. Don't fill in unknown parts just to make the diagram look complete.

These notes become especially useful when you return to the feature a few days later, need to explain your change in a review, or discover that another task touches the same area.

You don't need to know the whole repository before contributing

There is a point where more exploration stops helping.

You may be ready to make a small change when you can explain the relevant behavior, identify the likely files involved, understand the project's existing pattern, and name the checks that should give you confidence in the change.

That does not mean you understand every dependency or every architectural decision. It means you understand enough of the part you're touching to make a deliberate change—and you know what you still need to verify.

Start with a small, reviewable contribution. Run the relevant checks. Ask focused questions when you reach a boundary you cannot confidently cross. Expand your map as the work takes you into other parts of the system.

The first few weeks in a new engineering role can feel slow because you're learning a product's history and a team's way of working at the same time as you're learning the code.

That's normal. You don't have to turn the entire repository into a mental model on day one.

You just need to understand the next piece of the system well enough to work on it responsibly.

Choosing Local AI Hardware by Workload, Not Hype

Muhammad Azlaan Zubair — Wed, 28 Oct 2026 17:59:16 +0000

Local AI buying advice often starts with a GPU shortlist. A better starting point is the workload: what do you want to run, how much memory does it need, and how much latency or setup work can you tolerate?

The value-first answer is often less exciting than a flagship build: use the computer you already own, test a compact quantized model, and upgrade only after you can name the bottleneck.

Start with memory

For local language models, GPU VRAM is often the constraint that decides which models fit comfortably. Model weights are only part of the footprint: the runtime, context cache, image inputs and other applications also use memory. Quantization can reduce weight memory, but it does not remove all overhead and can affect output quality.

That is why shopping by parameter count or peak compute alone can mislead. Check the exact model variant, quantization, context length, runtime and operating-system support before spending. Hardware-fit tools provide estimates, not guarantees; compare them with model cards and your own workload.

Four value-first starting points

General use: chat, summaries and writing

Start with your current computer and try a compact quantized model such as Qwen3 8B. If you have enough memory and need image understanding, evaluate Gemma 4 12B. A smaller model can be easier to fit and iterate on, but may be less reliable on complex reasoning or long documents. Local inference also does not automatically provide current web knowledge or hosted-service convenience.

Engineering and development

A desktop GPU with 16 GB VRAM is a useful middle ground for coding assistants, repository Q&A, test generation and experimentation. Consider 24 GB if larger quantized models or longer contexts matter. Compare a current Qwen coding/instruct variant with Gemma 4 12B, and test patch quality, repository-level reasoning and tool calls against your own codebase. Keep a hosted model available for tasks where local quality falls short.

Design and video

Choose the actual image or video pipeline before buying hardware. 16–24 GB VRAM is a starting range to investigate for image workflows; heavier video-generation work can need more capacity. A multimodal LLM such as Gemma 4 can help understand images when the chosen variant supports image input, but it does not replace a dedicated image- or video-generation model. Check the requirements of the exact app, model and runtime.

Automation and assistants

Separate the always-on controller from inference. An existing mini PC or low-power desktop can host schedules, integrations, a workflow engine, logs, a database and persistent state. Evaluate Qwen3 8B/14B or Gemma 4 12B for bounded tasks, but test structured output, tool selection and recovery—not just fluent chat. Permissions, retries, observability and human approval are part of the system design.

A practical pre-purchase checklist

Define the real workload: chat, coding, image generation, video generation or agents.
Check VRAM or unified memory for the exact quantization and context.
Include system RAM, SSD capacity, power, cooling, noise and electricity in total cost.
Verify the model license, runtime support and integrations.
Compare local and hosted options using your own usage pattern.

This is a decision guide, not a lab benchmark. Prices and model releases change quickly; treat third-party price examples as snapshots and verify current local availability. The best-value setup is the least expensive reliable system that meets your quality, latency, privacy and workflow needs—with enough headroom to avoid replacing it immediately.

Useful starting points include LLM Configurator, Hugging Face Models, Artificial Analysis open models and the Ollama model library. Check exact model cards and licenses before adoption.

Originally published on Letinix.

Cron Syntax Explained: Fields, Operators, and Common Schedules

Muhammad Azlaan Zubair — Tue, 27 Oct 2026 09:07:31 +0000

A standard five-field cron expression describes when a scheduler should trigger a task: minute, hour, day of month, month, and day of week. For example, */5 * * * * matches every five-minute mark, while 0 9 * * 1-5 matches 09:00 Monday through Friday under the traditional Unix cron interpretation.

Those expressions are not universally portable. Cron implementations differ in supported fields, weekday numbering, timezone configuration, and how they interpret edge cases. Identify the scheduler first, then validate the expression against that scheduler's documentation.

The five fields in a cron expression

In traditional Unix crontab, the five time fields appear in this order:

The five time fields in a traditional Unix cron expression

Field	Common range	Example meaning
Minute	0–59	15 means at minute 15
Hour	0–23	9 means 9 AM
Day of month	1–31	1 means the first calendar day
Month	1–12	6 means June
Day of week	0–7 in Linux cron	1 means Monday; 0 or 7 means Sunday

The command or workflow configuration comes after the time fields in a crontab entry. A cron expression copied into a framework configuration may be parsed differently, so do not assume every field range or extension is identical. The Linux crontab(5) manual documents the traditional five-field layout and Linux cron's day-of-week convention.

The main cron operators

Common five-field cron formats use these operators:

* — every allowed value in that field.
, — a list of specific values, such as 1,15.
- — an inclusive range, such as 1-5.
/ — a step within a field, such as */5 for every fifth minute value.

Examples below use the traditional five-field Unix cron format. Their descriptions are expected interpretations of the expressions, not observations from a running scheduler.

Expression	Expected interpretation
* * * * *	Every minute
/5 * * *	Every five minutes, at minute values 0, 5, 10, and so on
0 * * * *	At the start of every hour
0 9 * * 1-5	At 09:00 Monday through Friday
0 0 1 * *	At midnight on the first day of each month
30 8 * * 0	At 08:30 every Sunday in Linux cron, where 0 means Sunday

A step is evaluated within its field's range. For example, */5 in the minute field means the minute values divisible by five. It does not mean “wait five minutes after this job finishes,” and it does not create a durable interval measured from the time the job was installed.

How to read an expression from left to right

Take this expression:

15 6 * * 1-5

Read each field in order:

15 — minute 15.
6 — hour 6, using a 24-hour clock.
* — every day of the month.
* — every month.
1-5 — Monday through Friday in the traditional Linux cron convention.

The expected interpretation is 06:15 on weekdays, in the timezone used by the scheduler. The expression itself does not contain a timezone.

The day-of-month and day-of-week rule

One of the easiest mistakes is restricting both the day-of-month and day-of-week fields and assuming they form an AND condition.

In the traditional Linux cron behavior documented by crontab(5), when both fields are restricted (neither is *), a job runs when either field matches. For example:

0 9 1 * 1

This is expected to match 09:00 on the first day of each month and 09:00 on Mondays, not only Mondays that fall on the first. This rule is implementation-specific; check the target scheduler before transferring the expression elsewhere.

If you intend “the first day of the month, only when it is Monday,” traditional cron's five fields do not express that AND condition directly. Use a scheduler with suitable calendar rules or make the additional condition explicit in the job itself.

Weekday numbering is not universal

Do not copy weekday numbers between schedulers without checking the target's convention. The Linux crontab(5) manual accepts 0 or 7 for Sunday and 1–6 for Monday through Saturday. GitHub Actions documents its own five-field schedule syntax with weekday values 0–6, where Sunday is 0. Workflow syntax for GitHub Actions

Names such as MON or JAN, a seconds field, and special tokens such as ? or L are not part of the portable five-field baseline. Some schedulers support extensions; others reject them or interpret them differently. Use the scheduler's primary documentation as the authority for its accepted syntax.

Time zones and daylight-saving time

A cron expression specifies calendar fields, not a timezone. The effective timezone comes from the scheduler or its configuration. A schedule intended for 09:00 local time may therefore run at a different local hour if the scheduler evaluates it in UTC or another zone.

Daylight-saving transitions are also scheduler-specific. The Linux crontab(5) manual notes that nonexistent local times during a spring-forward transition do not match, while repeated times during a fall-back transition can run twice. Other platforms may use different rules. If a task must run once per business day, avoid assuming that the same expression guarantees identical behavior across every DST transition.

For a production schedule, confirm all three items:

Which timezone the scheduler actually uses.
How that scheduler handles skipped or repeated local times.
Whether a delayed, skipped, or duplicate trigger is acceptable for the task.

A scheduled trigger is not a guarantee that the work will start exactly on time or finish successfully. The scheduler, host, queue, and job implementation all affect execution and recovery.

Build and check an expression

Start with the intended schedule in plain language—for example, “every weekday at 09:00 in the scheduler's configured timezone.” Translate that requirement into the target scheduler's dialect, then check the expression and its expected calendar matches before deploying it.

The Letinix Cron Expression Builder is the relevant utility when you want to construct or inspect an expression directly. Check its current format and preview assumptions against the scheduler you are configuring; a builder does not replace the scheduler's own validation or documentation.

A short validation checklist

Before saving a cron schedule, verify:

The target expects five fields rather than a six-field format with seconds.
Each field uses the target's accepted range and operators.
Weekday numbering matches the target scheduler.
Restricted day-of-month and day-of-week fields have the intended combined behavior.
The scheduler's timezone and daylight-saving behavior match the requirement.
The job is safe if it starts late, is skipped, runs twice, overlaps a previous run, or fails after being triggered.

For Linux-specific behavior, use the Linux crontab(5) manual. For GitHub Actions, use the official workflow syntax reference and events that trigger workflows. Other platforms should be checked against their own scheduler documentation.

References

crontab(5) — Linux manual page

Workflow syntax for GitHub Actions — GitHub Docs

Events that trigger workflows — GitHub Docs

This article was originally published on Letinix.

How to Estimate the Cost of an AI Agent Workflow

Muhammad Azlaan Zubair — Tue, 27 Oct 2026 09:07:10 +0000

To estimate AI agent API costs, calculate the cost of each model call in the workflow, add separately billed tools and infrastructure, then account for retries and multiply by expected task volume. A single user request can trigger several model calls, and later calls may resend a larger context than the first. Estimating only the initial prompt will usually miss part of the bill.

A practical estimate starts with a workflow map and a small set of scenarios. For every model call, record input tokens, output tokens, model and pricing tier. Add tool charges where the provider bills them separately. Then calculate a normal path, a high-usage path and the cost of failures or retries.

The Letinix LLM API Cost Calculator can calculate token-cost scenarios from prices and token counts you enter. It does not model an entire agent graph automatically, so use it for call classes or totals after you have mapped the workflow.

1. Map one task from start to finish

Start with a specific task, such as answering a support question using a knowledge base, summarizing a set of documents, or researching a topic and producing a report. Draw the sequence of steps that may require model inference or a paid tool.

Interpret the task: the model receives the instructions and user request.
Choose an action: the model decides whether to call a tool or continue reasoning.
Run the tool: your application executes a search, database lookup, code operation or external API call.
Read the result: the tool output is sent back to the model, which may choose another action.
Produce the answer: the model generates a final response or structured result.

This is an illustrative pattern, not a required architecture. A simple agent may use one model call and one tool; a longer-running agent may loop through multiple tools and model calls. Measure the workflow your application actually uses rather than assuming every task follows the same path.

For each model call, record the model and pricing tier; input tokens including instructions, conversation state, tool definitions, retrieved documents and previous tool results that are sent again; output tokens including reasoning tokens when billed; cache assumptions; and any separate tool or service fee.

2. Calculate each model call separately

For a simple text call without caching, use the provider’s current input and output rates per million tokens:

call input cost  = input tokens  / 1,000,000 × input rate
call output cost = output tokens / 1,000,000 × output rate
call cost        = call input cost + call output cost

Repeat this for every call in the workflow, then add the results. Do not count one user action as one model request unless the implementation really makes only one call.

For example, an agent may first receive a short request, then send the model a tool schema, then send a large search result, then make a final call with accumulated context. Those calls can have different token counts. If a call resends prior conversation or tool output, those tokens can be billed again as input unless the provider’s caching rules apply.

The official documentation explains provider-specific billing details: OpenAI API pricing Anthropic Claude pricing Gemini API pricing. Anthropic documents tool definitions and results as part of tool-use tokens, Google documents intermediate input and reasoning tokens in agentic loops, and OpenAI lists separate fees for some tools. Check the current page for the selected model and feature.

3. Build a per-task estimate

Consider this hypothetical workflow with four model calls. These are illustrative inputs, not measurements from a deployed agent. Assume an input rate of $2 per million tokens and an output rate of $8 per million tokens solely to show the arithmetic.

Model call	Input tokens	Output tokens	Estimated token cost
Interpret request	1,200	300	$0.0048
Choose tool/action	2,500	400	$0.0082
Process tool result	4,200	600	$0.0132
Generate final answer	5,000	900	$0.0172
Total	12,900	2,200	$0.0434

The example uses the same hypothetical rates for each call. The cost for each row is calculated from that row’s input and output tokens; the total is the sum of the four rows. The growing input count illustrates one source of cost: later calls may include tool output and more task context.

Now assume the workflow also incurs a hypothetical $0.01 in separate tool charges per completed task. The estimated total becomes $0.0534 per task before retries, infrastructure, logging, storage or other omitted costs. The $0.01 is an illustrative assumption, not a quoted provider fee.

For a real workflow, replace these values with API-reported token usage and current prices. If different calls use different models, use the corresponding rates for each call rather than applying one blended rate to the entire workflow.

Conceptual estimation model; actual workflow steps and charges vary by implementation.

4. Add tool fees and other billable operations

A tool call is not always billed as a single flat fee. Depending on the provider and tool, the cost may include model tokens associated with tool use; a separate fee for a server-side search or hosted execution feature; an external service’s fee; and application infrastructure such as compute, queueing, database reads, storage, observability and network services.

Avoid double-counting. If a provider’s usage report already includes tool-related tokens in model input/output, do not add those tokens a second time. Add a separate fee only when the relevant pricing or invoice identifies one. The provider docs govern provider charges; other vendors’ billing terms govern external services.

5. Model retries and incomplete tasks

Retries matter because a failed or incomplete attempt may already have incurred model or tool costs. Examples include a tool timeout after the model has generated a call, an invalid tool result that triggers another model request, or a final answer that fails validation and must be regenerated. Whether a particular failure is billable depends on the API and when the failure occurs; do not assume all failed requests are free or all are charged identically.

For a first estimate, use observed retry rates if you have traces. If you do not, state an explicit assumption and keep it separate from the base workflow cost.

expected cost per task = base cost × (1 + retry rate)
                       = $0.0534 × 1.08
                       = $0.057672

This simplified scenario assumes one additional full-cost attempt for 8% of tasks, with no further retries. Actual retries may cost less or more than the original path. If failures trigger only one small model call, estimate that retry path separately instead of multiplying the whole workflow.

6. Scale the estimate to monthly volume

Once you have a per-task estimate, multiply it by expected completed tasks per month. Keep task volume distinct from model-call volume: one task may contain multiple calls, and some tasks may fail or be retried.

Using the hypothetical $0.057672 expected cost above:

1,000 tasks per month: $57.67
10,000 tasks per month: $576.72
100,000 tasks per month: $5,767.20

These figures include the illustrative tool fee and simplified retry assumption, but exclude infrastructure and any other charges not listed. They are not a forecast for a particular provider or product.

Create at least three scenarios: low usage with shorter contexts and fewer tool calls; expected usage based on measured token counts and observed retry rates; and a high but plausible case with long retrieved results, additional loops or longer outputs. Recalculate when prompts, tools, retrieval limits, model routing or user behavior change.

7. A small estimator for call-level token usage

If you have a trace or spreadsheet of model calls, this Python example shows how to total token costs and add explicit per-task tool fees. It is a starting point for adapting your own usage export, not a tested integration with a provider API.

estimate_agent_cost.py

def estimate_task_cost(calls, input_rate, output_rate, tool_fees=0.0):
    """Estimate one workflow run from per-call token counts and explicit fees."""
    input_tokens = sum(call["input_tokens"] for call in calls)
    output_tokens = sum(call["output_tokens"] for call in calls)
    input_cost = input_tokens * input_rate / 1_000_000
    output_cost = output_tokens * output_rate / 1_000_000
    return {
        "input_tokens": input_tokens,
        "output_tokens": output_tokens,
        "token_cost": input_cost + output_cost,
        "tool_fees": tool_fees,
        "estimated_total": input_cost + output_cost + tool_fees,
    }

# Hypothetical example; replace with token usage from your own traces.
calls = [
    {"input_tokens": 1200, "output_tokens": 300},
    {"input_tokens": 2500, "output_tokens": 400},
    {"input_tokens": 4200, "output_tokens": 600},
    {"input_tokens": 5000, "output_tokens": 900},
]
print(estimate_task_cost(calls, input_rate=2.0, output_rate=8.0,
                         tool_fees=0.01))

Expected output for these hypothetical inputs is an estimated total of $0.0534, with 12,900 input tokens and 2,200 output tokens. The code does not fetch prices, infer retries, apply caching, or identify external infrastructure charges. Feed it actual usage for each model call, use the rate for that call’s model, and extend the model if calls use different rates or cached-input categories.

8. Use the calculator for the parts it supports

The Letinix LLM API Cost Calculator accepts input and output prices per million tokens, token counts per request, request volume and optional cached-input assumptions. It returns per-request, daily, monthly and yearly estimates.

For an agent workflow, enter representative token totals and a blended rate only if all included calls share that rate and billing category. Otherwise, calculate separate call classes when steps use different models, rates, cache behavior or tool fees, then add the results in a separate worksheet.

The calculator does not automatically model branching workflows, per-step model routing, retries, separate tool charges, every modality or application infrastructure. Its output is a scenario estimate, not an invoice. For more detail, see LLM input vs. output token pricing how to estimate LLM API costs before launch.

References

Pricing — OpenAI API

Web search — OpenAI API

Pricing — Anthropic Claude Platform

Gemini API pricing — Google AI for Developers

This article was originally published on Letinix.

How to Convert a JSON API Response to TypeScript Types

Muhammad Azlaan Zubair — Tue, 27 Oct 2026 09:06:43 +0000

A JSON API response can be converted into TypeScript declarations by mapping each JSON value to a TypeScript type, then extracting nested objects and array items into reusable declarations. A converter is a useful starting point, but the result describes the sample you supplied—not every response the API may return.

For a quick conversion, paste a representative payload into the Letinix JSON to TypeScript Converter. Then review optional fields, null values, and any documented API variants before relying on the generated declarations.

1. Start with a representative response

Use a response copied from API documentation or a non-sensitive response captured from your own development environment. The following payload is illustrative:

response.json

{
  "id": 42,
  "name": "Mina Chen",
  "active": true,
  "profile": {
    "website": null
  },
  "roles": ["editor", "reviewer"],
  "projects": [
    { "id": "p-1", "name": "Docs", "archived": false },
    { "id": "p-2", "name": "API", "archived": true }
  ]
}

The sample includes primitives, a nested object, arrays, and a nullable value. Those are the structures a generated declaration should represent. It does not show whether website can be a string, whether projects can be omitted, or whether the API sometimes returns other fields.

2. Convert the JSON into TypeScript declarations

The converter's documentation describes output for nested objects, arrays, and mixed-type unions. It also exposes options for interface versus type, adding or removing export, and marking properties readonly. The following TypeScript is an illustrative mapping of the JSON above, not a captured output from a live conversion:

api-types.ts

export interface ApiResponse {
  id: number;
  name: string;
  active: boolean;
  profile: Profile;
  roles: string[];
  projects: Project[];
}

export interface Profile {
  website: null;
}

export interface Project {
  id: string;
  name: string;
  archived: boolean;
}

Each object has a declaration, while the parent refers to nested declarations by name. The roles field becomes string[]; projects becomes an array of Project objects. JSON numbers map to TypeScript number, booleans to boolean, and strings to string.

For a small object shape, either an interface or a type alias can describe the object. Interfaces are useful for object contracts that may be extended or declaration-merged. Type aliases are more flexible when the type itself must express a union, tuple, or other composite type. Choose based on the model you need, not on a belief that one syntax is universally safer.

3. Review optional and nullable fields

A generated declaration can only infer what the sample reveals. In the example, website is always present and always null, so the declaration website: null is a literal description of that sample. It does not establish the API's full contract.

If the API documentation says that website is always present but can contain either a URL or null, model that documented contract as:

profile.ts

interface Profile {
  website: string | null;
}

If the property may be omitted, represent that separate condition with an optional property:

profile.ts

interface Profile {
  website?: string | null;
}

With strictNullChecks, reading an optional property also means accounting for undefined. Do not mark every property optional merely to make an interface permissive. Use API documentation, a schema, or multiple representative responses to decide what can actually be absent.

Objects in the same array can reveal shape differences. For example, if one project object includes archivedAt and another does not, a converter can infer that the key is optional across those observed array items. That is still evidence about the samples, not a guarantee about all future responses.

4. Handle mixed arrays and unknown shapes

JSON arrays can contain values of different types. For a sample such as:

mixed-values.json

{
  "values": [1, "pending", null]
}

A reasonable inferred element type is:

response.ts

type Response = {
  values: (number | string | null)[];
};

The union expresses the observed element types. It does not describe business rules such as which value combinations are valid, whether a string must match a particular set of statuses, or whether an API response can contain a different object variant.

An empty array is even less informative: there is no element from which to infer an item type. Treat an inferred unknown[] as a signal to consult the API contract or provide a more representative sample, rather than guessing the intended item shape.

5. Check the declaration against the API contract

Compare required fields with the API's documented response schema.
Check nullable properties separately from properties that may be missing.
Inspect arrays with multiple items and any documented response variants.
Review dates, identifiers, enum-like strings, and numeric fields. JSON alone does not establish that a string is a valid date or that a number is an integer.
Decide how to handle additional fields and API version changes.
If you can, generate types from a maintained API schema rather than relying only on one captured response.

For a stable, documented API contract, a schema-based workflow may be a better source of truth. For example, openapi-typescript generates TypeScript types from OpenAPI 3.0 and 3.1 schemas. That workflow starts from an API description rather than inferring a contract from a single payload.

6. Remember what TypeScript types do not do

TypeScript declarations help the compiler check how your code uses data. They do not inspect an HTTP response at runtime. A declaration such as const user = payload as User does not validate the payload or transform it into a User.

If the data is untrusted or can change independently of your application, validate it at the boundary. Libraries such as Zod let you define a schema and parse an input; Ajv can compile validators for JSON Schema or JSON Type Definition. The right choice depends on whether you already have a schema, how you want to express constraints, and what error handling your application needs.

Use the JSON to TypeScript Converter when you have a JSON sample and need a fast starting declaration. Treat the output as generated source code to review—not as proof that the API contract is complete or that the data is valid.

Common mistakes

Treating one response as the entire contract. Collect multiple samples or use the authoritative schema.
Confusing null with a missing property. T | null and property?: T model different conditions.
Assuming a type assertion validates data. It only changes what the compiler assumes.
Inferring semantics from primitive types. A string does not prove a value is a valid date, URL, or known status.
Ignoring contract drift. Regenerate or update types when the API contract changes, and validate incoming data where runtime correctness matters.

References

JSON to TypeScript Converter — Letinix

TypeScript Handbook: Object Types

TypeScript Handbook: Everyday Types

OpenAPI TypeScript: CLI

Zod: Basic usage

Ajv: Using with TypeScript

JSON Schema: Specification

This article was originally published on Letinix.