DEV Community: Anoop Kumar

How to Reduce Your AI API Costs by 40% Without Changing Models

Anoop Kumar — Sun, 27 Sep 2026 11:29:29 +0000

Most cost reduction advice for AI APIs focuses on switching to cheaper models. That works, but it trades quality for savings. These techniques reduce costs without touching model selection — by eliminating token waste that was never producing value in the first place.

After tracking 30 days of AI usage with TokenPulse, I identified patterns that consistently waste 30-50% of tokens. Here is how to eliminate them.

Technique 1: Targeted context pasting

The biggest single source of token waste is pasting entire files when you only need part of them.

The expensive pattern:

User: Here's my entire auth module (500 lines).
      Why is the token validation failing?

// 500 lines = ~30,000 tokens
// Re-sent on every subsequent message in the conversation

The efficient pattern:

User: Here's the validateToken function (20 lines)
      and the middleware calling it (10 lines).
      Why is validation failing?

// 30 lines = ~1,200 tokens
// 96% reduction for equivalent answer quality

For a 10-message debugging session, the efficient pattern saves approximately 260,000 tokens in context overhead — roughly $0.78 on Claude Sonnet or $0.65 on GPT-4o.

The rule: paste the specific function plus 10-15 lines of surrounding context. If the model needs more, it will ask.

Technique 2: Conversation restarts with summaries

Every message in a conversation re-sends the entire history. A 20-message conversation where each exchange averages 500 tokens accumulates 10,000 tokens of history that is re-sent on every new message.

Message 20 input cost:

History tokens: 20 exchanges × 500 tokens = 10,000 tokens
New message:    200 tokens
Total input:    10,200 tokens per message

Compare to starting a new conversation with a 200-word summary:

Summary:     ~150 tokens
New message: 200 tokens
Total input: 350 tokens per message

That is 96% less input tokens for message 20.

The summarize-and-restart pattern:

After 8-10 exchanges:

You: Summarize the key decisions, constraints, and 
current state of this implementation in under 150 words.

[Start new conversation]

You: Context: [paste summary]

     Continue from here: [your next question]

Implement this consistently and your per-conversation cost drops dramatically on long sessions.

Technique 3: System prompt caching (Claude only)

Anthropic's API supports prompt caching for system prompts longer than 1,024 tokens. Cached tokens cost 10% of the normal input price on subsequent requests.

If you are using the Claude API and have a long system prompt — project context, coding standards, architecture documentation — cache it:

const response = await anthropic.messages.create({
  model: 'claude-sonnet-4-5',
  max_tokens: 1024,
  system: [
    {
      type: 'text',
      text: longSystemPrompt, // Your project context
      cache_control: { type: 'ephemeral' }
    }
  ],
  messages: [{ role: 'user', content: userMessage }]
})

For a 5,000-token system prompt sent 100 times:

Without caching:

5,000 tokens × 100 requests × $3.00/1M = $1.50

With caching (first request full price, subsequent at 10%):

5,000 tokens × $3.00/1M           = $0.015 (first)
5,000 tokens × 99 × $0.30/1M      = $0.149 (cached)
Total: $0.164 — 89% reduction

Technique 4: Match message specificity to model

Vague questions produce long, hedged answers. Specific questions produce concise, targeted answers. Concise answers cost less.

Expensive prompt:

Explain authentication in web applications.

Produces: 800-1,500 token response covering every aspect of auth

Efficient prompt:

In my Express app using JWT, should I store 
the refresh token in httpOnly cookie or localStorage?
One paragraph.

Produces: 150-250 token response that answers exactly what you need

Adding "one paragraph", "in two sentences", or "bullet points only" to prompts consistently reduces output token consumption by 40-70% without losing useful information.

Technique 5: Batch related questions

Every API call has a minimum cost — the overhead of the request, the model's context loading, and the response structure. Making five separate one-question API calls costs more than one five-question call.

Five separate calls:

Call 1: "What does this function do?"         → $0.001
Call 2: "What are edge cases?"                → $0.001
Call 3: "How would you test this?"            → $0.001
Call 4: "Any performance concerns?"           → $0.001
Call 5: "Suggest a better variable name?"     → $0.001
Total: $0.005

One batched call:

"For this function: (1) what does it do, 
(2) edge cases, (3) how to test, 
(4) performance concerns, (5) better variable name?"
→ $0.002

Batching reduces overhead and gives the model context across all questions simultaneously — often producing better answers too.

Technique 6: Use cheaper models for context preparation

A common pattern: use a cheap model to prepare and filter context, then send only relevant context to the expensive model.

// Step 1: Use cheap model to identify relevant sections
const relevantSections = await anthropic.messages.create({
  model: 'claude-haiku-4-5', // $0.80/1M input
  max_tokens: 500,
  messages: [{
    role: 'user',
    content: `Given this 500-line file, identify only the 
    functions relevant to JWT authentication. 
    Return just the function names.

    File: ${entireFile}`
  }]
})

// Step 2: Extract only relevant code
const relevantCode = extractFunctions(
  entireFile,
  relevantSections.content[0].text
)

// Step 3: Send targeted context to expensive model
const analysis = await anthropic.messages.create({
  model: 'claude-sonnet-4-5', // $3.00/1M input
  max_tokens: 2048,
  messages: [{
    role: 'user',
    content: `Analyze this JWT authentication code: ${relevantCode}`
  }]
})

The Haiku call costs approximately $0.0003 to process the full file and identify relevant sections. The Sonnet call processes 90% less code. Net savings on large files: 60-80%.

Technique 7: Stop re-explaining context the model already has

A common pattern in long conversations is re-explaining context that the model already has:

Message 15: "As I mentioned earlier, this is a 
Next.js app using TypeScript and Prisma..."

The model already has this information — you told it in message 1. Re-stating it wastes tokens on every message where you do it. Trust the model's context and only re-state when you have genuinely lost track of what is in the conversation history.

Measuring the impact

Without tracking, these optimizations are invisible. After implementing all seven techniques, my monthly estimated AI cost dropped from $0.439 for 157 conversations to a projected $0.26-0.28 for the same volume — a 37-41% reduction.

The biggest single impact was targeted context pasting (technique 1), which alone accounted for approximately 25% of total savings.

TokenPulse makes it easy to see the cost impact of these changes in real time — cost per conversation updates as you chat, so you can immediately see whether a new approach is saving tokens.

Free, no API key, works on Claude, ChatGPT, Gemini, DeepSeek and Grok.

Claude vs ChatGPT — Which Model Actually Costs More Per Day?

Anoop Kumar — Tue, 22 Sep 2026 18:23:41 +0000

I tracked every AI conversation I had for 30 days using TokenPulse. 157 conversations, 8 active days, $0.439 estimated total cost. Here is what the data actually shows about the real cost difference between models.

The headline numbers

Most cost comparisons quote API pricing per million tokens. That number is meaningless in isolation. What matters is cost per actual conversation — how much does it cost to get a useful answer to a typical developer question?

From 30 days of tracking:

Platform	Conversations	Est. Cost	Cost per Conversation
ChatGPT (GPT-4o)	89	$0.198	$0.0022
Claude (Sonnet 4)	51	$0.187	$0.0037
Gemini (2.0 Flash)	12	$0.031	$0.0026
DeepSeek (V3)	5	$0.023	$0.0046

ChatGPT came out cheapest per conversation — not because GPT-4o is cheaper per token than Claude Sonnet (it is slightly more), but because my ChatGPT conversations were shorter. I used ChatGPT for quick lookups and formatting tasks. I used Claude for longer architectural discussions that burned more context.

Why per-conversation cost misleads

The cost-per-conversation number hides the most important variable: output quality per dollar.

A ChatGPT conversation that costs $0.002 and produces a mediocre answer that requires two follow-up questions actually costs more than a Claude conversation that costs $0.004 and answers the question completely.

The honest cost comparison has to account for:

How many follow-up questions were needed
Whether the first answer was actually used
Time spent reformulating prompts I cannot measure this precisely, but subjectively: I reach for Claude when the answer really matters and I cannot afford to iterate. I reach for ChatGPT when I need something fast and good enough. This explains the usage split in my data.

Where the real cost difference shows up

Long debugging sessions: Claude wins on cost. A 10-message debugging session on Claude Sonnet 4 costs approximately $0.008-0.012. The same session on GPT-4o costs $0.007-0.010. The difference is small, but Claude's larger context window (200k vs 128k tokens) means fewer session restarts — which are themselves a cost in time and context reconstruction.

Quick questions: ChatGPT wins on cost. For 1-3 message exchanges, GPT-4o mini at $0.15/1M input tokens is dramatically cheaper than Claude Sonnet at $3.00/1M. For the kind of quick question where any good model will do, GPT-4o mini is the obvious choice.

Document analysis: Gemini wins on cost. Gemini 2.0 Flash at $0.10/1M input tokens is 30x cheaper than Claude Sonnet and 25x cheaper than GPT-4o for input-heavy tasks. If you are summarizing long documents or analyzing large codebases, Gemini's cost advantage is significant.

Reasoning tasks: DeepSeek R1 is worth considering. At $0.55/1M input and $2.19/1M output for the reasoning model, it is cheaper than Claude Opus ($15/$75) and competitive with GPT-4o for tasks that require step-by-step reasoning.

The model selection heuristic that reduced my costs

After analyzing a month of data, I settled on this decision tree:

Is this a quick question (1-2 exchanges)?
  → GPT-4o mini ($0.15/1M input)

Is this a long document or large codebase?
  → Gemini 2.0 Flash ($0.10/1M input)

Does this require deep reasoning or math?
  → DeepSeek R1 ($0.55/1M input)

Is this an important architectural decision?
  → Claude Sonnet 4 ($3.00/1M input)

Does nothing else work?
  → Claude Opus 4 ($15.00/1M input)

This heuristic cut my estimated monthly spend by approximately 35% without any reduction in the quality of outputs that actually mattered.

The hidden cost: context window waste

The biggest cost optimization is not model selection — it is reducing context window waste.

When you paste a 500-line file into a conversation, you are sending approximately 30,000 tokens of input on every subsequent message in that conversation. If the conversation has 10 messages, that file is "re-sent" 9 times — costing 270,000 tokens of overhead for context you are not actively using.

Targeted context pasting (paste the specific function, not the whole file) typically reduces token consumption by 40-60% on code-heavy sessions without changing the quality of answers.

What $0.439 actually buys

$0.439 for 157 conversations works out to $0.0028 per conversation. At that cost, the question of "Claude vs ChatGPT" is almost meaningless from a budget perspective unless you are running a team at scale.

The more meaningful question is: are you using the right model for each task type? My data suggests most developers default to their preferred premium model for everything — using Claude Opus when Sonnet would do, using GPT-4o when 4o mini would do.

The cost of using the right model for each task is trivial to track. TokenPulse shows estimated cost per conversation and per day across all platforms — free, no API key, runs in your browser.

The actual cost comparison to use

Stop comparing per-million-token pricing. Start tracking your real cost per conversation by task type. After two weeks of data you will know exactly which model is cheapest for your actual workflow.

The numbers will probably surprise you.

How to Track Costs Across Multiple AI APIs Without a Backend

Anoop Kumar — Wed, 16 Sep 2026 18:38:16 +0000

Most AI cost tracking tools require you to route your API calls through their backend so they can intercept and log usage. TokenPulse does it entirely client-side — no backend, no API key, all data stays in your browser. Here is the full implementation.

The approach

Instead of intercepting API calls, we estimate costs from what is visible in the browser:

Read token counts from the platform (Claude) or estimate from DOM text
Detect which model is being used
Apply the model's pricing
Aggregate by conversation, day, and week in local storage Accuracy is ±8% — sufficient for understanding spend patterns, not for billing.

Token counting

For Claude, the real token count comes from the internal API response. For other platforms, we estimate:

function estimateTokens(text) {
  if (!text) return 0
  // ~4 characters per token for English text
  // ~3 characters per token for code
  // Using 4 as a reasonable average
  return Math.ceil(text.length / 4)
}

function getConversationTokens() {
  const messages = document.querySelectorAll(MESSAGE_SELECTORS[platform])
  let inputTokens = 0
  let outputTokens = 0

  messages.forEach(msg => {
    const isUser = msg.getAttribute('data-message-author-role') === 'user'
      || msg.closest('[data-testid*="user"]')

    const text = msg.textContent || ''
    const tokens = estimateTokens(text)

    if (isUser) {
      inputTokens += tokens
    } else {
      outputTokens += tokens
    }
  })

  return { inputTokens, outputTokens }
}

Separating input and output tokens matters because they have different pricing — output tokens typically cost 3-5x more than input tokens.

Model detection

Each platform exposes the current model somewhere in the DOM or URL:

const MODEL_DETECTORS = {
  claude: () => {
    // Claude shows model in a selector dropdown
    const selector = document.querySelector('[data-testid="model-selector-dropdown"]')
    if (selector) return selector.textContent.trim().toLowerCase()

    // Fallback: check URL params
    const params = new URLSearchParams(window.location.search)
    return params.get('model') || 'claude-sonnet-4-5'
  },

  chatgpt: () => {
    // ChatGPT shows model in the header
    const header = document.querySelector('[class*="model-switcher"]')
    if (header) {
      const text = header.textContent.toLowerCase()
      if (text.includes('4o')) return 'gpt-4o'
      if (text.includes('o1')) return 'o1'
      if (text.includes('4o mini')) return 'gpt-4o-mini'
    }
    return 'gpt-4o' // default
  },

  gemini: () => {
    const header = document.querySelector('[class*="model-label"]')
    if (header) {
      const text = header.textContent.toLowerCase()
      if (text.includes('2.0 flash')) return 'gemini-2.0-flash'
      if (text.includes('1.5 pro')) return 'gemini-1.5-pro'
    }
    return 'gemini-2.0-flash'
  },
}

function detectModel() {
  const detector = MODEL_DETECTORS[platform]
  return detector ? detector() : 'unknown'
}

Pricing table

Keep pricing in a single constant — update it when providers change rates:

// Prices in USD per 1,000,000 tokens
const PRICING = {
  // Claude models
  'claude-opus-4':       { input: 15.00, output: 75.00 },
  'claude-sonnet-4':     { input:  3.00, output: 15.00 },
  'claude-sonnet-4-5':   { input:  3.00, output: 15.00 },
  'claude-haiku-4-5':    { input:  0.80, output:  4.00 },

  // OpenAI models
  'gpt-4o':              { input:  2.50, output: 10.00 },
  'gpt-4o-mini':         { input:  0.15, output:  0.60 },
  'o1':                  { input: 15.00, output: 60.00 },
  'o1-mini':             { input:  3.00, output: 12.00 },

  // Google models
  'gemini-2.0-flash':    { input:  0.10, output:  0.40 },
  'gemini-1.5-pro':      { input:  1.25, output:  5.00 },

  // DeepSeek models
  'deepseek-v3':         { input:  0.27, output:  1.10 },
  'deepseek-r1':         { input:  0.55, output:  2.19 },

  // Grok models
  'grok-3':              { input:  3.00, output: 15.00 },
  'grok-3-mini':         { input:  0.30, output:  0.50 },
}

function estimateCost(inputTokens, outputTokens, model) {
  // Normalize model name for lookup
  const key = normalizeModelName(model)
  const price = PRICING[key]

  if (!price) {
    // Unknown model — use a reasonable default
    console.warn(`Unknown model: ${model}, using default pricing`)
    return (inputTokens + outputTokens) / 1_000_000 * 3.00
  }

  const inputCost  = (inputTokens  / 1_000_000) * price.input
  const outputCost = (outputTokens / 1_000_000) * price.output
  return inputCost + outputCost
}

function normalizeModelName(raw) {
  const lower = raw.toLowerCase()
  if (lower.includes('opus'))        return 'claude-opus-4'
  if (lower.includes('sonnet'))      return 'claude-sonnet-4-5'
  if (lower.includes('haiku'))       return 'claude-haiku-4-5'
  if (lower.includes('4o-mini'))     return 'gpt-4o-mini'
  if (lower.includes('4o'))          return 'gpt-4o'
  if (lower.includes('o1-mini'))     return 'o1-mini'
  if (lower.includes('o1'))          return 'o1'
  if (lower.includes('2.0 flash'))   return 'gemini-2.0-flash'
  if (lower.includes('1.5 pro'))     return 'gemini-1.5-pro'
  if (lower.includes('deepseek-r1')) return 'deepseek-r1'
  if (lower.includes('deepseek'))    return 'deepseek-v3'
  if (lower.includes('grok-3-mini')) return 'grok-3-mini'
  if (lower.includes('grok'))        return 'grok-3'
  return lower
}

Aggregating costs in local storage

Store costs at three granularities — conversation, day, and week:

const STORAGE_KEY = 'tp_cost_data'

async function recordConversationCost(platform, model, inputTokens, outputTokens) {
  const cost = estimateCost(inputTokens, outputTokens, model)
  const now = new Date()
  const dayKey = now.toISOString().split('T')[0] // YYYY-MM-DD
  const weekKey = getWeekKey(now)

  const stored = await chrome.storage.local.get(STORAGE_KEY)
  const data = stored[STORAGE_KEY] || {
    conversations: [],
    byDay: {},
    byWeek: {},
  }

  // Record conversation
  data.conversations.push({
    platform,
    model,
    inputTokens,
    outputTokens,
    cost,
    timestamp: now.toISOString(),
  })

  // Trim to last 90 days of conversations
  const cutoff = Date.now() - 90 * 24 * 60 * 60 * 1000
  data.conversations = data.conversations.filter(
    c => new Date(c.timestamp).getTime() > cutoff
  )

  // Aggregate by day
  if (!data.byDay[dayKey]) data.byDay[dayKey] = { cost: 0, tokens: 0, conversations: 0 }
  data.byDay[dayKey].cost += cost
  data.byDay[dayKey].tokens += inputTokens + outputTokens
  data.byDay[dayKey].conversations += 1

  // Aggregate by week
  if (!data.byWeek[weekKey]) data.byWeek[weekKey] = { cost: 0, tokens: 0 }
  data.byWeek[weekKey].cost += cost
  data.byWeek[weekKey].tokens += inputTokens + outputTokens

  await chrome.storage.local.set({ [STORAGE_KEY]: data })
  return cost
}

function getWeekKey(date) {
  const d = new Date(date)
  d.setHours(0, 0, 0, 0)
  d.setDate(d.getDate() - d.getDay()) // Start of week (Sunday)
  return d.toISOString().split('T')[0]
}

Reading aggregated costs for display

async function getCostSummary() {
  const stored = await chrome.storage.local.get(STORAGE_KEY)
  const data = stored[STORAGE_KEY] || { byDay: {}, byWeek: {}, conversations: [] }

  const now = new Date()
  const todayKey = now.toISOString().split('T')[0]
  const weekKey = getWeekKey(now)

  // This conversation (most recent in storage)
  const recent = data.conversations[data.conversations.length - 1]
  const thisConversation = recent?.cost || 0

  // Today
  const today = data.byDay[todayKey]?.cost || 0

  // This week
  const thisWeek = data.byWeek[weekKey]?.cost || 0

  // This month
  const thisMonth = Object.entries(data.byDay)
    .filter(([key]) => key.startsWith(now.toISOString().slice(0, 7)))
    .reduce((sum, [, val]) => sum + val.cost, 0)

  return {
    thisConversation: formatCost(thisConversation),
    today: formatCost(today),
    thisWeek: formatCost(thisWeek),
    thisMonth: formatCost(thisMonth),
  }
}

function formatCost(usd) {
  if (usd < 0.001) return '$0.000'
  if (usd < 0.01)  return `$${usd.toFixed(4)}`
  if (usd < 1)     return `$${usd.toFixed(3)}`
  return `$${usd.toFixed(2)}`
}

Displaying in the popup

// popup.js
async function renderCostSection() {
  const summary = await getCostSummary()

  document.getElementById('cost-conversation').textContent = summary.thisConversation
  document.getElementById('cost-today').textContent = summary.today
  document.getElementById('cost-week').textContent = summary.thisWeek
}

Keeping pricing current

Model prices change. The pricing table needs occasional updates. Two strategies:

Option 1: Hardcode and update with each extension version. Simple, no network requests.

Option 2: Fetch pricing from a remote JSON file:

async function fetchLatestPricing() {
  try {
    const res = await fetch('https://token-pulse.in/api/pricing.json')
    const data = await res.json()
    await chrome.storage.local.set({ pricing: data, pricingFetchedAt: Date.now() })
    return data
  } catch {
    // Fall back to bundled pricing
    return PRICING
  }
}

async function getPricing() {
  const stored = await chrome.storage.local.get(['pricing', 'pricingFetchedAt'])
  const age = Date.now() - (stored.pricingFetchedAt || 0)
  const ONE_DAY = 24 * 60 * 60 * 1000

  if (stored.pricing && age < ONE_DAY) return stored.pricing
  return fetchLatestPricing()
}

TokenPulse currently uses option 1 — the pricing table is bundled and updated with each release. Option 2 would be better long-term but requires a backend endpoint.

Full implementation at github.com/anu-ship-it/TokenPulse.
TokenPulse — free AI cost tracker, no backend required.

Building a Notification System That Doesn't Annoy Users

Anoop Kumar — Tue, 15 Sep 2026 04:31:58 +0000

The first version of TokenPulse's notification system was simple: fire a notification when usage crossed 75%, 90%, or 100%. Users turned it off within a day.

The problem was obvious in retrospect — the notification fired every time the condition was checked, not just when the threshold was first crossed. If you were at 82% and the extension polled every 5 minutes, you got a "you're at 75%" notification every 5 minutes indefinitely.

Here is the three-iteration journey to a notification system users actually keep enabled.

What a good threshold notification system needs

Before writing code, define the behavior precisely:

Fire once when usage first crosses a threshold
Do not fire again for the same threshold in the same window
Reset when usage drops below the threshold
Support multiple thresholds independently
Persist state across service worker restarts Most notification systems get 1 and 2 right but fail on 3 and 5.

Version 1 — fires every poll (wrong)

// Polled every 5 minutes
function checkNotifications(utilization) {
  if (utilization >= 0.75) {
    chrome.notifications.create({
      type: 'basic',
      iconUrl: chrome.runtime.getURL('icons/icon48.png'),
      title: 'TokenPulse',
      message: `You are at ${Math.round(utilization * 100)}% of your limit`,
    })
  }
}

Problem: fires on every poll cycle once above threshold. User gets a notification every 5 minutes.

Version 2 — tracks last notified (better, still wrong)

let lastNotifiedThreshold = 0 // In-memory — dies with service worker

function checkNotifications(utilization) {
  const thresholds = [75, 90, 100]
  const crossed = thresholds.filter(t => utilization * 100 >= t).pop() || 0

  if (crossed > lastNotifiedThreshold) {
    lastNotifiedThreshold = crossed
    fireNotification(crossed, utilization)
  }
}

Problem: lastNotifiedThreshold is in memory. When Chrome kills and restarts the service worker, it resets to 0. The user gets duplicate notifications after every service worker restart.

Also: never resets when usage drops. Once you hit 90%, you never get a 75% notification again even after the window resets.

Version 3 — persisted, threshold-aware, resetting (correct)

// service-worker.js

const THRESHOLDS = [50, 75, 90, 100]

async function shouldNotify(stateKey, currentPct, settings) {
  // Filter to enabled thresholds
  const enabled = THRESHOLDS.filter(t => {
    if (t === 50)  return settings.notify_50  !== false
    if (t === 75)  return settings.notify_75  !== false
    if (t === 90)  return settings.notify_90  !== false
    if (t === 100) return settings.notify_100 !== false
    return false
  })

  if (enabled.length === 0) return null

  // Find the highest threshold crossed
  const crossed = enabled.filter(t => currentPct >= t).pop() || 0

  // Read persisted state
  const stored = await chrome.storage.local.get('lastNotified')
  const lastNotified = stored.lastNotified || {}
  const last = lastNotified[stateKey] || 0

  // Reset: if usage has dropped below the last notified threshold
  if (crossed === 0 && last > 0) {
    lastNotified[stateKey] = 0
    await chrome.storage.local.set({ lastNotified })
    return null
  }

  // No new threshold crossed
  if (crossed <= 0 || crossed <= last) return null

  // New threshold crossed — persist and notify
  lastNotified[stateKey] = crossed
  await chrome.storage.local.set({ lastNotified })
  return crossed
}

The key insight: stateKey is a string that identifies which window and platform we are tracking — for example 'claude_5hour' or 'claude_7day'. This allows independent threshold tracking for each limit.

Firing the notification

async function checkRateLimitNotifications(usage) {
  const settings = await Storage.getSettings()

  // Check 5-hour limit
  const sessionPct = Math.round((usage.five_hour?.utilization || 0) * 100)
  const sessionThreshold = await shouldNotify(
    'claude_5hour',
    sessionPct,
    settings
  )

  if (sessionThreshold) {
    const messages = {
      50:  'Halfway through your 5-hour session limit.',
      75:  'Only 25% of your session remaining.',
      90:  'Almost out — consider wrapping up your session.',
      100: 'Session limit reached. Claude will be restricted.',
    }

    chrome.notifications.create(`tp_session_${sessionThreshold}`, {
      type: 'basic',
      iconUrl: chrome.runtime.getURL('icons/icon48.png'),
      title: `Claude session at ${sessionThreshold}%`,
      message: messages[sessionThreshold],
      priority: sessionThreshold >= 90 ? 2 : 1,
    })
  }

  // Check 7-day limit (same pattern, different key)
  const weeklyPct = Math.round((usage.seven_day?.utilization || 0) * 100)
  const weeklyThreshold = await shouldNotify(
    'claude_7day',
    weeklyPct,
    settings
  )

  if (weeklyThreshold) {
    chrome.notifications.create(`tp_weekly_${weeklyThreshold}`, {
      type: 'basic',
      iconUrl: chrome.runtime.getURL('icons/icon48.png'),
      title: `Claude weekly limit at ${weeklyThreshold}%`,
      message: `Your 7-day usage is at ${weeklyThreshold}%.`,
      priority: weeklyThreshold >= 90 ? 2 : 1,
    })
  }
}

The reset mechanism explained

The reset logic is the part most implementations get wrong.

When does a threshold reset? When the usage drops below the threshold level. If you were notified at 90% and your session resets, usage drops to 0%. The next check sees crossed === 0 and last === 90, so it clears lastNotified[stateKey] to 0.

On the next poll at, say, 30% usage, crossed === 0 still, so nothing fires. When usage climbs back to 76%, crossed === 75 and last === 0, so the 75% notification fires again.

This is exactly the right behavior: you get warned at each threshold once per window, and the warnings reset with the window.

Giving notifications meaningful content

Generic percentage notifications are less useful than contextual ones:

function buildNotificationMessage(threshold, limitType, utilization) {
  const pct = Math.round(utilization * 100)

  // Estimate time remaining based on burn rate
  // (simplified — real implementation tracks usage over time)
  const timeHints = {
    50: 'You probably have enough messages for another 2 hours at your current pace.',
    75: 'Consider wrapping up long sessions — roughly 30-45 minutes of usage left.',
    90: 'Almost out. Finish your current task before the limit hits.',
    100: 'Limit reached. Start a fresh session after the reset.',
  }

  return timeHints[threshold] || `Usage at ${pct}%.`
}

"You probably have enough messages for another 45 minutes" is more actionable than "You are at 75%." Users make better decisions with context.

User controls

Notifications the user cannot control get disabled. Add per-threshold toggles in settings:

// Default settings
const DEFAULT_SETTINGS = {
  notify_50:  false, // Off by default — too early for most users
  notify_75:  true,
  notify_90:  true,
  notify_100: true,
  notify_response_ready: true,
}

50% off by default reduces notification fatigue for new users. They can enable it if they want earlier warnings.

The response-ready notification

One notification that users love and is easy to miss: "your response is ready."

When Claude is generating a long response, developers often switch tabs. The response finishes but they don't notice for several minutes. A notification when generation completes brings them back:

// content-script.js
function watchForResponseCompletion() {
  const observer = new MutationObserver(() => {
    const isGenerating = document.querySelector('[data-generating="true"]')
    const wasGenerating = window._tpWasGenerating

    if (wasGenerating && !isGenerating) {
      chrome.runtime.sendMessage({ type: 'RESPONSE_READY' })
    }

    window._tpWasGenerating = !!isGenerating
  })

  observer.observe(document.body, { childList: true, subtree: true })
}

// service-worker.js
if (msg.type === 'RESPONSE_READY') {
  const settings = await Storage.getSettings()
  if (settings.notify_response_ready === false) return false

  chrome.notifications.create('tp_response_ready', {
    type: 'basic',
    iconUrl: chrome.runtime.getURL('icons/icon48.png'),
    title: 'Response ready',
    message: 'Your Claude response has finished generating.',
    priority: 1,
  })
  return false
}

Full implementation at github.com/anu-ship-it/TokenPulse.
TokenPulse — free AI usage tracker for Claude, ChatGPT, Gemini, DeepSeek and Grok.

How to Inject UI Into Any Website With a Chrome Extension

Anoop Kumar — Wed, 09 Sep 2026 16:15:47 +0000

Injecting UI into a page you don't control is one of the most powerful things a Chrome extension can do. It is also one of the most fragile — platforms update their DOM, your selectors break, and your injected elements disappear. Here is the approach that works reliably for TokenPulse across five different AI platforms.

The core challenge

Modern web applications are single-page apps. The DOM mutates constantly as routes change and components re-render. An element you injected at page load may be removed and recreated by the app's JavaScript five seconds later.

The naive approach — inject on DOMContentLoaded and hope — works on static pages but fails on SPAs.

The MutationObserver pattern

The reliable approach uses a MutationObserver to watch for the target element appearing in the DOM, inject when it appears, and re-inject if it disappears:

const INJECTION_ID = 'tp-bar-container'

function inject(targetElement) {
  // Already injected — do not double inject
  if (document.getElementById(INJECTION_ID)) return

  const container = document.createElement('div')
  container.id = INJECTION_ID
  container.innerHTML = buildBarHTML()

  // Insert before the target element
  targetElement.parentNode.insertBefore(container, targetElement)
}

function findAndInject() {
  const target = findTargetElement()
  if (target) inject(target)
}

function startObserver() {
  const observer = new MutationObserver(() => {
    findAndInject()
  })

  observer.observe(document.body, {
    childList: true,
    subtree: true,
  })

  // Also try immediately in case element already exists
  findAndInject()
}

document.addEventListener('DOMContentLoaded', startObserver)
// Also start on load for apps that render after DOMContentLoaded
window.addEventListener('load', startObserver)

Finding the target element reliably

Target elements move between platform versions. Use a priority list of selectors with fallbacks:

const TARGET_SELECTORS = {
  claude: [
    '[data-testid="chat-input-area"]',
    '.ProseMirror',
    '[contenteditable="true"]',
    'div[class*="InputArea"]',
  ],
  chatgpt: [
    '#prompt-textarea',
    '[data-id="prompt-textarea"]',
    'div[contenteditable="true"][class*="prose"]',
  ],
  gemini: [
    '.input-area-container',
    '[data-testid="input-area"]',
    'rich-textarea',
  ],
}

function findTargetElement() {
  const platform = detectPlatform()
  const selectors = TARGET_SELECTORS[platform] || []

  for (const selector of selectors) {
    const el = document.querySelector(selector)
    if (el) return el
  }
  return null
}

function detectPlatform() {
  const host = window.location.hostname
  if (host.includes('claude.ai')) return 'claude'
  if (host.includes('chatgpt.com')) return 'chatgpt'
  if (host.includes('gemini.google.com')) return 'gemini'
  return 'unknown'
}

Isolating your styles

Your injected CSS will conflict with the host page's styles unless you isolate it. Two approaches:

Approach 1: Shadow DOM (strongest isolation)

function inject(target) {
  const host = document.createElement('div')
  host.id = INJECTION_ID

  // Attach shadow root
  const shadow = host.attachShadow({ mode: 'closed' })

  // Add styles inside shadow
  const style = document.createElement('style')
  style.textContent = `
    .tp-bar {
      height: 4px;
      background: #1C1C22;
      border-radius: 2px;
      overflow: hidden;
      margin-bottom: 4px;
    }
    .tp-bar-fill {
      height: 100%;
      background: linear-gradient(90deg, #00b87a, #00E5A0);
      border-radius: 2px;
      transition: width 0.8s ease;
    }
  `
  shadow.appendChild(style)

  const bar = document.createElement('div')
  bar.className = 'tp-bar'
  bar.innerHTML = '<div class="tp-bar-fill" id="tp-fill"></div>'
  shadow.appendChild(bar)

  target.parentNode.insertBefore(host, target)
}

Shadow DOM means the host page's CSS cannot reach your elements and your CSS cannot leak out. The trade-off: you cannot use external CSS files easily.

Approach 2: Scoped class prefixes (simpler)

Prefix every class name with a unique string and use high-specificity selectors:

/* Injected via style element */
#tp-bar-container .tp-bar {
  all: initial; /* Reset inherited styles */
  display: block;
  height: 4px;
  background: #1C1C22;
  /* ... */
}

all: initial resets all inherited CSS properties to their initial values — effective at preventing host page styles from bleeding in.

Keeping the injection alive

SPAs often remove and recreate sections of the DOM when navigating between conversations. Your injection gets deleted along with the old DOM nodes.

The MutationObserver callback fires on every DOM change — including the deletion of your injected element — so findAndInject() naturally re-injects when needed.

But be careful about performance. MutationObserver callbacks can fire hundreds of times per second on active pages. Add a debounce:

let debounceTimer = null

const observer = new MutationObserver(() => {
  clearTimeout(debounceTimer)
  debounceTimer = setTimeout(findAndInject, 100)
})

100ms debounce means the injection check runs at most 10 times per second — fast enough to feel instant, slow enough not to impact page performance.

Updating injected content

Once injected, update the bar content without re-injecting:

function updateBar(pct) {
  // Target element inside shadow DOM or regular DOM
  const fill = document.getElementById('tp-fill')
    || document.querySelector('#tp-bar-container .tp-bar-fill')

  if (!fill) return // Not injected yet

  fill.style.width = `${Math.min(100, Math.max(0, pct * 100))}%`

  // Update color based on percentage
  if (pct >= 0.9) {
    fill.style.background = '#EF4444' // red
  } else if (pct >= 0.75) {
    fill.style.background = '#F59E0B' // yellow
  } else {
    fill.style.background = 'linear-gradient(90deg, #00b87a, #00E5A0)' // green
  }
}

Handling route changes in SPAs

Claude and ChatGPT use client-side routing. When you navigate to a new conversation, the URL changes but the page does not reload. Your content script stays alive but the DOM changes significantly.

Listen for popstate and pushState events:

// Monkey-patch pushState to detect programmatic navigation
const originalPushState = history.pushState

history.pushState = function(...args) {
  originalPushState.apply(this, args)
  // Small delay for the app to render the new route
  setTimeout(findAndInject, 500)
}

window.addEventListener('popstate', () => {
  setTimeout(findAndInject, 500)
})

The 500ms delay gives the SPA time to render the new route's DOM before you attempt injection.

Cleaning up on unload

Remove your injected elements when the content script unloads to avoid memory leaks:

window.addEventListener('beforeunload', () => {
  const container = document.getElementById(INJECTION_ID)
  if (container) container.remove()
  observer.disconnect()
})

The complete pattern

const INJECTION_ID = 'tp-bar-container'
let observer = null
let debounceTimer = null

function findAndInject() {
  const target = findTargetElement()
  if (target && !document.getElementById(INJECTION_ID)) {
    inject(target)
  }
}

function startObserving() {
  if (observer) observer.disconnect()

  observer = new MutationObserver(() => {
    clearTimeout(debounceTimer)
    debounceTimer = setTimeout(findAndInject, 100)
  })

  observer.observe(document.body, {
    childList: true,
    subtree: true,
  })

  findAndInject()
}

document.addEventListener('DOMContentLoaded', startObserving)
window.addEventListener('load', startObserving)

// Handle SPA navigation
const originalPushState = history.pushState
history.pushState = function(...args) {
  originalPushState.apply(this, args)
  setTimeout(findAndInject, 500)
}
window.addEventListener('popstate', () => setTimeout(findAndInject, 500))

This pattern has kept TokenPulse's bar injected through dozens of Claude and ChatGPT updates without manual selector fixes.

Full implementation at github.com/anu-ship-it/TokenPulse.
TokenPulse — free token tracker, installs in 30 seconds.

MV3 Chrome Extensions — Everything That Broke and How I Fixed It

Anoop Kumar — Mon, 07 Sep 2026 17:22:12 +0000

When I started building TokenPulse as a Manifest V3 extension, I knew MV3 was stricter than MV2. I did not know how many things would break silently, produce cryptic errors, or fail only in production. Here is every significant MV3 problem I hit and how I fixed each one.

Problem 1: Inline scripts violate CSP

The first error I saw after installing my extension:

Refused to execute inline script because it violates 
the following Content Security Policy directive: 
'script-src 'self''

MV3 enforces a strict Content Security Policy that blocks all inline JavaScript. This includes:

<script> tags with inline content
onclick, onload, and other inline event handlers
javascript: URLs
eval() and new Function() The fix: Move all JavaScript to external files and wire all event handlers via addEventListener after DOMContentLoaded.

Before (broken):

<button onclick="handleClick()">Click me</button>
<script>
  function handleClick() { ... }
</script>

After (correct):

<button id="my-btn">Click me</button>
<script src="popup.js"></script>

// popup.js
document.addEventListener('DOMContentLoaded', () => {
  document.getElementById('my-btn')
    .addEventListener('click', handleClick)
})

function handleClick() { ... }

Every single event handler in every HTML file needs this treatment. I had 23 inline handlers across my popup and welcome pages. Every one was silent until I opened the browser console.

Problem 2: Service worker dies between messages

This one cost me the most debugging time. The error:

Error: A listener indicated an asynchronous response by 
returning true, but the message channel closed before 
a response was received

MV3 service workers are not persistent. Chrome kills them after ~30 seconds of inactivity and restarts them on demand. Code that assumes the service worker is alive between messages will fail intermittently.

The broken pattern:

// service-worker.js
let cachedData = null // Dies with the worker

chrome.runtime.onMessage.addListener(async (msg, sender, sendResponse) => {
  // BROKEN: async on the listener returns a Promise, not true
  const data = await fetchSomething()
  sendResponse(data)
})

Two bugs here: async listener and in-memory state.

The fix:

chrome.runtime.onMessage.addListener((msg, sender, sendResponse) => {
  if (msg.type === 'GET_DATA') {
    // Correct: IIFE for async, returns true synchronously
    ;(async () => {
      const data = await Storage.getData() // Always from storage
      sendResponse(data)
    })()
    return true // Keep channel open
  }

  if (msg.type === 'SAVE_DATA') {
    // Fire and forget — no response needed
    Storage.saveData(msg.data)
    return false // Close channel immediately
  }
})

The rule: never return true unless you are going to call sendResponse. Never use async on the listener itself. Always use chrome.storage instead of in-memory state.

Problem 3: web_accessible_resources blocks script injection

My content script injected a page-context script to intercept fetch calls:

const script = document.createElement('script')
script.src = chrome.runtime.getURL('injected.js')
document.head.appendChild(script)

This silently failed — the script element was added to the DOM but the script never executed. No error in the console.

The fix: Declare the injected script in web_accessible_resources in manifest.json:

{
  "web_accessible_resources": [{
    "resources": ["injected.js"],
    "matches": ["https://claude.ai/*", "https://chatgpt.com/*"]
  }]
}

The matches array must include every domain where the script will be injected. Using <all_urls> works but is overkill and may trigger Chrome Web Store review scrutiny.

Problem 4: Alarms require permissions

I used chrome.alarms for periodic polling. The alarm was created but never fired.

chrome.alarms.create('poll', { periodInMinutes: 5 })
// Never fired

The fix: Add the alarms permission to manifest.json:

{
  "permissions": ["alarms", "storage", "notifications"]
}

Missing permissions fail silently in many cases — the API exists and the call succeeds but nothing happens. Always check the permissions list when a Chrome API does not behave as expected.

Problem 5: Notifications require both permission and icon

My notifications never appeared. No error, just silence.

chrome.notifications.create('alert', {
  type: 'basic',
  title: 'TokenPulse',
  message: 'You are at 90% of your limit',
  // Missing: iconUrl
})

The fix: iconUrl is required for notifications to display:

chrome.notifications.create('alert', {
  type: 'basic',
  iconUrl: chrome.runtime.getURL('icons/icon48.png'),
  title: 'TokenPulse',
  message: 'You are at 90% of your limit',
  priority: 1,
})

Also ensure the icon file is declared in web_accessible_resources if it is being accessed from a content script or injected page context.

Problem 6: Content script world isolation

My content script set a global variable that I expected to read from the injected page script:

// content-script.js
window.tokenPulseData = { ready: true }

// injected.js (page context)
console.log(window.tokenPulseData) // undefined

The reason: Content scripts run in an isolated world. They have their own window object separate from the page's window. Setting window.x in a content script does not affect window.x in the page.

The fix: Use custom events to communicate between contexts:

// content-script.js → page context
window.dispatchEvent(new CustomEvent('tp-init', {
  detail: { ready: true }
}))

// injected.js (page context)
window.addEventListener('tp-init', (e) => {
  console.log(e.detail.ready) // true
})

Problem 7: chrome.storage.local is async everywhere

MV2 background pages could use synchronous APIs. MV3 service workers cannot. Every chrome.storage operation is async:

// Broken — storage.get is async
const data = chrome.storage.local.get('key')
console.log(data.key) // undefined

// Correct
const result = await chrome.storage.local.get('key')
console.log(result.key) // correct value

// Or with callback
chrome.storage.local.get('key', (result) => {
  console.log(result.key)
})

This is a pervasive change if you are migrating from MV2. Every storage read needs to be awaited or use a callback.

Problem 8: Extension context invalidated after reload

When you reload an extension during development, content scripts already injected into open tabs become orphaned. Their chrome.runtime context is invalidated:

Error: Extension context invalidated

This only happens during development but it is confusing. The fix is to reload the tab after reloading the extension, or wrap chrome API calls in try-catch during development:

function safeSendMessage(msg) {
  try {
    chrome.runtime.sendMessage(msg)
  } catch (e) {
    if (e.message.includes('Extension context invalidated')) {
      // Context gone — page needs refresh
      return
    }
    throw e
  }
}

The debugging workflow that saved me

For content script issues: Add debugger statements and use Chrome's "Inspect" on the content script in DevTools → Sources → Content Scripts.

For service worker issues: Go to chrome://extensions → your extension → "Service Worker" link → opens a dedicated DevTools for the worker.

For popup issues: Right-click the extension icon → "Inspect popup" — opens DevTools attached to the popup window.

For manifest issues: chrome://extensions shows errors prominently. Always check here first when something silently fails.

Full source at github.com/anu-ship-it/TokenPulse.
TokenPulse — free token tracker for Claude, ChatGPT, Gemini, DeepSeek and Grok.

How I Built a Chrome Extension That Reads Claude's Internal API

Anoop Kumar — Fri, 04 Sep 2026 09:35:02 +0000

When I started building TokenPulse, I assumed I would need an API key to read Claude's rate limit data. I was wrong. Claude exposes exact utilization percentages and reset timestamps through an internal endpoint that your browser session already has access to — because Claude's own interface uses it.

Here is exactly how it works and how I built the integration.

Discovering the endpoint

The first step was understanding what data Claude's interface actually requests. I opened DevTools → Network tab, filtered by XHR and Fetch, and started a conversation with Claude while watching the requests.

Within a few messages, I spotted a request to an endpoint returning usage data:

{
  "raw_limits": {
    "five_hour": {
      "remaining_tokens": 18432,
      "total_tokens": 90000,
      "reset_at": "2026-07-15T14:14:00.000Z"
    },
    "seven_day": {
      "remaining_tokens": 58900,
      "total_tokens": 90000,
      "reset_at": "2026-07-21T21:00:00.000Z"
    }
  }
}

The browser already had permission to read this because the session cookie was included automatically. No API key needed.

Intercepting the response in a content script

Content scripts run in an isolated world — they cannot directly intercept fetch calls made by the page's JavaScript. The solution is injecting a script into the page's actual JavaScript context:

// content-script.js
function injectPageScript() {
  const script = document.createElement('script')
  script.src = chrome.runtime.getURL('injected.js')
  script.onload = () => script.remove()
  ;(document.head || document.documentElement).appendChild(script)
}

injectPageScript()

The injected script runs in page context and intercepts fetch calls:

// injected.js
const originalFetch = window.fetch

window.fetch = async function(...args) {
  const response = await originalFetch.apply(this, args)
  const url = typeof args[0] === 'string' ? args[0] : args[0].url

  if (url.includes('/api/usage') || url.includes('/rate_limits')) {
    const clone = response.clone()
    clone.json().then(data => {
      window.dispatchEvent(new CustomEvent('tp-usage-data', {
        detail: data
      }))
    }).catch(() => {})
  }

  return response
}

The content script listens for the custom event:

window.addEventListener('tp-usage-data', (event) => {
  processUsageData(event.detail)
})

Parsing and displaying the data

function parseRateLimits(raw) {
  const now = Date.now()

  function parseWindow(w) {
    if (!w) return null
    const total = w.total_tokens || 90000
    const remaining = w.remaining_tokens || total
    const used = total - remaining
    const resetAt = w.reset_at ? new Date(w.reset_at).getTime() : null
    const minutesUntilReset = resetAt
      ? Math.floor(Math.max(0, resetAt - now) / 60000)
      : null

    return {
      utilization: Math.min(1, used / total),
      minutesUntilReset,
      resetAt,
    }
  }

  return {
    five_hour: parseWindow(raw?.raw_limits?.five_hour),
    seven_day: parseWindow(raw?.raw_limits?.seven_day),
  }
}

function formatCountdown(minutes) {
  if (!minutes || minutes <= 0) return 'Resetting...'
  const h = Math.floor(minutes / 60)
  const m = minutes % 60
  if (h === 0) return `${m}m`
  if (m === 0) return `${h}h`
  return `${h}h ${m}m`
}

The MV3 manifest requirement

For the injected script to load, declare it in web_accessible_resources:

{
  "web_accessible_resources": [{
    "resources": ["injected.js", "icons/icon48.png"],
    "matches": ["https://claude.ai/*"]
  }]
}

Without this the injection silently fails.

What does not work and why

Direct fetch from service worker — fails with CORS because session cookies are not sent cross-origin from the service worker context.

Reading cookies directly — Claude's session tokens are HttpOnly, so JavaScript cannot read them.

The page-context injection pattern is the only reliable approach for intercepting authenticated requests made by the page's own JavaScript.

Full implementation at github.com/anu-ship-it/TokenPulse.
TokenPulse is free — tracks Claude, ChatGPT, Gemini, DeepSeek and Grok.

System Design Fundamentals I Wish I Had Learned Earlier

Anoop Kumar — Fri, 21 Aug 2026 15:17:19 +0000

Most system design resources focus on interview preparation — how to design Twitter in 45 minutes, how to draw boxes and arrows convincingly. This article is different. These are the concepts I wish I had internalized earlier because they changed how I approach every technical decision, including the ones I make building TokenPulse.

The fundamental trade-off: consistency vs availability

The CAP theorem states that a distributed system can guarantee at most two of three properties: Consistency, Availability, and Partition tolerance. Since network partitions are unavoidable in real systems, the practical trade-off is between consistency and availability.

Consistency means every read receives the most recent write. If user A writes data and user B immediately reads it, B sees A's write.

Availability means every request receives a response — not necessarily the most recent data, but a response.

Most web applications choose availability over strict consistency, accepting eventual consistency — the guarantee that data will become consistent eventually, even if reads may return slightly stale data during the propagation window.

The practical example: if you update your profile picture on a social platform, some users might see the old picture for a few seconds or minutes while the change propagates. This is acceptable. If your bank account balance is wrong for even one second, that is not acceptable.

Knowing which side of this trade-off your application falls on should drive every database and caching decision you make.

Horizontal vs vertical scaling — and when each applies

Vertical scaling (scaling up) means adding more resources to a single machine — more CPU, more RAM, faster storage. It is simple, requires no architectural changes, and works until you hit the hardware limit or the cost becomes prohibitive.

Horizontal scaling (scaling out) means adding more machines and distributing load across them. It requires your application to be stateless — requests from the same user can be handled by any server.

The question developers often ask is "which should I use?" The correct answer is: vertical first, horizontal when necessary.

Vertical scaling is dramatically simpler. You resize an instance and you are done. Horizontal scaling requires load balancers, session management, distributed caching, and careful thinking about where state lives.

The mistake is assuming you need horizontal scaling before you have validated the need. A well-optimized single server can handle enormous traffic. Stack Overflow ran on five servers for years at significant scale. Optimize your code and your queries before you add machines.

When you do need horizontal scaling, the key constraint is statelessness. Your application servers must not store session state locally. Sessions go in Redis. Files go in object storage. Database writes go to a shared database. Each server is identical and interchangeable.

Caching — the most important performance tool

Caching is storing the result of an expensive computation so you can serve it quickly on subsequent requests. It is the single highest-leverage performance improvement in most systems.

There are four places you can cache:

Client-side (browser cache): HTTP cache headers (Cache-Control, ETag, Last-Modified) tell browsers how long to cache static assets. A properly configured CDN and cache policy means your CSS and JavaScript files are served from the user's local cache — zero network round-trips.

CDN cache: Content delivery networks cache responses at edge nodes geographically close to users. Dynamic content that does not change per-user — marketing pages, blog posts, public API responses — should be cached at the CDN.

Application cache (Redis/Memcached): Server-side in-memory caching for database query results, computed aggregations, and anything expensive to regenerate. Redis is the standard choice — it is fast, supports data structures, and handles expiration automatically.

Database query cache: Most databases cache query results internally. This is largely automatic, but it means repeated identical queries are fast and schema changes or cache invalidation can cause temporary performance drops.

The hardest problem in caching is cache invalidation — knowing when to expire stale data. Three strategies:

TTL (time-to-live): Data expires after a fixed duration. Simple, but data may be stale within the TTL window.

Write-through: When data is written to the database, the cache is updated simultaneously. Consistent, but every write hits both database and cache.

Cache-aside: Application checks cache first. On a miss, reads from database and populates the cache. On a write, invalidates the cache entry. The most common pattern for read-heavy workloads.

Database indexing — what it is and why it matters

Without an index, a database must scan every row in a table to find matching records. This is a full table scan, and its cost grows linearly with the number of rows.

An index is a separate data structure (typically a B-tree) that allows the database to find matching rows in O(log n) time instead of O(n) time. For a table with one million rows, this is the difference between scanning 1,000,000 rows and scanning roughly 20.

The columns to index:

Primary keys — indexed automatically
Foreign keys — columns used in JOIN conditions
Columns in WHERE clauses — especially high-cardinality columns (many distinct values)
Columns in ORDER BY — if you frequently sort by a column The trade-off: indexes make reads fast and writes slow. Every INSERT, UPDATE, or DELETE must also update every relevant index. For read-heavy applications (most web applications), this trade-off is almost always worth it. For write-heavy applications (logging, analytics ingestion), index carefully.

Composite indexes — indexes on multiple columns — matter for query performance. An index on (user_id, created_at) supports queries that filter by user_id and sort by created_at, but only if the columns appear in that order in the WHERE clause.

Message queues — decoupling for resilience

A message queue is a buffer between a producer (something that generates work) and a consumer (something that processes work). The producer puts a message in the queue and moves on. The consumer reads from the queue and processes at its own pace.

Why this matters:

Handling traffic spikes. If your API receives 10,000 requests in one second, a queue absorbs the burst and processes it steadily rather than overwhelming your database.

Decoupling services. If the email service is down, you do not want to fail the user's registration request. Put the "send welcome email" task in a queue. When the email service recovers, it processes the queue.

Retry logic. If a queue consumer fails to process a message, the message stays in the queue (or moves to a dead letter queue) and is retried. Without a queue, failed tasks are lost.

The practical example from TokenPulse: when a user joins the Pro waitlist, the API endpoint puts two tasks in a queue — "send notification email" and "write to Google Sheets." If Google Sheets is temporarily unavailable, the task is retried automatically. The user's HTTP response does not wait for either task to complete.

Load balancing strategies

A load balancer distributes incoming requests across multiple servers. The distribution strategy matters:

Round robin: Requests go to servers in order — 1, 2, 3, 1, 2, 3. Simple, works when servers are identical and requests take similar time.

Least connections: Requests go to the server with the fewest active connections. Better when requests have variable processing time.

IP hash: The same client IP always routes to the same server. Useful when you cannot make your application fully stateless and need session affinity.

Weighted: Some servers receive more traffic than others — useful when servers have different capacities.

For most Next.js applications on Vercel, load balancing is handled automatically. Understanding the strategies matters when you operate your own infrastructure or need to reason about why requests are distributed the way they are.

The read replica pattern

Write operations go to a primary database. Read operations go to one or more replica databases that are kept in sync with the primary.

This pattern solves a specific problem: most web applications have dramatically more reads than writes. A read-heavy query that scans a large table on the primary database competes with write operations for I/O. Moving reads to a replica eliminates this contention.

The trade-off is replication lag — replicas may be slightly behind the primary. For most reads (show me recent posts, display the product catalog), this is acceptable. For reads that immediately follow a write (show me the thing I just created), you need to either read from the primary or implement a mechanism to wait for replication.

How to think about trade-offs like a senior engineer

Every system design decision is a trade-off. Senior engineers do not make universally correct decisions — they make decisions appropriate to their specific constraints.

The questions to ask for any architectural decision:

What is the actual scale? Do not design for 10 million users if you have 100. Design for 10x your current scale and revisit when you get there.

What is the cost of being wrong? If a caching strategy is wrong, you serve stale data. If a database schema is wrong, you have a migration. Different stakes require different confidence levels before deciding.

What can you change later? Some decisions are easily reversible — switching a caching library, changing an index. Others are difficult — changing your data model, switching databases. Spend more time on irreversible decisions.

What do you actually know? Optimize for known bottlenecks, not hypothetical ones. Use monitoring and profiling to find real problems before solving imagined ones.

If you are building AI-powered applications and want visibility into your token usage costs and rate limits, TokenPulse is a free Chrome extension that tracks usage across Claude, ChatGPT, Gemini, DeepSeek and Grok — no API key required.

What I Learned Building a Developer Tool for AI Usage — 6 Weeks of Engineering and Product Lessons

Anoop Kumar — Fri, 21 Aug 2026 15:15:18 +0000

Six weeks ago I started building TokenPulse after getting cut off mid-debugging session by Claude's rate limit one too many times. The extension is now live with active users across five platforms. Here is what I actually learned — the engineering decisions, product mistakes, and things I would do differently.

The problem was real but I almost built the wrong solution

My initial instinct was to build a browser extension that scraped Claude's rate limit data and sent it to a backend that users could check via a dashboard. I spent three days on the backend architecture before I stopped and asked the obvious question: why would someone open a separate dashboard to check information they need while they are already inside Claude?

The right solution was obvious once I asked it: inject the information directly into the page they are already on. A slim bar above the input box. Always visible, never intrusive, no tab switching required.

The lesson: solve the problem in the context where it actually occurs. Do not add steps.

MV3 was harder than the documentation suggested

Chrome's Manifest V3 requirements were the first major technical challenge. The biggest constraint is Content Security Policy — no inline scripts, no eval, no remotely loaded JavaScript.

This broke my initial implementation in three ways:

Inline event handlers. Every onclick="..." attribute in my HTML failed silently. The fix is addEventListener for everything, wired after DOMContentLoaded. Simple in hindsight, took me a day to fully debug.

Service worker lifecycle. MV3 background scripts are service workers — Chrome can kill them at any time. Code that assumes the background script is alive between messages will fail intermittently and be almost impossible to reproduce. Everything has to go through chrome.storage.local.

Message channel management. The chrome.runtime.onMessage listener has a specific contract: return true if you are going to call sendResponse asynchronously, return false (or nothing) if you are not. Getting this wrong causes the dreaded "message channel closed before response was received" error that appears randomly and is hard to trace.

The dual selector problem

Every platform — Claude, ChatGPT, Gemini, DeepSeek, Grok — updates its frontend regularly. I learned this the hard way when a ChatGPT update broke my content script three days after I published.

The solution is defensive selector arrays with fallbacks:

const INPUT_SELECTORS = [
  '#prompt-textarea',           // primary
  '[data-id="prompt-textarea"]', // fallback 1
  'div[contenteditable="true"]', // fallback 2
]

function findInput() {
  for (const selector of INPUT_SELECTORS) {
    const el = document.querySelector(selector)
    if (el) return el
  }
  return null
}

Never rely on a single selector. Always have at least two fallbacks. Check for null before every DOM operation.

I shipped a feature nobody asked for and skipped one everyone wanted

I spent significant time building a detailed token tips panel — a collapsible section in the popup with optimization advice. Nobody uses it. Zero feedback mentions it. It adds visual weight and code complexity for no user value.

Meanwhile, the most common piece of feedback in the first two weeks was "can you show me how much I've spent this week across all platforms?" — a weekly summary. I had cost tracking per conversation but no weekly rollup.

The lesson is not "do user research before building" — I had done that. The lesson is that users cannot tell you what they want until they are using the product. The tips panel sounded useful in theory. The weekly summary revealed itself from actual usage patterns.

Ship the minimum, watch what users do, build what they actually reach for.

The notification system needed three rewrites

My first notification implementation fired every time the threshold was crossed, which meant multiple notifications within the same session. Users turned it off immediately.

My second implementation stored the last notified percentage and only fired when the user crossed a higher threshold. Better — but it never reset, so once you hit 90% and your session reset, you would not get notified at 75% in the next session.

The third implementation tracks the last notified threshold per window, resets it when usage drops below the threshold, and fires once per crossing per window. This is the version that users actually keep enabled.

The algorithm sounds simple when written out. Getting there took three iterations and specific user feedback about each failure mode.

Choosing Vercel over AWS was correct

I spent two days considering hosting options for the marketing site before choosing Vercel. The alternative I seriously considered was AWS — EC2 with nginx and PM2.

Vercel was correct for three reasons:

Zero operational overhead. No instance management, no SSL renewal, no process monitoring. Deployment is git push.
The credentials problem doesn't change. Whether you deploy on Vercel or EC2, you still need Google Sheets API credentials in environment variables. Switching infrastructure doesn't make that easier.
Revenue is zero. Until the product generates money, infrastructure cost is a constraint. Vercel hobby tier is free. EC2 t3.micro is $8-10/month. The right time to consider AWS is when you have specific requirements Vercel cannot meet — custom runtime environments, GPU access, VPC networking, specific regional compliance. For a Next.js marketing site with API routes, those requirements do not exist yet.

The Chrome Web Store review process is slower than you expect

First submission: 3 days to review, rejected for missing privacy policy disclosures in the store listing.

Second submission after fixing: 2 days to review, approved.

This means plan for a minimum 5-7 day runway between "finished" and "published." If you are building toward a launch date, submit to the Chrome Web Store at least a week before you want to launch.

Also: the Chrome Web Store's description field is SEO. The keywords in your listing title and description affect where your extension appears in the store's own search. "TokenPulse — Claude Rate Limit & Token Tracker" performs better than "TokenPulse" alone.

What I would do differently

Build the popup last. The in-page bar is the core value. I built the popup first because it felt more like a "real" extension. The bar is what users actually use 90% of the time.

Write the content scripts with a test harness from day one. Testing content scripts means opening the browser, loading the extension, navigating to the target page, and observing behavior. This loop takes 2-3 minutes per test. A mock DOM environment for unit testing would have saved significant time on the notification system rewrites.

Set up error tracking earlier. I added error logging after shipping. I had no visibility into content script failures for the first two weeks. Users were silently experiencing broken functionality that I only discovered through direct feedback.

Launch on one platform, add others after. I launched with Claude and ChatGPT support simultaneously. The dual-platform debugging effort at launch was chaotic. Claude alone first, then ChatGPT, would have been cleaner.

TokenPulse is free and open source at github.com/anu-ship-it/TokenPulse. If you build with AI tools daily and want visibility into your usage, install it here — works on Claude, ChatGPT, Gemini, DeepSeek and Grok with no API key.

Happy to answer questions about any of the technical decisions in the comments.

Building a Full-Stack AI Application with Next.js — Architecture and Implementation

Anoop Kumar — Fri, 21 Aug 2026 15:13:55 +0000

Building an AI application is easy. Building one that handles streaming responses, manages costs, scales cleanly, and doesn't leak API keys takes more thought. This guide covers the architecture decisions and implementation patterns I have learned from building TokenPulse and several other AI-powered tools with Next.js.

The stack

Next.js 15 (App Router)
TypeScript
Tailwind CSS
Vercel (hosting + edge functions)
Resend (transactional email)

This stack handles most AI application requirements without introducing unnecessary complexity. Next.js App Router gives you server components, route handlers, and streaming out of the box.

Project structure

app/
  layout.tsx          — root layout, metadata, providers
  page.tsx            — landing page
  api/
    chat/route.ts     — AI streaming endpoint
    usage/route.ts    — usage tracking
  dashboard/
    page.tsx          — server component
    client.tsx        — client component with state
lib/
  ai.ts              — AI client configuration
  auth.ts            — session handling
  db.ts              — database client
components/
  chat/
    ChatWindow.tsx    — client component
    Message.tsx       — pure component
    StreamingText.tsx — streaming display

The key architectural decision: keep AI calls in server-side route handlers, never in client components. API keys stay server-side. The client receives streamed responses but never touches credentials.

Setting up the AI client

// lib/ai.ts
import Anthropic from '@anthropic-ai/sdk'

// Single instance — reused across requests
export const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
})

// Type-safe model constants
export const MODELS = {
  fast:    'claude-haiku-4-5',
  default: 'claude-sonnet-4-5',
  smart:   'claude-opus-4-5',
} as const

export type ModelKey = keyof typeof MODELS

Never instantiate the client per-request. The SDK handles connection pooling internally and a single instance is more efficient.

Streaming responses with App Router

Streaming is the single most important UX improvement for AI applications. Without streaming, users stare at a blank screen for 3-10 seconds. With streaming, they see content appear token by token.

// app/api/chat/route.ts
import { anthropic, MODELS } from '@/lib/ai'

export const runtime = 'nodejs' // required for streaming

export async function POST(req: Request) {
  const { messages, model = 'default' } = await req.json()

  // Validate input
  if (!messages?.length) {
    return Response.json({ error: 'Messages required' }, { status: 400 })
  }

  // Create streaming response
  const stream = anthropic.messages.stream({
    model: MODELS[model as keyof typeof MODELS] ?? MODELS.default,
    max_tokens: 4096,
    messages,
  })

  // Convert to ReadableStream for the Response
  const readable = new ReadableStream({
    async start(controller) {
      for await (const chunk of stream) {
        if (
          chunk.type === 'content_block_delta' &&
          chunk.delta.type === 'text_delta'
        ) {
          controller.enqueue(
            new TextEncoder().encode(chunk.delta.text)
          )
        }
      }
      controller.close()
    },
  })

  return new Response(readable, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
      'Transfer-Encoding': 'chunked',
      'X-Content-Type-Options': 'nosniff',
    },
  })
}

Client-side streaming consumption

// components/chat/ChatWindow.tsx
'use client'
import { useState, useRef } from 'react'

export default function ChatWindow() {
  const [messages, setMessages] = useState<Message[]>([])
  const [streaming, setStreaming] = useState(false)
  const abortRef = useRef<AbortController | null>(null)

  async function sendMessage(content: string) {
    const userMessage = { role: 'user' as const, content }
    const newMessages = [...messages, userMessage]
    setMessages(newMessages)
    setStreaming(true)

    // Add empty assistant message to fill in
    setMessages(prev => [...prev, { role: 'assistant', content: '' }])

    abortRef.current = new AbortController()

    try {
      const res = await fetch('/api/chat', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ messages: newMessages }),
        signal: abortRef.current.signal,
      })

      if (!res.ok) throw new Error('Request failed')

      const reader = res.body!.getReader()
      const decoder = new TextDecoder()

      while (true) {
        const { done, value } = await reader.read()
        if (done) break

        const text = decoder.decode(value, { stream: true })

        // Append to last message
        setMessages(prev => {
          const updated = [...prev]
          updated[updated.length - 1] = {
            role: 'assistant',
            content: updated[updated.length - 1].content + text,
          }
          return updated
        })
      }
    } catch (err) {
      if ((err as Error).name !== 'AbortError') {
        console.error('Stream error:', err)
      }
    } finally {
      setStreaming(false)
      abortRef.current = null
    }
  }

  function stopStreaming() {
    abortRef.current?.abort()
  }

  return (
    <div>
      {messages.map((msg, i) => (
        <div key={i} className={msg.role === 'user' ? 'user' : 'assistant'}>
          {msg.content}
        </div>
      ))}
      {/* Input and controls */}
    </div>
  )
}

Rate limiting API routes

Without rate limiting, a single user can exhaust your AI credits in minutes. Implement token bucket rate limiting at the route level:

// lib/ratelimit.ts
const requests = new Map<string, { count: number; reset: number }>()

export function rateLimit(
  identifier: string,
  limit = 20,
  windowMs = 60_000
): { success: boolean; remaining: number } {
  const now = Date.now()
  const record = requests.get(identifier)

  if (!record || now > record.reset) {
    requests.set(identifier, { count: 1, reset: now + windowMs })
    return { success: true, remaining: limit - 1 }
  }

  if (record.count >= limit) {
    return { success: false, remaining: 0 }
  }

  record.count++
  return { success: true, remaining: limit - record.count }
}

// In your route handler
import { rateLimit } from '@/lib/ratelimit'

export async function POST(req: Request) {
  const ip = req.headers.get('x-forwarded-for') ?? 'unknown'
  const { success, remaining } = rateLimit(ip)

  if (!success) {
    return Response.json(
      { error: 'Rate limit exceeded' },
      { status: 429, headers: { 'Retry-After': '60' } }
    )
  }
  // ...
}

For production, use Upstash Redis with their @upstash/ratelimit package — the in-memory approach above does not work across multiple serverless function instances.

Cost tracking per request

Track costs at the API layer so you have data on actual spend:

// After streaming completes, get usage data
const finalMessage = await stream.finalMessage()
const usage = finalMessage.usage

const cost = calculateCost(
  usage.input_tokens,
  usage.output_tokens,
  model
)

// Log to your database or analytics
await logUsage({
  userId: session?.userId,
  model,
  inputTokens: usage.input_tokens,
  outputTokens: usage.output_tokens,
  cost,
  timestamp: new Date(),
})

function calculateCost(
  inputTokens: number,
  outputTokens: number,
  model: string
): number {
  const pricing: Record<string, { input: number; output: number }> = {
    'claude-haiku-4-5':  { input: 0.80,  output: 4.00  },
    'claude-sonnet-4-5': { input: 3.00,  output: 15.00 },
    'claude-opus-4-5':   { input: 15.00, output: 75.00 },
  }

  const p = pricing[model]
  if (!p) return 0

  return (inputTokens / 1_000_000) * p.input +
         (outputTokens / 1_000_000) * p.output
}

Environment variables — never expose them

Three rules:

1. Never use NEXT_PUBLIC_ prefix for AI API keys. NEXT_PUBLIC_ variables are bundled into the client JavaScript and visible to anyone who opens DevTools.

2. Always validate on startup. Add a check in your root layout or a startup file:

// lib/env.ts
const required = [
  'ANTHROPIC_API_KEY',
  'DATABASE_URL',
]

for (const key of required) {
  if (!process.env[key]) {
    throw new Error(`Missing required environment variable: ${key}`)
  }
}

3. Use .env.example committed to git, .env.local never committed. Your .gitignore should always include .env.local.

Deployment on Vercel

One vercel.json setting that matters for streaming:

{
  "functions": {
    "app/api/chat/route.ts": {
      "maxDuration": 60
    }
  }
}

Default Vercel function timeout is 10 seconds on hobby plans — not enough for long AI responses. Set maxDuration to 60 on pro plans, or implement response chunking to stay within the limit.

Also add export const runtime = 'nodejs' to any route that uses streaming. The Edge runtime does not support all Node.js APIs that the AI SDKs depend on.

The architecture decision that matters most

Keep AI logic in server-side route handlers. The temptation is to call AI APIs directly from client components using the AI SDK's client-side helpers. Resist it.

Server-side AI calls give you:

API key security
Cost logging at the server level
Rate limiting before the request reaches the AI provider
Error handling in one place The extra round-trip through your API layer is worth every one of these.

The full source code for TokenPulse's website is available at github.com/anu-ship-it/TokenPulse. If you are building AI tools and want to track token usage in real time, TokenPulse is free and works with no API key.

How Context Windows Actually Work in Large Language Models

Anoop Kumar — Fri, 21 Aug 2026 15:06:58 +0000

The context window is the most misunderstood concept in practical LLM usage. Most developers treat it as a hard boundary — "the model can see X tokens" — but the reality is more nuanced and more consequential for how you structure your prompts.

This is a technical explanation of what context windows actually are, how attention mechanisms work, and why the useful context limit is significantly smaller than the advertised number.

What a context window contains

Every time you send a message to a language model, the entire conversation history is sent with it. The model does not remember previous messages — it receives them all, in order, as a single input sequence.

That sequence contains:

Your system prompt or instructions
Every user message in the conversation
Every model response in the conversation
Any documents, code, or files you have pasted
Tool use results if the model used function calling All of this is concatenated into a single sequence of tokens and processed together. The context window is the maximum length of this sequence.

Claude Sonnet 4 processes up to 200,000 tokens at once. GPT-4o handles 128,000. Gemini 1.5 Pro handles 1,000,000. These numbers sound enormous until you realize that a single verbose debugging session can consume 30,000-50,000 tokens.

How attention works — the mechanism behind context

To understand why context windows behave the way they do, you need to understand attention — specifically, self-attention in transformers.

When a model processes your input, every token attends to every other token in the sequence. The attention score between two tokens determines how much the model's representation of token A is influenced by token B.

Conceptually:

attention_score(query_A, key_B) = dot_product(Q_A, K_B) / sqrt(d_k)

Where Q_A is the query vector for token A and K_B is the key vector for token B. Higher scores mean more influence.

The critical insight: attention is computed across the full context window, but the scores are not uniform. Recent tokens tend to have higher attention scores because the model has been trained on sequences where recent context is more relevant. Earlier tokens receive lower attention scores and contribute less to the model's representations.

This is not a bug — it is the correct inductive bias for most language tasks. In normal text, what just happened is more relevant than what happened 10,000 tokens ago.

Why quality degrades before the hard limit

The practical consequence of attention distribution is that context quality is not binary. It does not work at 100% until 200,000 tokens and then fail. It degrades gradually as the context grows.

The degradation pattern:

0-40% capacity: Full quality. The model attends effectively to everything in the context. Constraints established early are respected. Earlier decisions are remembered.

40-60% capacity: Slight degradation. The model may occasionally miss a constraint from the beginning of the conversation. Earlier architectural decisions get less weight.

60-80% capacity: Noticeable degradation. Constraints from early in long conversations get ignored. The model may contradict earlier decisions. Code quality tends to decrease.

80-100% capacity: Significant degradation. The model effectively operates on a shorter effective context than the full window. Early content is present but not meaningfully attended to.

The practical takeaway: the useful context limit for complex tasks is roughly 60% of the advertised context window.

The "lost in the middle" problem

Research on long-context LLMs has identified a specific failure mode called "lost in the middle" — models perform well on information at the beginning and end of long contexts, but poorly on information in the middle.

This has a direct implication for how you structure long prompts:

Less effective:

[System instructions - beginning]
[Background context - middle, long]
[Specific question - end]

More effective:

[System instructions - beginning]
[Specific question - near beginning]
[Relevant background context - end]

Information you want the model to attend to most strongly should be either at the very beginning or the very end of the context — not buried in the middle of a long conversation.

Practical implications for developers

Restart conversations deliberately

The most effective strategy for long-running tasks is to restart conversations when the context reaches 60% capacity.

Ask for a summary before restarting:

Summarize the key decisions, constraints, current state, and 
any important code patterns established in this conversation. 
Keep it under 300 words.

Start a new conversation with that summary. You preserve everything that matters and reset the context overhead. The summary typically condenses 30,000-50,000 tokens of conversation into 1,000-2,000 tokens.

Front-load the constraints that matter most

If you have critical constraints — "never use class components," "all functions must be async," "this code runs on Node 18" — put them at the beginning of the system prompt AND repeat them at the end of your most recent message.

The repetition sounds redundant but is effective: the constraint appears at two positions that both receive strong attention — the beginning and the most recent content.

Use separate conversations for separate problems

Every unrelated message adds tokens without adding relevant context. If you are debugging three separate functions in a single conversation, each exchange about function A is reducing the effective context available for function C.

One problem per conversation is more token-efficient and produces higher quality responses.

Monitor context usage in real time

Most platforms hide context window usage from users. TokenPulse injects a live bar above the input box on Claude, ChatGPT, Gemini, DeepSeek and Grok showing your current context window percentage. When it hits 60%, that is the signal to summarize and restart.

The 1M token context window — does it actually help?

Gemini 1.5 Pro's 1,000,000 token context window is a genuine technical achievement. But the attention degradation problem scales with context length — a 1M token context window does not give you 5x the effective context of a 200k window.

For tasks where you need to search across a very large codebase or document set, large context windows genuinely help. For typical development tasks — debugging, code review, architecture discussion — the effective range of 200k is more than sufficient when used well.

The practical benchmark: if you are regularly hitting context limits on Claude or GPT-4o, the problem is almost always conversation management, not context window size.

Summary

Context windows contain the entire conversation history sent on every request
Attention mechanisms weight recent tokens more heavily than earlier ones
Quality degrades noticeably at 60-70% capacity, not just at the hard limit
Information in the middle of long contexts gets less attention than beginning and end
Restart at 60% with a summary, not when the model tells you the conversation is too long
One problem per conversation is more efficient than long multi-topic sessions

What Are AI Tokens and Why Should Developers Care?

Anoop Kumar — Fri, 21 Aug 2026 15:04:31 +0000

If you have used Claude, ChatGPT, or any large language model API, you have encountered tokens. They show up in pricing pages, rate limit errors, and API documentation. But most explanations of tokens are either too vague ("roughly 4 characters") or too academic (subword tokenization, BPE encoding).

This article explains tokens at the level a developer actually needs — what they are, why they matter for the things you care about, and how to think about them when building with AI APIs.

What a token actually is

A token is the basic unit of text that a language model processes. It is not a character, and it is not a word — it sits somewhere in between.

Modern language models use a technique called Byte Pair Encoding (BPE) to split text into tokens. The tokenizer learns which character sequences appear frequently together and groups them into single tokens. Common words become single tokens. Rare words get split into multiple tokens. Punctuation, spaces, and special characters all become tokens too.

Some examples using GPT-4's tokenizer:

"hello"        → 1 token
"tokenization" → 3 tokens  (token + ization split further)
"ChatGPT"      → 3 tokens
" the"         → 1 token   (space included)
"2026"         → 1 token
"\n"           → 1 token

The 4-characters-per-token approximation works because English text averages roughly 4 characters per token across typical writing. Code, which has more symbols and shorter identifiers, tends to be closer to 3 characters per token. Dense technical writing can be 5 or more.

Why tokens matter for developers

1. Cost

Every major AI API charges by token. Claude Sonnet 4 costs $3.00 per million input tokens and $15.00 per million output tokens. GPT-4o costs $2.50 per million input and $10.00 per million output.

A typical back-and-forth debugging session might consume 10,000-50,000 tokens total. At Claude Sonnet pricing, that is $0.03 to $0.15 per session — which sounds trivial until you have a team of 10 developers each running 5 sessions per day.

The output token cost matters more than most developers realize. When you ask Claude to write a 500-line function, you are paying 5x more per token for the output than the input. Prompts that ask for verbose explanations cost significantly more than prompts that ask for concise code.

2. Context window limits

Every model has a context window — the maximum number of tokens it can process in a single request, including both input and output. Claude Sonnet 4 has a 200,000 token context window. GPT-4o has 128,000. Gemini 1.5 Pro has 1,000,000.

This means a 200,000 token context window is approximately:

800,000 characters of text
150,000 words
About 600 pages of a typical book
A large codebase (though not an entire monorepo) In practice, you hit the context limit much faster than these numbers suggest because:

Your system prompt consumes tokens
Every message in the conversation accumulates
Every response the model generates adds to the total
Large code pastes consume tokens proportional to their length A single paste of a 500-line file is roughly 25,000-30,000 tokens — 12-15% of Claude's context window in one shot.

3. Rate limits

AI providers enforce rate limits in tokens per unit time, not requests per unit time. Claude has a 5-hour session limit and a 7-day weekly limit, both measured in token consumption. OpenAI's API has tokens-per-minute (TPM) and tokens-per-day (TPD) limits.

This means two things developers often get wrong:

Message count is not what matters. Sending 100 short messages consumes far fewer tokens than sending 10 messages with large code pastes. Rate limits are about total token consumption, not request frequency.

Output tokens count too. If you ask a model for a very long, detailed response, you are consuming output tokens against your rate limit just as much as the input.

How to estimate tokens without calling the API

For quick estimates, the 4-characters-per-token approximation is sufficient:

function estimateTokens(text) {
  return Math.ceil(text.length / 4)
}

// Examples:
estimateTokens("Hello, world!")          // → 4
estimateTokens("function authenticate(") // → 6
estimateTokens(largeCodeFile)            // → roughly accurate ±10%

For precise counts, use the official tokenizers:

OpenAI: tiktoken library (Python and JavaScript)

import { encoding_for_model } from 'tiktoken'
const enc = encoding_for_model('gpt-4o')
const tokens = enc.encode(text).length

Anthropic: Claude's API has a count_tokens endpoint:

const response = await anthropic.messages.countTokens({
  model: 'claude-sonnet-4-5',
  messages: [{ role: 'user', content: text }]
})
console.log(response.input_tokens)

For browser extensions and client-side tools that cannot call the API, client-side estimation with the ±8% approximation is the practical approach — it is accurate enough for showing users whether they are at 20% or 80% of their context window.

Practical token optimization for developers

Be specific, not verbose. "Refactor this function to handle null values" consumes far fewer tokens than a paragraph explaining the same thing. Models understand concise instructions.

Paste targeted context, not entire files. Instead of pasting a 500-line file, paste the specific function plus 10-15 lines of surrounding context. You get equally good answers and use a fraction of the tokens.

Summarize long conversations. After a long debugging session, ask the model to summarize key decisions and findings in under 200 words, start a new conversation with that summary, and continue. You preserve context that matters and reset token overhead.

Use faster models for iteration. If you are iterating on boilerplate, formatting, or simple transformations, use Haiku or GPT-4o mini. They consume the same rate limit quota but cost 10-20x less per token. Reserve the premium models for complex reasoning.

Monitor your actual usage. Most developers significantly underestimate their token consumption because the interface hides it. Tools like TokenPulse show real-time token counts, context window percentage, and estimated cost directly in the browser — no API key required.

The key numbers to remember

Model	Context Window	Input Cost / 1M	Output Cost / 1M
Claude Sonnet 4	200k tokens	$3.00	$15.00
Claude Opus 4	200k tokens	$15.00	$75.00
GPT-4o	128k tokens	$2.50	$10.00
Gemini 2.0 Flash	1M tokens	$0.10	$0.40
DeepSeek V3	128k tokens	$0.27	$1.10

Understanding tokens is the foundation of working effectively with AI APIs. Once you internalize that everything is measured in tokens — cost, limits, context — the behavior of these systems becomes much more predictable.