DEV Community: SPANLENS

Helicone was acquired by Mintlify. Here is a migration checklist if you are moving off.

SPANLENS — Wed, 08 Jul 2026 03:42:36 +0000

On March 3, 2026, Helicone announced it was joining Mintlify. If you run Helicone in production, the practical question is not whether the acquisition is good or bad. It is what changes for you, and whether you need to do anything about it.

Here is the honest version, and a checklist if you decide to move.

What actually changed

Helicone's founders joined Mintlify, and active feature development on the standalone product has wound down. The team has said security patches, bug fixes, and new model support will continue. New features and roadmap work are the part that stopped.

For a lot of teams that is fine for a while. A logging proxy that already works does not stop working the day the roadmap freezes. But two situations make people start looking. You are on Helicone Cloud and you want to know the plan is still moving forward, not just being kept alive. Or you self-host and you were counting on features that are now unlikely to ship.

Helicone was one of three observability tools acquired in a few months. ClickHouse bought Langfuse and Cisco bought Galileo in the same window. If you are picking a replacement, that pattern is worth keeping in mind. More on that at the end.

Do you even need to move right now

Worth saying plainly. If you self-host Helicone, you are happy with it, and you do not need anything new from it, there is no fire. The code keeps running. You can migrate on your own schedule instead of someone else's.

The case for moving sooner is stronger if you are on the hosted product, if you depend on the gateway staying current with new providers and models, or if you would rather switch once now than watch and decide later. If that is you, the rest of this is for you.

The migration checklist

Helicone and Spanlens are both drop in proxies, so the mechanical part is short. The work is mostly finding every place your code sets a base URL and updating headers.

1. Swap the base URL

This is the one required change.

// Before, Helicone
const openai = new OpenAI({
  baseURL: 'https://ai-gateway.helicone.ai',
  apiKey: process.env.HELICONE_API_KEY,
})

// After, Spanlens
const openai = new OpenAI({
  baseURL: 'https://server.spanlens.io/proxy/openai/v1',
  apiKey: process.env.SPANLENS_API_KEY,
})

2. Map your metadata headers

If you tagged requests with user, session, or prompt identifiers, those headers have direct equivalents.

Helicone header	Spanlens header
`Helicone-User-Id`	`x-spanlens-user`
`Helicone-Session-Id`	`x-spanlens-session`
`Helicone-Prompt-Id`	`x-spanlens-prompt-version`

3. Move your provider key server side

Helicone had you pass both keys. Spanlens registers your real OpenAI or Anthropic key once in the dashboard and keeps it server side, so your application only ever sends the Spanlens key. One less secret in your request path.

4. Update the other providers

If you route Anthropic, Gemini, or Azure through the proxy too, each has its own base URL. The full list and the copy paste version of every step is in the Helicone to Spanlens migration guide.

5. Verify before you delete anything

Send one day of real traffic through the new path and reconcile the cost and token numbers against your provider invoice before you tear out Helicone. Keep both running in parallel for a day. This is the step people skip and regret.

Before you pick any replacement, not just this one

Three observability tools got acquired in one quarter. Whatever you move to, the useful question is what happens to your data if the same thing happens again.

Two things worth checking on any tool you evaluate. Can you export your raw trace data without a support ticket. And can you self host the whole thing, so that a change of ownership is your decision to act on rather than something that happens to you. Spanlens is MIT licensed and self hostable with one Docker command for exactly this reason, but the point stands no matter what you choose. Own your exit before you need it.

If you want to compare the two side by side first, the Spanlens vs Helicone breakdown goes through pricing, hosting, and instrumentation in detail.

Spanlens is open source and free to start. You can try it in one line or read the migration guide first. If it saves you time, a star on GitHub helps other people find it.

Tracking token usage across OpenAI, Anthropic, and Gemini: every streaming gotcha I hit

SPANLENS — Sat, 20 Jun 2026 09:33:40 +0000

OpenAI, Anthropic, and Gemini each report token usage differently, and it stops being trivia the moment you track LLM cost. I build Spanlens, an open-source LLM observability tool that sits in front of all three as a proxy and records every call with its model, latency, tokens, and cost. To do the cost part I read the token usage back out of every response, including the streaming ones.

I assumed the three providers would report usage in roughly the same way. They send the same kind of data, after all: input tokens, output tokens, maybe a cached count. How different could it be.

Pretty different, it turns out. Here is the whole thing in one table, then each gotcha in detail with the real parser code from the repo.

Provider	Where usage lives (streaming)	Cache accounting	Field names
OpenAI	final chunk, needs `stream_options: { include_usage: true }`	`prompt_tokens` includes cache	`prompt_tokens` / `completion_tokens`
Anthropic	split across `message_start` + `message_delta`	`input_tokens` excludes cache, so add it	`input_tokens` / `output_tokens`
Gemini	`usageMetadata`, two stream formats	not applicable	`promptTokenCount` / `candidatesTokenCount`

Gotcha 1: the usage numbers live in different places in the stream

For a non-streaming call this is boring. Every provider hands you a usage object on the response body and you read it. Streaming is where it gets weird, because the token counts are not in the content chunks. They show up somewhere else, and "somewhere else" is different for each provider.

OpenAI puts the usage in a final chunk, after all the content, right before [DONE]. You only get it if you ask for it with stream_options: { include_usage: true }. Miss that flag and you stream the whole response and end up with no usage at all.

export function parseOpenAIStreamChunk(line: string): Partial<ParsedUsage> | null {
  if (!line.startsWith('data: ')) return null
  const data = line.slice(6).trim()
  if (data === '[DONE]') return null
  const json = JSON.parse(data)
  const usage = json.usage
  if (!usage) return null   // most chunks land here; only the last one has usage
  return {
    promptTokens: usage.prompt_tokens ?? 0,
    completionTokens: usage.completion_tokens ?? 0,
    totalTokens: usage.total_tokens ?? 0,
  }
}

Anthropic splits it across two different events. The input tokens come early, in message_start. The output tokens come at the end, in message_delta. If you only listen for one event, half your number is missing.

// input side: arrives in message_start
if (json.type === 'message_start') {
  const usage = json.message?.usage
  return { promptTokens: usage.input_tokens /* + cache, see below */ }
}

// output side: arrives later, in message_delta
if (json.type === 'message_delta') {
  return { completionTokens: json.usage?.output_tokens ?? 0 }
}

So for OpenAI I keep the last chunk, and for Anthropic I have to stitch together the first event and a later one. Two providers, two mental models, already.

Gotcha 2: cached tokens are counted with opposite conventions

This is the one that can quietly corrupt your cost numbers, so it is worth slowing down on.

Both OpenAI and Anthropic support prompt caching, and both report a cached-token count. The trap is what the "input tokens" number means relative to that cached count.

On OpenAI, prompt_tokens already includes the cached tokens. The cached count is a subset of it. If you want the uncached portion you subtract.

On Anthropic, input_tokens is the uncached portion only. The cached tokens are reported separately and are not in that number. To get the real total you add them up.

Same idea, opposite math. Here is how I normalize Anthropic so that my promptTokens column always means "total input including cache" no matter which provider it came from:

const inputTokens = usage.input_tokens ?? 0
const cacheRead = usage.cache_read_input_tokens ?? 0
const cacheWrite = usage.cache_creation_input_tokens ?? 0
const promptTokens = inputTokens + cacheRead + cacheWrite   // Anthropic: add

And OpenAI, where the cached count is already inside prompt_tokens:

const promptTokens = usage.prompt_tokens ?? 0               // OpenAI: already total
const cacheReadTokens = usage.prompt_tokens_details?.cached_tokens ?? 0  // subset

If you write one function and feed both providers through it without thinking about this, you do not get an error. You get a cost number that is wrong by the size of the cache, and cache hits are exactly the high-volume calls where the error is largest. Wrong financial data that never throws is the worst kind of bug, so I now treat the cache convention as a per-provider fact I have to look up rather than guess.

Gotcha 3: Gemini streams in two different shapes

OpenAI and Anthropic both stream server-sent events, lines that start with data:. Gemini can do that too, but only if you append ?alt=sse to the URL. Without it, the default streamGenerateContent endpoint streams a single giant JSON array, one big [ ... ] delivered character by character.

So a Gemini stream parser has to handle both. Mine tries SSE first, then falls back to parsing the buffer as a JSON array, then falls back again to scanning line by line for anything that looks like a chunk:

// 1. SSE form ("data: {json}")
for (const line of lines) {
  if (line.startsWith('data: ')) appendTextFromGeminiChunk(line.slice(6).trim(), parts)
}
// 2. default form: one JSON array streamed char by char
const joined = lines.join('\n').trim()
if (parts.length === 0 && joined.startsWith('[')) {
  for (const item of JSON.parse(joined)) appendTextFromGeminiChunk(JSON.stringify(item), parts)
}

The field names are different too. OpenAI gives you prompt_tokens and completion_tokens. Gemini gives you promptTokenCount and candidatesTokenCount inside a usageMetadata object. None of it lines up, so the normalizer earns its keep.

A smaller one: the tier you asked for is not always the tier you got

All three providers can report a service tier (default, flex, priority, and so on), and the cost depends on it. The thing to know is that the tier in the response is the tier they actually served, which is not always the one you requested. OpenAI can downgrade a priority request to default under load, and that downgrade only shows up in the response. So I always trust the served tier from the response over whatever the request asked for, because that is what you are billed on.

Gemini also reports the tier with inconsistent casing, sometimes a plain flex, sometimes a ..._FLEX screaming-snake constant, so that needed its own small coercion step.

What I would tell past me

If you are normalizing usage across providers, do not write the shared function first. Write one parser per provider, get each one right against real responses, and only then collapse them behind a common shape. The differences are not cosmetic. Where the number lives, whether cache is included, and what the field is called all change per provider, and a single early abstraction hides exactly the parts that differ.

The other lesson is to assert on cost-bearing numbers loudly. Type errors get caught on the first request in dev. A token count that is off by the cache size ships silently and shows up as a billing discrepancy weeks later. That asymmetry is worth a test.

All of this lives in apps/server/src/parsers/ in the repo if you want to see the full versions, including the streaming reassembly and the tier handling I trimmed here.

This is the second gotcha writeup from building Spanlens. The first was on moving LLM logs from Postgres to ClickHouse, if you are weighing that migration.

Spanlens is open source (MIT). If you want the token, cost, and latency of every LLM call logged with a one-line baseURL swap, you can try it free or self-host it with one Docker command.

If this saved you a debugging session, a star on GitHub genuinely helps other people find it.

What gotchas have you hit normalizing usage or cost across providers? I would like to hear them.

5 gotchas I hit moving LLM logs from Postgres to ClickHouse

SPANLENS — Wed, 27 May 2026 13:00:47 +0000

The problem

LLM request logs are the one table that outgrows Postgres fast. I am building Spanlens, an open-source LLM observability platform: every call to OpenAI, Anthropic, or Gemini gets recorded with its model, latency, tokens, cost, and full request and response body. At low traffic on Supabase Postgres this was fine, but I could already see a few signs that this specific table would not stay fine for long.

The requests table will dominate the DB at any meaningful scale. Every other table is bounded by org or project counts, but requests grows with every API call.
Date-range scans on created_at are the dashboard's primary query pattern, and Postgres cannot compress the JSON body columns well, so these queries would get slower as the table grew.
Daily backup size scales with table size, and backing up large JSON bodies in Postgres looked like a clear path to pain.
Aggregation queries like "top expensive prompts in 7 days" that work in milliseconds on a small table are not going to scale that way.

So I migrated to ClickHouse early, before it became a fire. This post is what I wish I had known before the migration, with 5 gotchas that bit me and the fallback design I built so I would not lose data while finding them.

If you want to see the full implementation in context, Spanlens is open source on GitHub under MIT. For where it sits in the broader LLM observability landscape, the honest comparison vs Langfuse walks through the architectural differences side by side.

The 5 gotchas at a glance

#	Gotcha	Fix
1	DateTime64 rejects the `Z` suffix	`toClickhouseTimestamp()` helper
2	Every number comes back as a string	coerce with `Number()` at the boundary
3	No `ilike`, no `NULLS LAST`	`positionCaseInsensitive`, explicit ordering
4	New column means deploy ordering matters	run the migration before the deploy
5	Every failed insert is data loss	Postgres fallback queue + replay

Why ClickHouse and not something else

Here is the shortlist I evaluated.

TimescaleDB was too close to Postgres and didn't really solve the storage cost concern.
DuckDB is great for embedded analytics but I wanted a managed service.
BigQuery's pricing model felt unpredictable.
ClickHouse Cloud's Development tier was cheap to start with, and columnar compression handles JSON body blobs well.

The decision driver was the workload shape. It is insert-heavy, append-only, and almost all reads are time-range scans with one or two equality filters like organization_id and model. ClickHouse is built for exactly this.

The new architecture

I kept Postgres (Supabase) for everything relational with RLS, including orgs, projects, members, API keys, prompts, alerts, and billing. ClickHouse holds only one table called requests and it is the one that will grow.

Web/Server  -> ClickHouse  (write path, fire-and-forget INSERT)
            |  on failure
            v
            -> Supabase requests_fallback  (durable queue)
            |  every 5 minutes (cron)
            v
            -> replay back into ClickHouse

This is the outbox-ish pattern. I will come back to why.

Reads always go through a single helper that injects scope and retention.

// apps/server/src/lib/requests-query.ts
export async function requestsScope(
  orgId: string,
  opts: { ignoreRetention?: boolean } = {},
) {
  const plan = await getOrgPlan(orgId)            // 'free' | 'pro' | 'team'
  const retentionDays = LOG_RETENTION_DAYS[plan]  // 14 | 90 | 365
  const whereScope = opts.ignoreRetention
    ? 'organization_id = {orgId:UUID}'
    : 'organization_id = {orgId:UUID} ' +
      'AND created_at >= now() - INTERVAL {retentionDays:UInt32} DAY'
  return { whereScope, scopeParams: { orgId, retentionDays }, plan }
}

Direct getClickhouse().query() calls outside this helper are something I avoid. Multi-tenant data leaks are the worst kind of bug, and ClickHouse has no row-level security, so the discipline has to live in the query layer.

Gotcha 1. DateTime64 rejects the Z suffix

JavaScript's new Date().toISOString() returns 2026-05-16T11:49:23.749Z. ClickHouse expects 2026-05-16 11:49:23.749 with a space instead of T and no trailing Z. Insert with the JS default and you get this.

Code: 27. DB::Exception: Cannot parse input: expected " " but got "T"

I added a tiny helper and banned .toISOString() in any code path that writes to ClickHouse.

export function toClickhouseTimestamp(d: Date): string {
  return d.toISOString().replace('T', ' ').replace('Z', '');
}

Reading is the reverse. If you parse a DateTime64 back into a JS Date, you need to put T and Z back.

const date = new Date(row.created_at.replace(' ', 'T') + 'Z');

Gotcha 2. Every number comes back as a string

This one is sneaky because the bug is silent.

ClickHouse's JSONEachRow format returns all numeric columns including Decimal(18, 8), UInt64, and Int32 as JSON strings, not numbers.

{ "cost_usd": "0.00012345", "tokens": "421" }

Then your innocent r.cost_usd + 1 does string concatenation, so "0.00012345" + 1 === "0.000123451". No error. Just wrong.

The fix is mechanical but you have to do it everywhere.

const rows = (await ch.query(...)).map(r => ({
  ...r,
  cost_usd: Number(r.cost_usd ?? 0),
  tokens:   Number(r.tokens ?? 0),
}));

I now treat this as a strict boundary. The helper that wraps ch.query() does the coercion before anything else touches the rows.

Gotcha 3. No ilike, no nulls last

ClickHouse does not have Postgres's ILIKE. If your previous code looked like .ilike('model', '%gpt%'), the direct rewrite is this.

WHERE positionCaseInsensitive(model, 'gpt') > 0

NULLS LAST ordering is a similar story. It has to be explicit instead of implicit.

ORDER BY cost_usd DESC NULLS LAST

Both are easy fixes once you know about them, but they are easy to miss if you rewrite queries in a hurry because they are syntactic differences, not semantic ones. The previous query keeps "working" with no SQL error but silently changes behavior.

Gotcha 4. New column means deploy ordering matters

ClickHouse rejects unknown fields in JSONEachRow inserts by default.

Code: 117. Unknown field 'truncated'

If you add a new column in your INSERT code, ship that code, and your production ClickHouse cluster has not run the migration yet, every insert from that pod fails until the migration lands. Streaming and non-streaming, all of it.

I addressed this with two patterns.

First, run the migration before deploying the code. ALTER TABLE ... ADD COLUMN IF NOT EXISTS lands first, then the code that writes the column ships.

Second, I added a belt-and-suspenders setting.

// lib/clickhouse.ts
const ch = createClient({
  url,
  clickhouse_settings: { input_format_skip_unknown_fields: 1 },
});

This silently skips unknown columns instead of failing the insert. It rescues you from deployment-order mistakes, but it also hides typos. Pair it with a smoke-test that asserts new fields actually make it in.

Gotcha 5. Every failed insert is data loss unless you queue

ClickHouse has good uptime, but "good" isn't 100%. And LLM logs are write-heavy in a way that means every dropped insert is a dollar of cost data you will never get back.

I added a Postgres-backed fallback queue.

export async function logRequestAsync(data: RequestLogData) {
  try {
    await getClickhouse().insert({
      table: 'requests',
      format: 'JSONEachRow',
      values: [data],
    })
  } catch (err) {
    // ClickHouse rejecting or unreachable, queue to durable Postgres backup
    const message = err instanceof Error ? err.message : String(err)
    await supabaseAdmin.from('requests_fallback').insert({
      payload: data,
      organization_id: data.organization_id,
      last_error: message.slice(0, 500),
    })
  }
}

A cron job at /cron/replay-fallback runs every 5 minutes and drains the queue.

export async function replayFallbackQueue() {
  // 1. Drop poisoned rows first (7+ days old or 100+ retries) in one DELETE
  //    so the limited batch budget goes to fresh entries.
  const expiry = new Date(Date.now() - 7 * 86400_000).toISOString()
  await supabaseAdmin.from('requests_fallback')
    .delete()
    .or(`created_at.lt.${expiry},retry_count.gte.100`)

  // 2. Pull the next 50 in FIFO order so a long outage drains in arrival order.
  const { data: rows } = await supabaseAdmin.from('requests_fallback')
    .select('id, payload')
    .order('created_at', { ascending: true })
    .limit(50)

  if (!rows?.length) return

  // 3. One bulk INSERT for the whole batch instead of N round trips.
  //    ClickHouse JSONEachRow accepts arrays trivially.
  try {
    await getClickhouse().insert({
      table: 'requests',
      format: 'JSONEachRow',
      values: rows.map(r => r.payload),
    })
    // 4. Success ??delete the entire batch in one query.
    await supabaseAdmin.from('requests_fallback')
      .delete()
      .in('id', rows.map(r => r.id))
  } catch {
    // Leave them in the queue. Next cron run picks them up and the
    // expiry step above eventually drops them if they stay stuck.
  }
}

Two design choices worth calling out.

I use Postgres for the queue instead of Redis. I already had Postgres for transactional state. Adding Redis just for a recovery queue would be a separate failure domain. Postgres going down is so much worse than ClickHouse going down that pairing them is fine, because if Postgres is also down then the whole product is down and that is a more obvious incident than missing logs.

I do not deduplicate. The requests table has no unique constraint, so a race could insert a row twice. I accepted this trade-off because duplicate logs are a UI cosmetic problem, not a billing problem. If I ever sell on "exactly-once logging" I will redesign this.

Where I am now

It is still early days at Spanlens, so I do not have dramatic before-and-after benchmarks to share. What I can say qualitatively is this.

The architecture feels right. Reads are fast, writes are fire-and-forget, and the storage shape no longer fights the workload shape.
The fallback queue exists and tests pass, but I have not yet had a real long outage to stress it. I would rather have it and not need it than the other way around.
I am no longer worried about the requests table dominating my Postgres backups or query times as Spanlens grows.

The single biggest lesson is to invest in the safety net before you need it.

The live dashboard at spanlens.io is the surface running on top of the ClickHouse-backed queries described above. The request log, traces, and cost views are all reading through the requestsScope helper.

What I would do differently

A few things in hindsight.

Set input_format_skip_unknown_fields: 1 from day one. It is a small change that buys real resilience against deploy-order mistakes.

Add a synthetic write and read smoke test on every deploy that confirms a known row makes it through. This catches typos that the unknown-field setting now hides.

Make helpers the API boundary loudly. Direct ch.query() calls are the easiest way to introduce a multi-tenant leak. I plan to enforce this with CI lint rules.

Wrapping up

If you are building anything write-heavy with time-range queries on top, whether that is observability, audit logs, event streams, or IoT telemetry, ClickHouse is worth considering early instead of waiting until Postgres becomes a fire. The footguns are real but they all show up early, and the runtime characteristics after that are very pleasant.

If you are evaluating LLM observability options in 2026, the Spanlens alternatives hub lists Langfuse, Helicone, LangSmith, Braintrust, and Arize Phoenix side by side with the same honesty as this post.

Spanlens, the open-source LLM observability platform I built this for, is on GitHub at github.com/spanlens/Spanlens under MIT. The migration helpers quoted here are all under apps/server/src/lib/. If you have done a similar migration or are staring at one, I would love to hear what gotchas hit you in the comments.

The next writeup in this series is on token usage across OpenAI, Anthropic, and Gemini, every streaming gotcha I hit.