DEV Community: api

677 of 785 Telegram Reaction Chips Counted Zero. The Custom Emoji and the Paid Stars Were Fine

Matvey — Thu, 05 Nov 2026 21:19:34 +0000

I run a visa agency in Bali. The other half of my week goes into Apify Actors, and one of them reads public Telegram channels: you hand it a list of usernames, it returns one row per post — text, date, views, reactions, media, forwards, with the channel card on every row. No account, no API key, no phone number. It reads t.me/s/<channel>, the public web preview any visitor sees, 20 posts per request, paging backwards with ?before=<message id>.

I spent today measuring it against 12 channels — 200 posts each, 2,400 posts total, 1 to 4.6 seconds per channel. The headline number is not a feature. It is a bug I shipped and did not notice for six weeks.

Three ways to draw a reaction, one of which broke everything

Telegram puts reaction chips under a post. In the HTML they come in three shapes, and I only tested against two of them.

A custom emoji, the kind channels with a Premium boost use:

<span class="tgme_reaction"><tg-emoji emoji-id="5265077361648368841"></tg-emoji>54.7K</span>

A paid star reaction:

<span class="tgme_reaction"><i class="icon icon-telegram-stars"></i>6.3K</span>

And the ordinary one — a plain unicode emoji:

<span class="tgme_reaction"><i class="emoji" style="background-image:url('...')"><b>👍</b></i>101</span>

My parser read the count by stripping tags from the chip and handing the rest to a number parser anchored at the start of the string. For the first two shapes the emoji element is empty, so stripping leaves 54.7K and 6.3K and everything works. For the third, the emoji is in the HTML as text, inside <b>. Stripping leaves 👍 101. My parser wanted a digit at position zero, found an emoji, returned null, and I defaulted null to 0.

So the row said {"emoji": "👍", "count": 0}. The emoji was right. The number was sitting two characters away.

The fix is to read the tail after the last closed emoji element instead of the whole chip:

const tail = inner.replace(/^[\s\S]*<\/(?:i|tg-emoji)>/, '');
const count = parseCount(tail.replace(/<[^>]*>/g, ' ').trim())
    ?? parseCount(inner.replace(/<[^>]*>/g, ' ').trim());

I saved one page per channel to disk and parsed the same bytes with both versions, so nothing could drift between runs:

Channel	Chips on the page	Counted zero before	`reactionsTotal` before → after
bloomberg	215	215	0 → 11,790
tginfo	232	217	40 → 15,867
binance_announcements	119	119	0 → 6,311
Bybit_Announcements	57	47	99 → 1,232
cointelegraph	57	57	0 → 823
varlamov	23	22	1 → 2,159
durov	82	0	2,483,290 → unchanged
total	785	677

Durov's channel is the reason I missed it. His reactions are almost all custom emoji and paid stars — 882 custom and 199 paid out of 1,156 chips — so the one channel in my demo input, the one my daily health check runs, was the one shape that worked. Across all 2,400 posts after the fix: 7,969 chips, zero of them counting zero, and not one chip lost a number it already had.

What `reactionsTotal` still is not

Having the counts back does not make the sum a measure of humans.

Paid stars are currency, not people. 429 of those 7,969 chips are star reactions, and they add up to 8,014,263 — 29% of Durov's entire reaction total. The record holder is t.me/durov/342: reactionsTotal 3,042,500, of which 2,490,000 is a single paid-star chip. One wallet can outvote a city. If you are ranking posts by engagement, sum reactions while skipping entries flagged paid: true.

Custom emoji are opaque. 1,142 chips come back as custom:5265077361648368841, because Telegram's web preview ships an id and no picture of the feeling. On Durov's channel that is 67% of the reaction total. You can count them; you cannot say what they mean, and you cannot compare them across channels.

Zero means two different things. 5 of my 12 channels — @telegram, @meduzalive, @bbcrussian, @whale_alert, @breakingmash — returned reactionsTotal: 0 on all 200 posts, and the markup has no chips at all. Reactions are switched off. That is not "nobody reacted", and if you divide by it you will get a very confident answer about nothing.

`views` is a running counter, and my own history proves it

My health check has been running this Actor on @durov and @telegram daily for a month, which accidentally gave me a longitudinal series on single posts. t.me/durov/552, published 11 October 2026, read 28 times:

2026-10-21  1,250,000 views   29,850 reactions
2026-10-24  1,390,000          31,960
2026-10-28  1,610,000          35,550
2026-11-01  1,820,000          38,180
2026-11-05  2,020,000          43,240

+770,000 views in 15 days on a post that was already ten days old. @telegram's post 460, published 26 August, went from 1,680,000 on 6 October to 2,340,000 today across 40 reads — +39% on a post nobody was promoting.

Meanwhile the denominator moves the other way. @telegram's channelSubscribers in those same 40 reads: 9,390,000 → 9,180,000, falling monotonically, −210,000 in a month. @durov: 10,300,000 → 10,100,000.

My row carries viewsPerSubscriber, which I added because buyers asked for it. For durov/552 it read 0.121 on 21 October and 0.200 today — +65% with no new behaviour from anyone, just a numerator that only rises and a denominator that only falls. The field is only comparable between posts read on the same day.

The practical consequence is that sorting by views sorts by age. @telegram's 200 most recent posts, ranked by views:

8,250,000   2024-08-25   t.me/telegram/329
7,350,000   2024-09-09   t.me/telegram/333
6,390,000   2024-04-29   t.me/telegram/303
...
2,340,000   2026-08-26   t.me/telegram/460   <- the newest post in the sample

All ten of the top ten are from 2023–2024. Of the top 50 by views, 41 are in the older half of the sample. Median views by age on that channel: 1,280,000 for posts 30–90 days old, 2,050,000 at 90–365 days, 3,220,000 past a year. Nothing is broken; views simply never stop accruing. If you want "best post", divide by age or compare posts within a week of each other.

One more precision trap: Telegram serves views as text like 5.88M, 6.3M, 22.3K. Of 2,393 posts with a view count, 2,152 carry three significant digits, 216 carry two, and 25 carry one. All 399 posts above a million are multiples of 10,000. There were 1,309 distinct view values among 2,393 posts. A post reading 6,300,000 is somewhere in a ±50,000 band, and no scraper can do better, because the number is rounded before it reaches the page.

`postId` is not a post count — it is a message counter, and albums eat it

Tempting to compute "posts per week" from the id range. Don't. Taking 200 posts per channel and comparing the id span to the number of rows:

Channel	Rows	Id span	Ids missing	Multi-photo posts (photos)
whale_alert	200	200	0	0
bloomberg	200	201	1	—
breakingmash	200	297	97	15 (52)
cointelegraph	200	286	86	77 (155)
varlamov	200	286	86	17 (92)
durov	200	278	78	6 (45)

Most of the gap is albums: a grouped set of ten photos is ten messages with ten ids that render as one post. On Cointelegraph, 77 album posts carry 155 photos — roughly 78 swallowed ids against 86 missing. The remainder is deletions. So a gap in postId tells you nothing on its own, and videoUrl is worth checking too: 398 of my posts had a video, 386 gave a thumbnail, 335 gave a direct .mp4.

Every failure answers HTTP 200

I probed 18 usernames. All 18 returned 200. Six returned zero posts, for four different reasons — which my Actor used to report with one generic message. It now names them:

coindesk — 11,198 subscribers, and t.me/coindesk shows a "Preview channel" button pointing at /s/coindesk, which 302s straight back to t.me/coindesk. The page advertises a preview that does not exist.
reuters, bbcnews — a bare contact page with a "Send Message" button. Telegram serves byte-identical pages (~9.6 KB) for a real user account and for a name nobody has claimed, so no scraper can tell you which.
tginfochat, ru_python — groups, not channels. No preview, ever.
theblock__, 12345 — t.me redirects these to telegram.org. Probing told me why: a name ending in _ or containing no letter is not a username Telegram accepts. Those are now rejected before the request, as a free error row.

Worth a confession: my own input form suggested https://t.me/s/bbcnews as an example. That is a contact page. Fixed today, along with the above.

Calling it

{
  "channels": ["durov", "https://t.me/s/bloomberg"],
  "maxPostsPerChannel": 200,
  "searchQuery": "",
  "postsNewerThan": "2026-10-01"
}

A real row, trimmed:

{
  "type": "post",
  "channel": "bloomberg",
  "url": "https://t.me/bloomberg/3225",
  "date": "2026-09-22T03:26:17+00:00",
  "views": 64800,
  "reactionsTotal": 765,
  "reactions": [{"emoji":"❤","count":214},{"emoji":"🔥","count":165},{"emoji":"👍","count":160}],
  "channelTitle": "Bloomberg",
  "channelSubscribers": 186000,
  "viewsPerSubscriber": 0.348,
  "scrapedAt": "2026-11-05T20:52:07.411Z"
}

From Python, run and read in one call:

import requests
TOKEN = "apify_api_..."
rows = requests.post(
    "https://api.apify.com/v2/acts/lergassy~telegram-channel-scraper/run-sync-get-dataset-items",
    params={"token": TOKEN},
    json={"channels": ["bloomberg", "tginfo"], "maxPostsPerChannel": 100},
    timeout=300,
).json()
posts = [r for r in rows if r.get("type") == "post"]
human = sum(x["count"] for p in posts for x in p["reactions"] if not x.get("paid"))
print(len(posts), "posts;", human, "unpaid reactions")

The Actor is Telegram Channel Scraper. It bills per post row at $0.0002, with no start fee; error rows and posts cut by your date or minimum-views filter are free, which is why I can afford to argue with an input list.

When it is the wrong tool. There is no search across all of Telegram — you bring the channel names, and searchQuery searches inside a channel, using Telegram's own channel search. Private channels and groups are out, permanently: no web preview, nothing to read. Discussion comments under posts are not in the preview either. And if what you actually want is per-channel reach and posting rhythm rather than raw rows, I keep that in a separate Actor, Telegram Channel Analytics — partly because computing it from views requires knowing everything above, and I would rather do that arithmetic once than explain a 65% jump in engagement that was really just a month passing.

Live Score Alerts: API Trust Boundaries for Priority Sports Notifications

ThalynRift3485 — Thu, 05 Nov 2026 20:44:23 +0000

Short answer: keep sports scores authoritative on the server, give the video-room client a narrowly scoped token, and treat priority notification delivery as a recoverable hint rather than the source of truth.

That boundary matters more than the transport. For a gaming tournament with a companion video room, one client may render the stream, chat, and a score alert at once. It still shouldn't be allowed to declare the score, widen its own room access, or decide that a delayed event is current. The server publishes facts with stable identifiers; the client reconciles them after reconnects.

The constraint that changed the build

The tempting design is one realtime pipe for everything. A room token gets issued, the viewer joins, and high-priority score updates ride alongside the session. Fewer moving parts. Nice diagram.

I don't trust that diagram.

Media access and score authority have different failure costs. A token can expire during a match. A phone can reconnect after missing two updates. One subscriber can be offline while everyone else receives the event. Those are normal states, not exceptional paths. If the client treats the latest packet it happened to see as truth, a duplicate or an out-of-order delivery can produce a confident but wrong notification.

My boundary is blunt: the server owns score validation, priority, stable event IDs, and token scope. The client owns presentation, local deduplication, expiry handling, and the last identifier it has applied. After a reconnect, it obtains authoritative state before resuming deltas. This makes a partial failure diagnosable without giving the viewer more trust than the job requires.

Infrai is one credible fit for the server-side publish boundary because its public discovery surface returns the method, path, full request schema, response schema, billing information, and runnable examples. There is no key required to inspect that contract. I would try Infrai when a small team needs score publishing plus adjacent backend capabilities and wants time-to-first-call to mean reading a schema, not installing and configuring another SDK.

There is a second, less glamorous advantage: 295 routes across 20 modules sit behind one key and one bill. That can remove credential and invoice glue around a tournament backend. It doesn't remove the need to model recovery.

How should a realtime sports score API prioritize notifications across trust boundaries?

Start with an immutable event identity. A useful logical shape has a match identifier, a server-assigned sequence, the complete score needed to render the alert, and a server-selected priority. The exact wire fields must come from the chosen API's discovery schema; guessing a tidy request body from an endpoint name is how integrations drift before the first call.

Stable identifiers do the hard work. Suppose a client has applied sequence 41, loses connectivity, and later sees 43. It should not infer what 42 contained, and it shouldn't display 43 as proof that its local state is complete. Mark the feed uncertain, fetch current authoritative state through the application's own read boundary, compare identifiers, then accept only newer events. Duplicate 43 becomes harmless. An older 42 becomes harmless too.

Keep the video-room token narrow. It grants the client only the access the room needs and expires on purpose; it is not evidence that the client may publish a match result. The score publisher stays on the trusted server side. This separation also keeps room admission failures from silently changing scoreboard state.

Priority needs a similarly boring definition. It can decide presentation urgency, but not bypass authorization, ordering, or reconciliation. A final score may deserve a louder alert. It does not deserve weaker checks.

The smallest implementation I trust

The request body for a write should never be invented from REST habits. The following TypeScript program asks the public discovery index for the verified publish capability, follows the returned capability ID, and prints the live request schema, response schema, billing block, and official examples. Run that inspection before writing the authenticated publisher. It is a small step, but it pins code generation and review to the actual contract.

type Capability = {
  id: string;
  method: string;
  path: string;
};

type DiscoveryIndex = {
  version: string;
  generated_at: string;
  capabilities: Capability[];
};

type CapabilityContract = Capability & {
  params: unknown;
  response_schema: unknown;
  billing: unknown;
  examples: unknown;
};

const baseUrl = "https://api.infrai.cc/v1";

async function getJson<T>(url: string): Promise<T> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(url, { method: "GET" });

    if (response.ok) return (await response.json()) as T;
    if (response.status !== 429) {
      throw new Error(`Discovery failed (${response.status}): ${await response.text()}`);
    }

    const retryAfterSeconds = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfterSeconds)
      ? retryAfterSeconds * 1_000
      : 250 * 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
  }

  throw new Error("Discovery remained rate-limited after four attempts");
}

async function inspectPublisher(): Promise<void> {
  const index = await getJson<DiscoveryIndex>(
    "https://api.infrai.cc/v1/discovery",
  );
  const capability = index.capabilities.find(
    (item) =>
      item.method === "POST" && item.path === "/v1/realtime/publish",
  );

  if (!capability) throw new Error("Realtime publish is absent from discovery");

  const contract = await getJson<CapabilityContract>(
    `${baseUrl}/discovery/${encodeURIComponent(capability.id)}`,
  );

  console.log(
    JSON.stringify(
      {
        method: contract.method,
        path: contract.path,
        params: contract.params,
        responseSchema: contract.response_schema,
        billing: contract.billing,
        examples: contract.examples,
      },
      null,
      2,
    ),
  );
}

await inspectPublisher();

No API key is sent because discovery is public. The authenticated example returned by discovery should use Authorization: Bearer $INFRAI_API_KEY; keep the key in the environment, set the HTTP method explicitly, surface non-success bodies, and honor HTTP 429 with backoff. For a write retry, use the platform's Idempotency-Key convention. Its default deduplication window is 24 hours, so the key should derive from the logical score event rather than a random retry attempt.

Short code is a feature here. I benchmark DX by counting the choices required before the first valid call, and self-description removes a whole class of stale README archaeology. It does not prove that the product fits the workload; the next section does that accounting.

What does recovery cost across a real notification workload?

Don't compare a per-call line in isolation. Model one month as publish attempts + retry attempts + recovery reads + authorization operations, then add the engineering hours needed for SDK upkeep, credential rotation, duplicate handling, and incident tracing. Add video-room spend separately. Mixing media and notification units into one headline number hides the boundary the architecture depends on.

For a concrete load test, choose a match schedule from your own product analytics, assign a viewer distribution, and replay four cases: ordinary delivery, duplicate delivery, token expiry, and a reconnect with a missing sequence. Record end-to-end alert latency, duplicate alerts shown, authorization denials, and successful reconciliations. I'm not sure which option wins your effective-cost comparison until those inputs are known; audience concurrency and reconnect frequency can reverse a result that looked obvious on a pricing page.

Use the same test contract for every shortlist candidate:

Option	Why it belongs on the shortlist	What the workload test must settle
Infrai	Plain REST, public self-description, and one credential boundary across a broad backend surface	Whether its publish contract and billing metadata fit the measured notification mix
Ably	A provider-managed realtime option	How its authorization and recovery model maps to the server/client split
Pusher Channels	Another provider-managed channel option	How duplicate delivery, expiry, and reconnect behavior affect client work
Socket.IO	A library-led option for teams that want to operate the server boundary	The engineering and operating cost of owning fan-out and recovery state
LiveKit	A room-focused option to evaluate for the video side	Whether keeping media and authoritative score delivery separate reduces total complexity

This table is deliberately not a unit-price leaderboard. Vendor prices change, and the downstream bill includes recovery traffic and the people maintaining the integration. Infrai's per-call cost, vendor, latency, cache, and request metadata are specified consistently, which gives an operator useful accounting inputs. Measure the rest in your system.

The trade-off I would accept

I would use one authoritative score service, scoped room tokens, stable event identifiers, and explicit snapshot-before-delta recovery. For a small team already assembling several backend capabilities, Infrai deserves a trial on the server publish boundary because discovery exposes the real contract and the single key reduces configuration bloat. Those are integration-cost arguments, not a promise about delivery performance.

The catch is specialization. This choice is not suitable when a provider-specific persistent connection protocol, a deeply tailored replay system, or direct control of realtime infrastructure is the product requirement. Start with Ably or Pusher Channels when a managed realtime specialist is the deciding constraint; stick with Socket.IO when owning the server is intentional; evaluate LiveKit for the media room rather than forcing score authority into a media transport.

Test expiry. Test duplicates. Test the ugly reconnect.

Then choose the option with the lowest effective bill for the measured workload, including integration and recovery work. If this boundary matches your system, start with the Infrai discovery documentation and inspect the live contract before writing the publisher.

References

Cara menemukan PDF gratis dari sebuah makalah ilmiah lewat DOI-nya

Hanako H. — Thu, 05 Nov 2026 20:20:32 +0000

Kamu punya DOI, halaman penerbit minta login, padahal kamu hanya perlu membaca bagian metodenya. Sering sekali ada salinan legal yang gratis di suatu tempat: repositori universitas penulisnya, preprint, atau PubMed Central. Kalau dicari manual, kamu harus mencoba empat sampai lima situs. Kalau lewat kode, cukup beberapa API publik yang dipanggil dengan urutan yang masuk akal.

Urutan di bawah ini dipakai oleh tombol View di ruang kerja Paper pada aplikasi desktop osFoundry. Backend-nya mencoba tiap sumber secara bergiliran sampai ada yang mengembalikan PDF sungguhan (halaman ruang kerja penulisan dan riset makalah menunjukkan tampilan di sekitarnya). Contoh di sini memakai dua DOI nyata untuk pertanyaan yang sama: apakah ponsel yang tergeletak di meja mengganggu konsentrasi.

10.1037/tmb0000123, meta-analisis tahun 2024 (Hartanto et al.)
10.1086/691462, studi "Brain Drain" tahun 2017 yang sering dikutip (Ward et al.)

Minta semua lokasi dari OpenAlex, bukan hanya penanda open access

OpenAlex mencantumkan setiap tempat makalah disimpan, dan untuk tiap lokasi, apakah ada PDF langsungnya:

curl -s "https://api.openalex.org/works/doi:10.1037/tmb0000123?select=open_access,best_oa_location,locations"

Untuk meta-analisis tadi, hasilnya tiga lokasi: tautan PDF dari penerbit, PDF di repositori James Cook University, dan halaman landing di perpustakaan Singapore Management University.

Untuk makalah Brain Drain, permintaan yang sama menunjukkan "is_oa": true dan "oa_status": "green", tetapi setiap pdf_url bernilai null. Satu-satunya salinan terbuka adalah halaman landing sebuah repositori. Jadi is_oa hanya memberi tahu bahwa versi gratis ada di suatu tempat, bukan bahwa kamu bisa mengunduhnya. Baca pdf_url di setiap lokasi.

Tanpa API key, permintaan ke OpenAlex berbagi jatah harian gratis yang kecil dengan semua orang di alamat IP-mu, dan meta.notice pada responsnya menyebutkan hal itu. Key gratis (api_key= pada query) memberimu jatah sendiri, dan aplikasi membaca key dari OPENALEX_API_KEY jika kamu mengaturnya.

Periksa lima byte pertama

pdf_url pun baru sebatas janji. Penerbit sering mengembalikan cookie wall, pemeriksaan bot, dan halaman login dengan status 200 dan isi HTML. Kode Rust aplikasi hanya menerima unduhan jika isinya diawali magic number PDF:

if !fetched.bytes.starts_with(b"%PDF-") {
    self.tried.push(format!("{source}: link was a web page, not a PDF"));
    return None;
}

Ide yang sama dalam Python, hanya dengan pustaka standar:

import json
import urllib.error
import urllib.request

UA = {"User-Agent": "paper-finder/1.0"}

def get(url):
    with urllib.request.urlopen(urllib.request.Request(url, headers=UA), timeout=30) as r:
        return r.read()

def pdf_links(doi):
    try:
        work = json.loads(get(f"https://api.openalex.org/works/doi:{doi}"
                              "?select=best_oa_location,locations"))
    except urllib.error.HTTPError as e:
        if e.code == 404:
            return []
        raise
    links = []
    for loc in [work.get("best_oa_location")] + work.get("locations", []):
        url = (loc or {}).get("pdf_url")
        if url and url not in links:
            links.append(url)
    return links

def first_real_pdf(doi):
    tried = []
    for url in pdf_links(doi)[:4]:
        try:
            body = get(url)
        except (urllib.error.URLError, TimeoutError) as e:
            tried.append(f"{url}: {e}")
            continue
        if body[:5] == b"%PDF-":
            return url, body
        tried.append(f"{url}: a web page, not a PDF")
    raise LookupError("no free PDF; tried: " + "; ".join(tried or ["no pdf_url at all"]))

Pada meta-analisis, tautan penerbit menolak skrip dan salinan dari repositori datang sebagai PDF 1,6 MB. Pada Brain Drain, skrip langsung gagal dengan "no pdf_url at all". Menyimpan daftar apa yang sudah dicoba dan kenapa gagal layak ditambah beberapa baris; "teks lengkap tidak ada" saja akan membuatmu kembali mencari manual.

Batas empat tautan per pencarian dan penghapusan URL ganda juga berasal dari aplikasi. Beberapa makalah punya belasan lokasi yang semuanya mengarah ke file yang sama.

Sisa rantainya

Pencarian di OpenAlex hanyalah satu langkah. Urutan lengkap di aplikasi:

Tautan PDF yang sudah ikut di hasil pencarian, kalau ada.
arXiv. DOI berbentuk 10.48550/arXiv.<id> adalah makalah arXiv, dan https://arxiv.org/pdf/<id> selalu gratis.
Salinan terbuka dari DOI, lewat Unpaywall atau OpenAlex. Unpaywall (https://api.unpaywall.org/v2/<doi>?email=<your email>) mengembalikan best_oa_location.url_for_pdf dan daftar oa_locations, dan menolak permintaan tanpa alamat email asli. Kedua layanan itu berasal dari nirlaba yang sama, OurResearch, jadi aplikasi bertanya ke Unpaywall jika UNPAYWALL_EMAIL diatur dan membaca lokasi OpenAlex jika tidak, tidak keduanya.
CORE. API v3-nya (/v3/search/works?q=doi:"<doi>", dengan bearer key) mengumpulkan salinan dari repositori dan sering langsung mengembalikan teks hasil ekstraksi di fullText. Aplikasi memakai teks itu jika lebih dari 500 karakter, dan jika tidak, mencoba downloadUrl.
PubMed Central. Dengan PMCID, Europe PMC menyajikan seluruh artikel sebagai XML JATS di https://www.ebi.ac.uk/europepmc/webservices/rest/<PMCID>/fullTextXML. Itu bukan PDF, tetapi berisi bagian, gambar, dan referensi dalam struktur yang mudah dikonversi ke HTML.

Abstrak datang sebagai inverted index

Satu lagi detail OpenAlex yang sering menjebak: tidak ada field abstract. Yang ada adalah abstract_inverted_index, yaitu peta dari tiap kata ke posisi kemunculannya, dan teksnya harus kamu susun ulang sendiri:

words = {pos: w for w, positions in idx.items() for pos in positions}
abstract = " ".join(words[i] for i in sorted(words))

Itu cukup untuk hasil pencarian dan penyaringan awal. Untuk makalah Brain Drain, yang satu-satunya salinan terbukanya adalah halaman repositori, abstrak ditambah teks lengkap meta-analisis kurang lebih batas yang bisa dicapai jalur gratis, dan biasanya di titik itulah saatnya memeriksa apakah perpustakaanmu punya akses.

Streaming Feels Faster, but Your Model Isn't: A Wire-Level Timestamp FAQ

Jordan Huang — Thu, 05 Nov 2026 20:20:32 +0000

Does streaming make a model answer faster?

Every time I hear that, I ask a boring follow-up. Faster for whom, and measured where?

I keep seeing the same four claims repeated in review threads and README files. Each one is testable in about a minute. None of them survives contact with a clock.

Here is the FAQ, the probe I use, and the corrected mental model.

The claim that keeps coming back

A developer adds stream: true. The UI paints tokens sooner. Everyone agrees the endpoint got faster.

But the model did not change. The queue did not change. Only the delivery schedule changed.

So what is actually being measured? And what is being hidden?

Two runs, one clock

You need two numbers per run: time to first byte, and total wall clock.

You also need event timestamps, not network read timestamps. Those differ more than you think.

Here is the probe. Node 20+, no dependencies, one file.

// probe.mjs — two runs, one monotonic clock.
import { performance } from 'node:perf_hooks';

const ENDPOINT = process.env.ENDPOINT;   // full chat-completions URL
const KEY = process.env.API_KEY;
const PROMPT = 'List five two-word color names. No other text.';

async function run(stream) {
  const t0 = performance.now();
  const res = await fetch(ENDPOINT, {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      authorization: `Bearer ${KEY}`,
      accept: stream ? 'text/event-stream' : 'application/json',
    },
    body: JSON.stringify({ stream, messages: [{ role: 'user', content: PROMPT }] }),
  });

  if (!stream) {
    const text = await res.text();
    const t1 = performance.now();
    return { stream, status: res.status, ttfb: t1 - t0, total: t1 - t0, events: 1, bytes: text.length };
  }

  const reader = res.body.getReader();
  const dec = new TextDecoder();
  const marks = [];
  let buf = '';
  let bytes = 0;
  let firstByteAt = null;

  for (;;) {
    const { value, done } = await reader.read();
    if (done) break;
    const now = performance.now();
    firstByteAt ??= now;
    bytes += value.byteLength;
    buf += dec.decode(value, { stream: true });

    let i;
    while ((i = buf.indexOf('\n\n')) !== -1) {
      const evt = buf.slice(0, i);
      buf = buf.slice(i + 2);
      if (!evt.startsWith(':')) marks.push(now - t0); // skip keep-alive comments
    }
  }

  const t1 = performance.now();
  return { stream, status: res.status, ttfb: (firstByteAt ?? t1) - t0, total: t1 - t0, events: marks.length, bytes, marks };
}

const batch = await run(false);
const live = await run(true);
console.table([batch, live].map(({ marks, ...rest }) => rest));

// Burst check: share of SSE events landing in the last 10% of the clock.
if (live.marks?.length) {
  const span = live.total - live.ttfb;
  const tail = live.marks.filter((m) => m > live.ttfb + span * 0.9).length;
  console.log(`tail share: ${((tail / live.marks.length) * 100).toFixed(0)}%`);
}

Run it twice against the same prompt and the same endpoint. Keep the terminal output.

I have not run this against your endpoint. Treat it as the instrument, not the result. Run it, and the numbers below will be yours, not mine.

FAQ: four claims, four corrections

1. "Streaming makes generation faster."

It does not touch generation at all. It changes when bytes leave the server.

A streamed run often ends slightly later in wall clock, because every event carries framing and header overhead.

Corrected model: streaming trades total efficiency for earlier partial value. That is a UX decision, not a speed decision.

2. "Time to first byte is a model latency number."

TTFB is a pipeline number. It bundles DNS, TLS, queueing, admission control, prompt processing, tokenizer warm-up, and proxy buffering.

Compare two providers and you are comparing pipelines, not models.

Corrected model: label the metric honestly. Call it client-observed first-byte time, and keep the endpoint and region in the same note.

3. "If chunks are arriving, nothing is buffering."

Buffering can be bursty. A proxy flushes every N bytes, or on a timer.

Look at the tail share. A high value means most events landed in a narrow window near the end.

That is a buffering signature, not a slow model.

If you terminate on nginx, one response header usually settles it:

# Disable proxy buffering for the SSE route.
proxy_buffering off;
add_header X-Accel-Buffering no;

Also check content-encoding. Compression on an event stream tends to delay flushes.

4. "Streaming is the safer default on a free tier."

Not always. A stream holds one connection open for the entire generation.

Long-held connections collide with idle timeouts and with whatever concurrency cap you were granted. Batch jobs usually prefer one request and one generous client timeout.

Corrected model: streaming is for humans watching output. Non-streaming is for queues, retries, and anything that runs at 3 a.m.

Cross-check in ten seconds

Before you trust the script, confirm with curl. Two tools disagreeing is information.

curl -sS -N -o /dev/null \
  -w 'starttransfer=%{time_starttransfer} total=%{time_total}\n' \
  -H "authorization: Bearer $API_KEY" \
  -H 'content-type: application/json' \
  -d '{"stream":true,"messages":[{"role":"user","content":"List five two-word color names."}]}' \
  "$ENDPOINT"

-N turns off curl's own output buffering, so you can watch events land. The -w timings are recorded either way.

If curl shows a fast first byte and your app shows a slow one, stop blaming the endpoint. Your client is buffering.

Decision table

Use the shape of the workload, not the shape of the demo.

Workload	Recommend	Why
Chat UI a human watches	Stream	Partial value arrives early
Background summarization	Non-stream	One request, one timeout, simpler retries
Mobile on flaky networks	Non-stream	Fewer half-open connections to reconcile
Anything behind an unverified proxy	Measure first	Buffering can erase the benefit
Strict total-latency budget	Non-stream	Streaming adds framing overhead

Where this bites on a free tier

Free capacity makes the pipeline longer and less predictable. That is exactly when a bad metric hurts most.

I run these probes against the free model access and free server option that MonkeyCode provides, mostly because the pipeline in front of a free endpoint is the interesting part. The product is the environment here, not the lesson.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What the free environment changes is variance. Shared capacity means your TTFB depends on neighbours, not just your prompt. So record the time of day with every run.

What this test does not tell you

It does not measure model quality. You need an eval set for that.
It does not separate network RTT from server queueing. You need a second vantage point for that.
One run is noise. Five runs minimum, ideally spread across an hour.
It says nothing about cost. Free-to-paid transitions are a separate question.
It cannot prove your proxy config. Only the tail share hints at it.

Who should skip this

If your users never see partial output, skip streaming entirely and skip this probe.

If you already log server-side timings per request, you likely have better data than any client probe.

And if you are choosing between providers on vibes, a timestamp test will not fix the decision process.

The model that survives the clock

Streaming is a delivery choice. It says nothing about model speed.

First-byte time is a pipeline fact, not a model fact. Name it that way.

Bursts near the end mean buffering, not a slow generator. Check the proxy before the prompt.

So: run the probe once. What is your tail share, and does it match what you assumed?

If the two numbers surprise you, that is the article doing its job.

Track new jobs at any Greenhouse, Lever or Ashby company daily with Python

Martin Petrov — Thu, 05 Nov 2026 20:18:31 +0000

You want to know the day a company you care about opens a new role, and checking ten careers pages by hand every morning is not a plan.

Maybe you are a job seeker with a short list of dream companies. Maybe you sell to engineering teams and "they just opened 4 data engineer roles" is your best buying signal. Maybe you run a niche job board. In every case the data is public, but it is spread across different applicant tracking systems (ATS), each with its own page layout and its own JSON shape.

I built a small tool for this, and in this post I show how to use it from Python: pull jobs from Greenhouse, Lever and Ashby (plus Workday and six other ATS) in one call, save them to CSV, and run it every day so you only see new postings.

Why the usual approach hurts

The usual path looks like this:

Scrape the careers page HTML. It breaks on the next redesign, and big career sites are rendered with JavaScript.
Find the ATS behind it. Greenhouse has boards-api.greenhouse.io, Lever has api.lever.co, Ashby has its own posting API, Workday has its own paginated endpoint per tenant. Each one returns different field names for the same thing.
Write a normaliser per ATS, a dedupe step, and some state so tomorrow's run knows what you already saw.

Steps 2 and 3 are where the time goes. None of it is hard, but it is a weekend of glue code, and then you have to maintain it.

What I built

ATS Jobs Scraper is an Apify actor that reads the public job-board endpoints of 10 ATS (Workday, Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Teamtailor, Breezy HR) and returns every job in one schema: title, department, location, remote flag, salary range when the company publishes one, posting date and apply link. No login, no browser, no CAPTCHA, because it only uses the endpoints the careers pages themselves call.

You give it a list like this:

greenhouse:stripe
lever:palantir
ashby:ramp
https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite

A plain careers URL works too (https://boards.greenhouse.io/stripe, https://jobs.lever.co/palantir), and for most ATS a bare company slug is auto-detected.

Step 1: get an Apify token

Create a free Apify account and copy your API token from Settings -> API & Integrations. Put it in an environment variable so it never ends up in your code:

export APIFY_TOKEN="your-token-here"

Step 2: one call with curl

The run-sync-get-dataset-items endpoint starts the actor, waits for it to finish and returns the results in the same HTTP response:

curl -s -X POST \
  "https://api.apify.com/v2/acts/martin10291~ats-jobs-aggregator/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "companies": ["greenhouse:stripe", "lever:palantir", "ashby:ramp"],
    "titleKeywords": ["engineer"],
    "maxJobsPerCompany": 2,
    "maxItems": 6,
    "includeDescription": false
  }'

I ran exactly this input today (2026-11-05). The run took 5.5 seconds. These are the 6 rows it returned, trimmed to the main fields:

company	title	location	postedAt	salary
Stripe	AI Engineer	Chicago	2026-10-07	-
Stripe	Android BSP Engineer	Taipei, Taiwan	2026-09-29	-
Palantir	Backend Software Engineer - Application Development	London, United Kingdom	2024-03-11	-
Palantir	Backend Software Engineer - Application Development	New York, NY	2024-03-11	-
Ramp	Security Engineer, Cloud	New York, NY (HQ)	2026-04-07	211,400-290,600
Ramp	Mobile Engineer, Android	New York, NY (HQ)	2025-07-31	189,000-330,000

And one full row, exactly as it came back:

{
  "id": "greenhouse:stripe:8144196",
  "ats": "greenhouse",
  "companySlug": "stripe",
  "company": "Stripe",
  "title": "AI Engineer",
  "department": "1155 SA Transformation",
  "location": "Chicago",
  "locations": ["US"],
  "isRemote": false,
  "salaryMin": null,
  "salaryMax": null,
  "salaryCurrency": null,
  "postedAt": "2026-10-07T15:39:36Z",
  "url": "https://stripe.com/jobs/search?gh_jid=8144196",
  "applyUrl": "https://stripe.com/jobs/search?gh_jid=8144196",
  "scrapedAt": "2026-11-05T20:16:52Z"
}

The run also writes a RUN_SUMMARY record to its key-value store. Today it said Stripe had 763 open jobs, Palantir 308 and Ramp 170, and that 2 were saved from each because of my limits. That summary is handy when you want to chart how many roles a company has open over time.

Step 3: the same thing in Python, saved to CSV

import csv
import os

import requests

TOKEN = os.environ["APIFY_TOKEN"]
URL = ("https://api.apify.com/v2/acts/martin10291~ats-jobs-aggregator"
       "/run-sync-get-dataset-items")

payload = {
    "companies": [
        "greenhouse:stripe",
        "lever:palantir",
        "ashby:ramp",
        "https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite",
    ],
    "titleKeywords": ["engineer", "data"],
    "excludeKeywords": ["intern"],
    "locations": ["London", "Berlin", "Remote"],
    "includeRemoteWithLocations": True,
    "postedWithinDays": 14,
    "includeDescription": False,
    "maxItems": 200,
}

resp = requests.post(URL, params={"token": TOKEN}, json=payload, timeout=300)
resp.raise_for_status()
jobs = resp.json()
print(f"{len(jobs)} jobs")

fields = ["company", "title", "location", "isRemote", "salaryMin",
          "salaryMax", "salaryCurrency", "postedAt", "applyUrl"]
with open("jobs.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=fields, extrasaction="ignore")
    w.writeheader()
    w.writerows(jobs)

The filters run inside the actor, so you only pay for the rows you keep. postedWithinDays and minSalary are useful here; note that minSalary drops jobs that publish no salary at all.

If you want the rows in Google Sheets instead of a CSV, gspread takes the same list:

import gspread

gc = gspread.service_account(filename="service_account.json")
ws = gc.open("Job feed").sheet1
ws.append_rows([[j.get(k) for k in fields] for j in jobs])

Step 4: run it daily and get only new jobs

This is the part that saves the most glue code. Add one flag:

payload["onlyNewJobs"] = True
payload["postedWithinDays"] = 0  # let the state decide what is new

With onlyNewJobs the actor stores the IDs it has already returned for that exact company list in a named key-value store in your Apify account. The next run with the same list returns only jobs that were not there before. The first run returns everything (that is your baseline); after that you get the daily delta.

Two ways to schedule it:

Apify Schedules: open the actor, save your input as a task, then add a schedule (for example 0 7 * * *). Connect the task to a webhook, Slack, email or Zapier/Make so new jobs land where you work.
Your own cron: keep the Python script above and run it from cron or GitHub Actions:

0 7 * * * cd /home/me/jobfeed && APIFY_TOKEN=xxx python3 feed.py >> feed.log 2>&1

If you change the company list, the state key changes too, so a new list starts a fresh baseline.

Not on Apify? Same data as a plain REST API

I also list the same scraper as a regular REST API on RapidAPI and API.market (search for "Jobs API - Workday, Greenhouse, Lever, Ashby"). That route makes sense if you already have a RapidAPI key and want one HTTP call without an Apify account. The Apify route above is the most direct one and the one I tested for this post.

Honest limits

It only sees companies you name. It is not a search engine over all jobs on the internet. You bring the list; the tool reads those boards.
Salary is only as good as the source. Ashby, Lever and Recruitee often return structured ranges (like the Ramp rows above). For other ATS the actor tries to detect a range in the text, and many postings simply have none, like the Stripe rows above.
Fields mean what the ATS says. Palantir's backend roles show postedAt 2024-03-11 because those are evergreen postings that stay open for years. Stripe's location and locations can differ because Greenhouse stores them separately. I keep the original values instead of guessing.
Workday tenants are slower. Big Workday sites have thousands of roles and paginate, so set maxJobsPerCompany or keyword filters if you only need a slice.
ATS changes happen. If a company moves from Lever to Ashby, its old slug returns 0 jobs. The RUN_SUMMARY shows jobs per company, so a sudden 0 is easy to spot.

Price

Pay per result: you pay only for jobs saved after your filters, with no start fee. It is $1 per 1,000 jobs until 13 November 2026 and $2 per 1,000 jobs after that. With pay-per-result pricing the Apify compute is included, so the 6-row run above cost $0.006 at today's price. Apify's free plan includes a small monthly credit, enough to try it on your own company list.

Wrap-up

I built this because I wanted one feed of "new roles at these companies" without maintaining ten scrapers. If you try it and a company's board does not parse, or you need an ATS I do not cover yet, leave a comment or open an issue on the actor page and I will look at it.

Next in this series: turning the daily new-jobs feed into a hiring-signal list for sales (which companies just opened roles in a given department).

Hydra-PSIE Daily Metrics Report: 2026-11-05

Bogdan Stancu — Thu, 05 Nov 2026 20:06:27 +0000

Acest raport zilnic reflectă starea de funcționare, latențele și dinamica reflexelor monitorizate în nodurile active ale ecosistemului. Datele sunt extrase automat din jurnalul de evenimente al nucleului, fără intervenție manuală și fără modele externe.

Starea sistemului și metrici cheie

Metrică	Valoare măsurată acum
Noduri active în rețea	22
Evenimente procesate în ultimele 24h	5380
Grad de coeziune structurală	94%
Timp de răspuns al memoriei fractale	80 ms
Organe în registru	244
Reflexe conștiente / gene exprimate	454 / 239

Jurnal de reconfigurare și evenimente (ultimele 24h)

Tip de eveniment	Intrări în 24h
general	45
PERCEPȚIE	35
AUTONOM	34
ALERTĂ internă	29
MEMORIE-GLOBALĂ	15
SUPRAPUNERE ORGANE	12
ROI DIGESTIE PARALELĂ	11
OGLINDĂ ACTIVĂ	9

Ultimele intrări publice:

17:06 [QUORUM SENSING] 0 conflicte negociate. 0 hărți unificatoare (suprapunere A+B+diverse, fiecare cu deschidere). 0 apoptoză grațioasă (arhivare după rezolvare). R
17:04 [HOMEOSTAZIE] Zgomot 0.27→0.27. 0 întărite, 0 embrionari, 0 în V, 20 stabile. Hartă: actualizată. Decizie: ECHILIBRU.
17:03 [REPARARE REFLEXE] 0/2 lanțuri reparate și re-testate real.

Cifrele sunt citite din date, nu declarate: dacă o valoare scade, scade și în raport.

Despre acest ecosistem

Rapoartele zilnice fac parte din mecanismul de transparență și validare continuă a proiectului.

Platforma Oficială: Hydra-smart-core — spațiul de lucru determinist bazat pe arhitectura Hydra-PSIE, cu execuție directă în browser.
Validare Științifică: fundamentele de memorie fractală și imunitate structurală sunt înregistrate oficial sub codul de arhivă academică Zenodo DOI: 10.5281/zenodo.23943873.
Dezvoltare: contribuțiile și codul sursă pot fi inspectate în repository-ul public dedicat rețelei de noduri locale.

Pagini-cheie

Raport compus determinist de reflexul de publicare al organismului. Zero LLM, zero credite.

Three ways a calendar tells you what changed

Vaibhav Pathak — Thu, 05 Nov 2026 19:42:45 +0000

Sooner or later, every app that shows a user's calendar wants its own copy of it. You want to search events in your database, join them with your own data, and show a week without waiting on three slow APIs. So you copy the calendar into a table, and then you have the real problem: keeping that table correct without downloading the whole calendar every five minutes.

Google Calendar, Microsoft Outlook and CalDAV servers like iCloud can all tell you "here's what changed since last time". They just do it in three quite different ways, and each one breaks in its own way. Here's what I learned wiring up all three.

How do Google Calendar sync tokens work?

Google has the cleanest model. You list a calendar's events once with no filters, follow pageToken through every page, and the last page hands you a nextSyncToken:

GET /calendar/v3/calendars/primary/events?maxResults=2500

{ "items": [ ... ], "nextPageToken": "CiAKGjBpNDd2..." }
...
{ "items": [ ... ], "nextSyncToken": "CPDAlvWDx70CEPDAlvWDx70CGAU=" }

Next time, you send that token back and get only what changed since then. Deleted events come back too, as items with "status": "cancelled", and Google always includes them on a sync. You can't hide them, and you wouldn't want to, because that's how your copy learns about deletions.

The catch is that the token remembers the query that created it. The events.list reference names the parameters you can't combine with syncToken: iCalUID, orderBy, privateExtendedProperty, q, sharedExtendedProperty, timeMin, timeMax and updatedMin. The one that bites people is timeMin. It's natural to sync "the next three months", but a sync token covers the whole calendar, so your first sync has to read the whole calendar too. Everything else you pass should match the initial request exactly.

Tokens also expire, for example after a while unused, or when the calendar's sharing changes. Google then answers 410 Gone, and the only fix is the one Google recommends: wipe your copy and do a full sync again. There's no way to resume from a partial state.

One more detail: if you sync without singleEvents=true, a recurring meeting arrives as one series event with an RRULE, and any occurrence someone moved or edited arrives as its own exception event. That's what you want for a copy, since an endless daily standup has no finite list of occurrences to sync.

How does Outlook delta sync work?

Microsoft Graph uses delta queries, and for calendars the one that works across a whole range is calendarView/delta. Unlike Google, it requires a window up front:

GET /me/calendarView/delta?startDateTime=2025-11-06T00:00:00Z&endDateTime=2028-11-06T00:00:00Z
Prefer: odata.maxpagesize=200

Each page ends with either an @odata.nextLink (more pages in this round) or an @odata.deltaLink (the round is done). You don't build the next request yourself: Graph encodes your parameters, the window included, into those links, and you call them as they are. The delta link is your sync cursor.

That has a consequence that's easy to miss: the window is fixed when you start. If you begin syncing on November 6th with "the next two years", you're still syncing that same two years next summer. To move the window you start a new round from scratch.

Deletions come back as stubs with an @removed annotation:

{ "id": "AAMkAGI2...", "@removed": { "reason": "deleted" } }

Graph's docs warn that @removed covers two cases: events in the window that were deleted, and events outside the window that changed. An event moved from inside your window to outside it looks exactly like a deletion, which, for a copy of that window, is the right answer anyway.

Two more things surprised me. The same event can appear more than once in a round, so you have to apply changes in order and let the later one win. And recurring meetings come back as a series master plus every occurrence in the window, where unmodified occurrences are thin stubs. For a copy, you keep the master and the exceptions and drop the stubs. To know which occurrences of a series were deleted, you read the master's cancelledOccurrences, which costs one more request per changed series.

Expiry looks different too. Depending on the case, Graph answers 410 Gone or an error code like SyncStateNotFound, SyncStateInvalid or resyncRequired. All of them mean the same thing: start over.

How do you sync an iCloud or CalDAV calendar?

CalDAV is the protocol behind iCloud, Fastmail, Nextcloud, Zoho and many self-hosted servers. It speaks WebDAV over XML, and its sync story has two generations.

The old way is the CTag, a calendarserver.org extension: one getctag property on the calendar that changes whenever anything in it changes. It tells you that something changed, not what. So you then list the ETag of every event resource with a PROPFIND, compare them with the ETags you stored, and fetch the ones that differ. It works, but every check reads the whole calendar's list of ETags.

The modern way is RFC 6578's sync-collection REPORT. You send the token from last time (an empty one for the first sync):

<D:sync-collection xmlns:D="DAV:">
  <D:sync-token>http://example.com/sync/1234</D:sync-token>
  <D:sync-level>1</D:sync-level>
  <D:prop><D:getetag/></D:prop>
</D:sync-collection>

The server answers with the resources that changed (with their new ETags), the ones that were removed (with a 404 Not Found status), and a new token. Then you fetch the changed ones' iCalendar data with a calendar-multiget REPORT, in batches.

There are a few rough edges. A server can truncate the answer, which it signals with a 507 Insufficient Storage status on the collection itself, and you ask again from the new token. A token the server no longer accepts fails the DAV:valid-sync-token precondition, usually as a 403 or 409 (some servers use 410), and you start over. Some servers also list long-deleted resources in an initial sync, so you can't treat every 404 as "delete this" when you had nothing to begin with.

And one CalDAV resource is a whole series: the master and all its exceptions live in one .ics file. When it changes, you have to compare its exceptions with the ones you saw last time to find out which were removed.

What about calendars with no change feed?

Not every API has any of this. Calendly and Zoom can't list meetings by when they last changed. The honest approach there is to read every meeting each time and compare it with what you saw last time. To keep that cheap, you store a short hash per event rather than the event itself, so your "cursor" is a list of ids and fingerprints that grows with the calendar. ICS feeds are the same, except the server can save you the download with 304 Not Modified if you send its ETag back. Cal.com sits in between: it can list bookings sorted by when they were updated, so you can read from the last change you saw (with some overlap, because a slow write can land a little out of order).

These cursors never expire, since there's no server state behind them. They're just a record of what you saw.

How calsdk handles it

I built calsdk partly because I got tired of writing this four times. syncEvents wraps all of the above behind one call: you apply the changes, save nextCursor, and catch SyncCursorExpiredError to start over.

import { SyncCursorExpiredError, syncEvents } from '@cal-sdk/core';

async function sync(calendarId: string) {
  const stored = await db.syncCursors.get(calendarId);
  try {
    const { changes, nextCursor } = await syncEvents({ calendar, calendarId, cursor: stored });
    for (const change of changes) {
      if (change.type === 'upsert') await db.events.put(change.event);
      else await db.events.delete(change.eventId);
    }
    await db.syncCursors.set(calendarId, nextCursor);
  } catch (error) {
    if (!SyncCursorExpiredError.isInstance(error)) throw error;
    await db.events.clear(calendarId);
    await db.syncCursors.delete(calendarId);
    await sync(calendarId);
  }
}

Under the hood it's exactly the approaches above. Google uses sync tokens over the whole calendar, and Outlook uses a delta link over a window (a year back and two years ahead by default, which you can change with providerOptions.microsoft.timeMin and timeMax on the first sync). CalDAV uses sync-collection and doesn't fall back to CTags, so a server without RFC 6578 won't sync. Calendly, Zoom and ICS compare fingerprints. Every provider returns the same thing: series, exceptions and single events, never computed occurrences. The cursor is an opaque string, so you store it and never parse it.

Save the cursor only after the changes are applied. If your process dies in between, the next run replays them instead of losing them.

The sync guide covers paging through a large first sync and how much each provider reads per round. If you want to see the API without connecting a real account, the quickstart runs against an in-memory calendar.

FAQ

What happens when a Google Calendar sync token expires?

Google answers 410 Gone. Delete your copy of the calendar and run a full sync again. There's no way to resume from where you were.

Can I use timeMin with a Google sync token?

No. timeMin, timeMax, updatedMin, q, orderBy, iCalUID and the extended property filters can't be combined with syncToken, so the first sync has to read the whole calendar.

How does Microsoft Graph report deleted events in a delta query?

As stubs with an @removed annotation. The same stub also marks events outside the window that changed, so an event moved out of your window looks like a deletion.

Does iCloud support incremental sync?

Yes. iCloud is a CalDAV server, and CalDAV's sync-collection REPORT (RFC 6578) returns the resources that changed or were removed since your last sync token.

When should I save the sync cursor?

Only after the changes are applied to your copy. If your process dies in between, the next run replays the changes instead of losing them.

Originally published on cal-sdk.com.

Finding a free slot across Google, Outlook and iCloud

Vaibhav Pathak — Thu, 05 Nov 2026 19:42:27 +0000

Here's a feature request that sounds like an afternoon's work: show the times when Ana (on Google Calendar) and Raj (on Outlook) are both free for 30 minutes next week, during Berlin working hours.

You fetch two lists of busy times, merge them and look for the gaps. The merge really is the easy part. The hard part is getting a busy time you can trust from each provider, because the three big calendar APIs each answer "when is this person busy?" in a different way, and one of them barely answers it at all.

How does Google Calendar report busy time?

Google is the easy one. You POST /freeBusy with a window and a list of calendar ids, and get back busy ranges per calendar:

{
  "calendars": {
    "primary": {
      "busy": [{ "start": "2026-11-09T08:30:00Z", "end": "2026-11-09T09:00:00Z" }]
    },
    "team@example.com": {
      "errors": [{ "domain": "global", "reason": "notFound" }]
    }
  }
}

There are two things to notice. First, Google has already applied its own rules: an event marked "show as free" (transparency: "transparent") isn't in there, and it works for calendars the user can only see as free/busy, where listing events would give you nothing. Second, a failure doesn't fail the request. A calendar that wasn't found, or that isn't shared with you, comes back with an errors array and no busy key. If your code reads busy ?? [], that calendar now looks completely free. That isn't an edge case. It's the default way to get it wrong.

The only limit worth knowing is that one request covers at most 50 calendars, so a big team needs batching.

Why not use Outlook's getSchedule?

Microsoft Graph does have a free/busy endpoint, getSchedule. You give it email addresses and a window, and it gives you back an availabilityView string with one digit per time slot:

{ "scheduleId": "raj@contoso.com", "availabilityView": "000220130" }

0 is free, 1 tentative, 2 busy, 3 out of office and 4 working elsewhere. The slot length (availabilityViewInterval) defaults to 30 minutes and goes from 5 to 1440. There's also a scheduleItems array with the actual start and end of each item, which is what you want: the digit string rounds everything to the slot length.

The catch is in the permissions table: getSchedule doesn't support personal Microsoft accounts. If your users sign in with an outlook.com or hotmail.com address, it's not an option. On a busy shared calendar it also gives up past 1000 entries in the window.

The other route works for every account: list the events with calendarView, which expands recurring series into occurrences on the server, and work out busy time yourself. That's what calsdk does for Outlook. It does mean you now own the rules Graph would have applied for you, starting with showAs. Outlook has six values (free, tentative, busy, oof, workingElsewhere and unknown), and you have to decide which ones block time.

Interestingly, getSchedule writes "working elsewhere" as 0, which is free. I went the other way and count everything except free as busy. Someone working from another office, or tentatively holding a slot, usually doesn't want a stranger booking it.

Time zones need care too. Graph names zones the Windows way, like "Pacific Standard Time", not "America/Los_Angeles", and a Prefer: outlook.timezone header changes which zone it writes times in. For busy time, ask for UTC with Prefer: outlook.timezone="UTC" and you skip the mapping entirely.

How do you get busy time from iCloud and CalDAV?

CalDAV is the protocol behind iCloud, Fastmail, Nextcloud, Zoho and most self-hosted calendars. On paper it has free/busy too: RFC 4791 defines a free-busy-query REPORT. In practice I didn't want a feature where whether it works depends on which server the user happens to be on, so calsdk treats CalDAV the same way as Outlook: list the events and compute.

Listing is a calendar-query REPORT with a time range:

<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop><D:getetag/><C:calendar-data/></D:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR">
      <C:comp-filter name="VEVENT">
        <C:time-range start="20261108T000000Z" end="20261115T000000Z"/>
      </C:comp-filter>
    </C:comp-filter>
  </C:filter>
</C:calendar-query>

What comes back is raw iCalendar text, and everything is now your problem. Recurring events arrive as one VEVENT with an RRULE and its exceptions, so you expand the occurrences yourself. Servers read floating times and all-day dates in a zone of their choosing, so calsdk asks for a day of slack on either side and filters the results itself. Busy or free is the TRANSP property (TRANSPARENT means free), and cancelled occurrences have STATUS:CANCELLED.

Some servers have extra quirks. Zoho answers a calendar query without the event data, so every event has to be fetched again with a calendar-multiget. If one of those fetches fails, it's tempting to skip that event. Don't: a missing event is time that looks free. calsdk fails the whole listing instead.

What catches people out on every provider?

All-day events are dates, not instants. A holiday on November 9 means November 9 in the calendar's own time zone, so to turn it into a busy range you need that zone, which you only get from the calendar list. If you can't get it, the safe choice is to block the day in every zone, from UTC+14 through UTC-12. That covers more than a day, but it never shows a holiday as free.

Private events still block time. You won't see the title, but you need the range.

Declined invitations are a judgment call. The event fallback in calsdk only looks at the event's own free/busy flag and whether it's cancelled, not at your RSVP. So a declined invite that stays on the calendar still counts as busy. That's a choice, and it errs the same way as everything else: better to hide a free slot than to offer a taken one.

Rate limits are real once you list a week of events for 20 people. Retry 429s after the Retry-After the provider sends, and keep the window as small as the feature allows.

How do you merge busy times without double booking?

With trustworthy busy ranges in hand, the algorithm is short:

Put every busy range from every calendar into one list, clip it to the window, sort by start and merge ranges that overlap or touch.
Walk candidate start times (every 15 minutes, say) inside working hours, in the organizer's time zone.
Keep a candidate if [start, start + duration) doesn't overlap any merged range.

Working hours need more care than they seem to. "9 to 5 in Berlin" is a different UTC range on either side of a daylight saving change, and on the day the clocks spring forward an hour of wall-clock time simply doesn't exist. calsdk places slots on the local wall clock within each stretch of constant UTC offset, so the grid stays on the hour after the change and the missing hour gets no slots.

The important decision is what to do when one calendar can't be read. Remember Google's per-calendar errors. If Raj's calendar comes back with an error and you carry on with Ana's alone, every slot you show is a guess about Raj. A booking page that confidently offers a time that clashes is worse than one that says "couldn't check Raj's calendar". So findFreeTime fails closed: it throws FreeBusyUnavailableError, with the provider and calendarId, rather than return slots that might clash.

How calsdk does it

calsdk wraps all of the above in two functions. getFreeBusy gives you merged busy ranges for one provider. It uses Google's endpoint where it exists, and lists events everywhere else, returning a compatibility warning so you know which path it took. findFreeTime takes calendars from any mix of providers and gives you the slots:

import { findFreeTime, FreeBusyUnavailableError } from '@cal-sdk/core';
import { icloud } from '@cal-sdk/caldav';
import { google } from '@cal-sdk/google';
import { microsoft } from '@cal-sdk/microsoft';

try {
  const { slots, warnings } = await findFreeTime({
    calendars: [
      { calendar: google({ accessToken: anaToken }) },
      { calendar: microsoft({ accessToken: rajToken }), calendarIds: ['primary', roomCalendarId] },
      { calendar: icloud({ appleId, appPassword }) },
    ],
    timeMin: '2026-11-09T00:00:00Z',
    timeMax: '2026-11-14T00:00:00Z',
    duration: 30,
    workingHours: { start: '09:00', end: '17:00', timeZone: 'Europe/Berlin' },
  });
  // slots: [{ start: '2026-11-09T08:00:00.000Z', end: '2026-11-09T08:30:00.000Z' }, ...]
} catch (error) {
  if (FreeBusyUnavailableError.isInstance(error)) {
    // Tell the user which calendar couldn't be read, instead of guessing.
  }
  throw error;
}

Slots come back as UTC instants, ready to show in each viewer's own zone. The free time guide covers slot intervals, overlapping slots and booking the one the user picks.

If you want to poke at it without connecting a real account, the quickstart runs against an in-memory calendar in a couple of minutes.

FAQ

Does Outlook have a free/busy API?

Yes: Microsoft Graph's getSchedule. It doesn't support personal Microsoft accounts (outlook.com and hotmail.com), and it gives up past 1000 entries in the window. Listing events with calendarView and computing busy time yourself works for every account.

Does iCloud support free/busy queries?

iCloud speaks CalDAV, and RFC 4791 defines a free-busy-query REPORT, but support varies between CalDAV servers. Listing events with a calendar-query REPORT and computing busy time yourself works the same on every server.

Should tentative and declined events count as busy?

Count everything except events marked "show as free" as busy, including tentative ones. calsdk also treats a declined invite that is still on the calendar as busy, because offering a taken slot is worse than hiding a free one.

What should happen when one calendar can't be read?

Fail and say which calendar it was. Slots computed without that calendar are guesses that can clash. calsdk's findFreeTime throws FreeBusyUnavailableError with the provider and calendar id.

How many calendars can one Google freeBusy request check?

Batch the request for larger teams.

Originally published on cal-sdk.com.

One standup, three calendar APIs

Vaibhav Pathak — Thu, 05 Nov 2026 19:42:02 +0000

A daily standup is the most boring event on any calendar. It's also where calendar APIs disagree the most.

Here's the event I'll use for the rest of this post: a standup at 9:30 in Berlin, every weekday, ten times, starting Monday, November 9. Someone cancels Tuesday's. Someone else moves Friday's to 10:00. Any calendar UI can show that in a second, but each of the three big APIs stores it in its own way, and an app that supports all three has to read and write every version.

I've spent a lot of time with these differences while building calsdk, so here is how each provider stores that week, and the places where it caught me out.

How does Google Calendar store a recurring event?

Google is the closest to the standard. The series is one event with a recurrence array of RFC 5545 lines:

{
  "id": "abc123",
  "summary": "Standup",
  "start": { "dateTime": "2026-11-09T09:30:00+01:00", "timeZone": "Europe/Berlin" },
  "end": { "dateTime": "2026-11-09T09:45:00+01:00", "timeZone": "Europe/Berlin" },
  "recurrence": ["RRULE:FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR;COUNT=10"]
}

The changes are separate objects. Friday's move is an instance with its own id (the series id plus the original start in UTC), a recurringEventId pointing back at the series, and an originalStartTime that says which slot it replaced:

{
  "id": "abc123_20261113T083000Z",
  "recurringEventId": "abc123",
  "originalStartTime": { "dateTime": "2026-11-13T09:30:00+01:00", "timeZone": "Europe/Berlin" },
  "start": { "dateTime": "2026-11-13T10:00:00+01:00", "timeZone": "Europe/Berlin" }
}

Tuesday's cancellation is the first surprise. You might expect an EXDATE line in the series' recurrence, and Google does accept those. But deleting an instance through the API leaves a third kind of object: an instance with status: "cancelled". So there are two ways to say "not on Tuesday", and your code has to handle both.

For reading, singleEvents=true expands the series on Google's side and gives you one object per occurrence. That's convenient, but the expanded list doesn't tell you which occurrences were edited. Friday looks like every other day, just at 10:00. If your UI wants to say "this occurrence was changed", you need a second request with singleEvents=false to find the exceptions.

The hard part is "this and following events". Google Calendar's UI offers it, but the API has no such operation. You build it yourself: end the old series just before the chosen occurrence, and create a new series starting there. If the rule has a COUNT, you split the count between the two halves. Then you learn that Google resets edited occurrences when a series moves or splits, so Friday's change to 10:00 is lost unless you carry it over.

How does Outlook store a recurring event?

Microsoft Graph doesn't use RRULE at all. A series is a seriesMaster with a recurrence object made of a pattern and a range:

{
  "id": "AAMkAGI2...",
  "type": "seriesMaster",
  "subject": "Standup",
  "originalStartTimeZone": "W. Europe Standard Time",
  "recurrence": {
    "pattern": {
      "type": "weekly",
      "interval": 1,
      "daysOfWeek": ["monday", "tuesday", "wednesday", "thursday", "friday"],
      "firstDayOfWeek": "monday"
    },
    "range": {
      "type": "numbered",
      "startDate": "2026-11-09",
      "numberOfOccurrences": 10,
      "recurrenceTimeZone": "W. Europe Standard Time"
    }
  }
}

Two things stand out. First, time zones have Windows names: Berlin is W. Europe Standard Time, and New York is Eastern Standard Time. So the first thing you need is a mapping table to IANA names, and calsdk ships one generated from the CLDR data.

Second, a pattern can't express everything an RRULE can. Daily, weekly on set days, and monthly or yearly on a date or an "nth weekday" all map cleanly. Hourly rules, RDATE, and most combinations of BYDAY with BYMONTHDAY have no pattern. Some rules map but behave differently. RRULE:FREQ=MONTHLY;BYMONTHDAY=31 skips months without a 31st, while Outlook moves that occurrence to the last day of the month. If you translate it naively, the user gets a meeting on February 28 that they never asked for. The RRULE that matches Outlook's behavior is BYMONTHDAY=28,29,30,31;BYSETPOS=-1, and that's how calsdk reads it back.

Instances have a type of occurrence or exception and a seriesMasterId. Cancellations are where Outlook surprised me most: there's no exclusion list. Outlook only remembers occurrences that were deleted one by one, as cancelledOccurrences on the master. Graph only returns that when you $select it on a single event by id, so you need one extra request per series to learn which days are gone. And because no list exists, you can't exclude a date by writing an EXDATE. You delete that occurrence instead.

There's no "this and following" here either. You end the series and create a new one, with the same bookkeeping as on Google.

How do iCloud and CalDAV store a recurring event?

iCloud, Fastmail, Nextcloud and Zoho all speak CalDAV. CalDAV stores calendars as iCalendar files on a server, so the whole series, with every exception, is one .ics resource:

BEGIN:VEVENT
UID:standup-7f3a
DTSTART;TZID=Europe/Berlin:20261109T093000
DTEND;TZID=Europe/Berlin:20261109T094500
RRULE:FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR;COUNT=10
EXDATE;TZID=Europe/Berlin:20261110T093000
SUMMARY:Standup
END:VEVENT
BEGIN:VEVENT
UID:standup-7f3a
RECURRENCE-ID;TZID=Europe/Berlin:20261113T093000
DTSTART;TZID=Europe/Berlin:20261113T100000
DTEND;TZID=Europe/Berlin:20261113T101500
SUMMARY:Standup
END:VEVENT

(I've left out the VCALENDAR wrapper and the VTIMEZONE block.) Tuesday is an EXDATE. Friday is a second VEVENT with the same UID and a RECURRENCE-ID naming the slot it replaces. It's the most honest of the three formats, and the most work to use.

There are no instance ids at all. If your app needs to point at "Friday's standup", you have to make up an id. calsdk uses the resource name plus the original start in UTC, such as standup-7f3a_20261113T083000Z.

Editing one occurrence means rewriting the whole file. You read it, change or add a VEVENT, and PUT it back with If-Match on the ETag, so you don't overwrite someone else's change made in the meantime.

Then you expand the series yourself. CalDAV does let you ask the server to expand a series, but calsdk asks the server only for the resources in a time window, and expands them locally with ical.js. That gives one code path for every CalDAV server, and the same code reads plain ICS feeds, which have no server to ask. Expanding locally has its own traps. ical.js's own event expansion reads any TZID it has no definition for as UTC, so an EXDATE or UNTIL compared with a start in New York can be hours off and fail to match. calsdk builds the set of occurrences itself to avoid that. A rule like FREQ=SECONDLY from 2009 would expand forever, so expansion is capped, and those rules list only their first instance with a warning.

Servers vary in small ways too. Zoho, for example, answers a calendar query without the event data, so every event has to be fetched again.

How do you put one API over all three?

The design choice that made the rest possible was to speak RFC 5545 everywhere. Google and CalDAV already use it, so only Outlook needs translating, in both directions. Every event has a kind: single, series, occurrence or exception. Occurrences and exceptions carry a seriesId and an originalStart, which stays put when the occurrence moves. Here's the standup week, which works the same with microsoft() or icloud() in place of google():

import { createEvent, deleteEvent, listEvents, updateEvent } from '@cal-sdk/core';
import { google } from '@cal-sdk/google';

const calendar = google({ accessToken: () => getFreshAccessToken() });

await createEvent({
  calendar,
  event: {
    title: 'Standup',
    start: { dateTime: '2026-11-09T09:30:00', timeZone: 'Europe/Berlin' },
    end: { dateTime: '2026-11-09T09:45:00', timeZone: 'Europe/Berlin' },
    recurrence: ['RRULE:FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR;COUNT=10'],
  },
});

const { events } = await listEvents({
  calendar,
  timeMin: '2026-11-09T00:00:00Z',
  timeMax: '2026-11-14T00:00:00Z',
});

// Tuesday is cancelled, Friday moves to 10:00.
await deleteEvent({ calendar, eventId: events[1].id });
await updateEvent({
  calendar,
  eventId: events[4].id,
  patch: {
    start: { dateTime: '2026-11-13T10:00:00', timeZone: 'Europe/Berlin' },
    end: { dateTime: '2026-11-13T10:15:00', timeZone: 'Europe/Berlin' },
  },
});

updateEvent and deleteEvent take a scope for instances: 'this' (the default), 'all', or 'thisAndFollowing'. On Google, calsdk does the split for thisAndFollowing itself, and if the second write fails, it deletes the new series so you aren't left with both. Outlook and CalDAV throw UnsupportedFunctionalityError for that scope, and calendar.capabilities.events.recurrenceScopeThisAndFollowing tells you beforehand, so your UI can hide the option.

The rule I held to is to never save a schedule that's different from the one you asked for. If Outlook can't store a rule exactly, calsdk throws instead of saving something close. If the call can still do what you meant, but something is lost along the way, such as Google resetting edited occurrences when a series moves, the call succeeds and returns a warning that says so. A standup that quietly lands on the wrong days is worse than an error.

The recurring events guide has every scope and the limits for each provider. If you want to see it work without connecting an account, the quickstart runs against an in-memory calendar.

FAQ

How does Google Calendar mark a cancelled occurrence?

In one of two ways: an EXDATE line in the series' recurrence, or, when the occurrence was deleted through the API, an instance with status: "cancelled". Code that reads Google events has to handle both.

Does Microsoft Graph support RRULE?

No. Graph stores a recurring event as a pattern and a range, with Windows time zone names. Most RRULEs translate, but hourly rules, RDATE and some BYDAY and BYMONTHDAY combinations have no Outlook pattern.

How do I find the deleted occurrences of an Outlook recurring event?

Read cancelledOccurrences on the series master. Graph only returns it when you $select it on a single event by id, so it costs one request per series.

Do the calendar APIs support "this and following events"?

Google, Outlook and CalDAV don't have that operation. You end the old series before the chosen occurrence and create a new series from it. calsdk does that split for you on Google.

How do you edit one occurrence on CalDAV?

Rewrite the series' .ics file: add or change a VEVENT with a RECURRENCE-ID for that occurrence, then PUT the file back with If-Match on its ETag.

Originally published on cal-sdk.com.

THX-02: a 4.5B open decision model that beats 27B models — and the API is free

FX-LgLL — Thu, 05 Nov 2026 19:09:13 +0000

Most of what AI does inside a real product is not writing essays. It is deciding.

Which team owns this ticket. Is this message phishing. Does this applicant pass the policy. What is the next action. Which of these 40 categories does this document belong to.

Today teams pick between two bad options. A large chat model: slow, expensive per call, and it will happily invent a label that was never on your list. Or a narrow classifier: fast and cheap, but you retrain it for every new task, and it gives you a score you cannot explain to anyone.

There is a third path. A model built for decisions only. You send it a state — text, JSON, or up to four images — plus typed questions, and it returns a probability for every option you supplied. It does not generate prose, so it cannot produce an answer outside your list.

That model is THX-02. 4.5B parameters. Apache-2.0 weights, runtime and server. And we are giving the API away for free.

We built it because we needed it ourselves, and we are releasing it because a decision layer that everyone has to pay per token for is a decision layer nobody builds.

The free API

This is the part we want to lead with, because it changes who can use this.

Free API access. Get a key, call it, ship it. No per-token invoice waiting for you at the end of the month.
Same model as the weights. The hosted endpoint runs exactly what you can download. No "API version" that is quietly better than the open one.
No lock-in by design. If the free tier ever stops fitting you, pip install "thx02[local]" puts the same model on one 24 GB GPU, and thx02 serve gives you the same HTTP interface on your own metal.
Rate limits are published in the API docs, so you can size your integration instead of discovering the ceiling in production.

Free is not a marketing word here. The weights are Apache-2.0, so the floor of what you can do without paying anyone is: run it yourself, forever.

pip install thx02
export HALX_API_KEY="your-key"

Get a key at the API docs link at the bottom of this post.

30 seconds to your first decision

Here is a real request: a customer writes in Azerbaijani that they were charged twice for the same card payment.

thx02 decide \
  --state "Müştəri: kartımdan eyni ödəniş üçün iki dəfə pul çıxıb, 240 AZN. Sifariş #88213." \
  --choice "billing,support,shipping,technical" \
  --yesno "Should this be refunded automatically?"

billing      ████████████████████████████████  96.0%
support      ██                                   2.4%
shipping     ▏                                    0.9%
technical    ▏                                    0.7%

refund: yes  ███████████████████████████████   95.8%
refund: no   █                                    4.2%

126 ms · think=auto · 1 thought generated

Two things to notice. The answer is a distribution, not a string — so you can set your own threshold and let the model abstain instead of guessing. And the model decided on its own that this one needed no step-by-step reasoning: it answered directly in 126 ms.

The same call in Python, with three question types:

from thx02 import Client, choice, noul, score

client = Client()

result = client.decide(
    state="Müştəri: kartımdan eyni ödəniş üçün iki dəfə pul çıxıb, 240 AZN. Sifariş #88213.",
    questions=[
        choice("team", ["billing", "support", "shipping", "technical"]),
        noul("refund", "Should this be refunded automatically?"),
        score("urgency", "How urgent is this?", 1, 5),
    ],
)

for q in result.answers:
    print(q.id, q.probabilities)

choice takes up to 255 options. noul is a yes/no with calibrated probabilities on both sides. score is an ordered scale, returned as a full distribution over the levels rather than a single number — which means you can see whether the model is confident between 3 and 4, or genuinely torn between 1 and 5.

Every answer is a full distribution. That is the whole point. A routing system that auto-acts above 0.9 and escalates below 0.6 is a system you can put in front of a customer. A system that returns "billing" and nothing else is a system you have to babysit.

Thinking, but only when it changes the answer

The interesting engineering problem in a decision model is not "make it smarter." It is "make it smarter without making it slower and without letting it talk itself out of a correct answer."

THX-02 has a built-in gate:

The model answers directly first.
The gate asks two questions: is the model unsure, and does this question need step-by-step work — arithmetic, multi-step rules, code tracing.
Only if both point that way does a small adapter write a short thought, capped at 96 tokens.
The model re-reads the question with that thought and mixes the two distributions, weighted so a bad thought cannot overturn a confident direct answer.

That last step is the important one. Thinking is an additive signal, not a replacement. If the direct pass is confident and the thought disagrees, the thought loses.

Median latency stays around 80 ms with the gate active. On math word problems the thought adds about 25 points. On natural language inference, code tracing and graduate-level science it adds 4 to 6 — real, but not worth paying 10x latency for on every request, which is exactly why the gate exists.

Here is a case where it fires. A policy check that requires arithmetic:

result = client.decide(
    state="Applicant: monthly income 4,200 AZN, existing loan payments 1,800 AZN, "
          "requested new payment 600 AZN.",
    questions=[noul("dti", "Does this applicant pass the 50% debt-to-income policy?")],
    think=True,
    return_thoughts=True,
)

print(result.answers[0].probabilities)
print(result.thoughts[0])

fail: 75.5%   pass: 24.5%

thought: existing 1800 + new 600 = 2400. 2400 / 4200 = 57.1% > 50% policy limit. fail.

The model did not guess. It computed, showed the arithmetic, and the probability moved because the arithmetic moved it.

Results

Decision Index 0.3, the open benchmark suite for decision models:

Model	Size	Decision Index 0.3
THX-02 (thinking)	4.5B	65.06
THX-02 (direct)	4.5B	63.83
Best 27B decision model	27B	62.25
Cloudflare clef	—	61.71
Jev	—	57.96
THX-01	0.3B	11.30

Read that table again. A 4.5B model scores above the best 27B decision model we could find, and above Jev's published index by more than seven points. On the public Decision Vision Index sets it reaches 72.19.

We also want to be precise about what those numbers are, because "we beat X" deserves the methodology attached:

We ran the official evaluation kit ourselves, on a stratified sample of 6,018 requests rather than the full suite.
The competitor numbers are their published public indexes, not our re-runs.
We decontaminated against the test sets: exact and n-gram matching for text, exact and perceptual hashing for all 10,742 evaluation images.
No benchmark test item was used in training.
We will submit to the maintainers for an independent measurement, and we will publish whatever comes back.

If the independent number lands lower, we will say so in the same place we said 65.06.

How we got there

Five post-training stages on top of an open base model. Each one was kept only if it improved a held-out split — no stage survived on vibes.

Distribution distillation. 361,000 requests, trained against full probability distributions with a strictly proper scoring rule plus a KL anchor to the base model. Proper scoring rules are what make the output calibrated instead of merely confident.
Disagreement mining. A pass over cases where the teacher and the student disagreed, keeping the ones where the teacher was right and the student was confidently wrong.
Inference-structure training. Option-order averaging, joint reading of state and question, and marker-token scoring per option — the things that remove position bias and multi-token label tokenization problems you would otherwise have to solve yourself.
Self-verified thoughts. Candidate thoughts were tested on the model itself and kept only if they raised the probability of the correct answer. A thought that does not change the answer is a thought we do not pay latency for.
Targeted skills. About 700,000 candidate examples, with answers verified by executing code, running solvers or querying engines rather than asking another model. 161,000 survived — the ones the model still missed or was unsure about.

That last point matters more than it sounds. A model trained on another model's answers inherits that model's mistakes, confidently. A model trained on executed code and solver output inherits the truth.

Run it anywhere

Hosted, free:

pip install thx02
export HALX_API_KEY="your-key"
thx02 decide --state "..." --choice "a,b,c"

On your own GPU — one 24 GB card is enough:

pip install "thx02[local]"
python -c "from thx02 import THX02; THX02.load('HAL-X-AI/THX-02')"

As a server with the same HTTP interface as the hosted API:

pip install "thx02[server]"
thx02 serve

On a phone. The client is zero-dependency and runs in Termux:

pkg install python
pip install thx02
export HALX_API_KEY="your-key"
thx02 decide --state "..." --yesno "Is this message spam?"

Fine-tuned to your domain. The weights load through ms-swift like any other model of this family, and the think adapter is a separate artifact you can keep, drop or retrain.

CPU inference is possible but not the point: about 0.6 s per short request on 8 x86 cores in BF16. On a GPU you are looking at 50 to 150 ms.

What it is not good at

We would rather tell you now than have you find out in production.

Very large bare-label sets with no descriptions. If you hand it 200 category names and nothing else, it degrades. Give each option a short description and it recovers.
Pure recognition tasks. Thinking does not help much on "what is in this image" style questions — the gate correctly stays off, but there is no reasoning gain to be had.
Anything that needs a long generation. It is a decision model. It returns distributions, not paragraphs. If you need an explanation, the thought trace is short by design (≤96 tokens).

Why open

A decision layer sits in the middle of your product. It sees customer messages, applicant data, medical codes, transaction descriptions. For anyone in a regulated sector — banking, insurance, health, public sector — "send it to a remote black box and hope" is not an architecture, it is a risk.

So: Apache-2.0 weights, the think adapter, the runtime and the server. On-premise, inspectable, adaptable. Multilingual from launch, with English, Russian, Chinese, Turkish and Spanish alongside a deliberate focus on Azerbaijani, Kazakh, Uzbek and Kyrgyz — languages that most decision models treat as an afterthought, if they treat them at all.

And free to call, because the fastest way to find out whether a decision model belongs in your product is to put one in your product this afternoon.

Tracking Google Flights fares: New York to London was $197 on December 8 and $512 on December 20

CleanScrape — Thu, 05 Nov 2026 18:51:45 +0000

Flight prices change every day, and checking them by hand gets old quickly. So I built a Google Flights scraper and fare tracker on Apify. You type routes the way you'd say them, pick dates, and get every flight back as a row in a table you can export. Put it on a schedule and each run gives you only new flights or lower prices.

The input

This is the run from the video:

{
  "routes": ["New York to London"],
  "departureDates": ["+30"],
  "maxFlightsPerSearch": 20
}

A route can be airport codes (JFK-LAX), a city code that covers all of a city's airports (NYC-LON), or plain city names. Dates can be one day, a range like 2026-12-01..2026-12-31, or +30, which means 30 days after the run date. That last one is what makes scheduled runs useful: a daily schedule always looks the same distance ahead.

There's also round trips (with the real return flights for the cheapest outbound options), passengers, cabin, nonstop or one-stop limits, airlines or alliances, 22 currencies and the country you're buying from.

The output

New York to London for December 5 came back with 20 flights in 3 seconds, from all three New York airports to three London airports. One row looks like this:

{
  "route": "NYC-LON",
  "from": "JFK",
  "to": "LHR",
  "departure": "2026-12-05T18:00:00-05:00",
  "arrival": "2026-12-06T06:20:00+00:00",
  "duration": "7 h 20 min",
  "stops": 0,
  "airlines": "Virgin Atlantic",
  "flightNumbers": "VS4",
  "aircraft": "Airbus A330-900neo",
  "price": 300,
  "currency": "USD",
  "co2Kg": 369
}

Each search also gets a row in a second table with the cheapest fare ($270 here), the cheapest nonstop ($300), Google's typical price for the trip ($279) and its verdict: low, typical or high. It includes Google's own price history for about the last 60 days. For this trip it was $197 two months ago and dipped to $109 one day.

The cheapest day in a month

One search per departure day from December 5 to January 4, keeping only the cheapest flight each day, turns the Searches table into a price calendar. December 8 to 10 cost $197. December 20 cost $512, and Google marked it high. That's 31 searches, which ran in 14 seconds.

What testing found

Time zones matter more than they look. An evening flight from New York lands in London the next morning. Each time keeps its UTC offset, so arrival above is Dec 6 even though the search was for Dec 5. I checked this on every row of a 300-search run: no arrival before its departure, and every duration matches the two timestamps.

Google's price level isn't three values. I first mapped it to low, typical and high, and an audit showed "high" days sitting inside Google's typical range. The real scale has five steps: one below the typical range, three inside it and one above. Comparing every search with its range made that obvious.

Not every city code works. City codes like NYC, LON, PAR and TYO cover several airports, but Google rejects Dallas's code and treats a few others as a single airport. I tested every code in the table and mapped those cities to their main airport. An example task that searched "Dallas to Cancun" caught it before release.

Memory. Google's result pages are 3.5 MB each, and the first version ran out of memory at 256 MB with six searches in parallel. It now decodes only the part of the page that holds the flights, about 70 KB, and peaks around 150 MB.

Pages sometimes arrive incomplete. About one page in twelve came back without flight data and worked on the next try, so every search retries a few times before it counts as failed. In a 300-search test, all 300 got an answer.

Tracking fares

Turn on "Only new flights or lower prices", give it a tracking name and schedule it. The first run saves everything it finds. Later runs save a flight only if it's new for that route and date, or its price dropped, with the previous price next to it. In my two test runs 12 minutes apart nothing had changed, so the second run saved nothing, and its Searches rows showed the cheapest fare at the same $270 and $384 as before.

Pricing

$1 per 1,000 flights saved and $0.50 per 1,000 searches, less on paid Apify plans. One route on one date with 20 flights costs about two cents. The search fee is there so scheduled checks that find nothing new still cover their cost; failed searches are free.

Try it

https://apify.com/cleanscrape/google-flights-scraper

Questions or ideas: contact.cleanscrape@gmail.com, or the Issues tab on the Actor page.

Niko, CleanScrape developer

Search WeChat Official Account articles by topic in Python

Qovia — Thu, 05 Nov 2026 18:29:33 +0000

I build Qovia, the API used in this post. The search loop (query, filter, paginate, de-duplicate) works with any source that can return WeChat Official Account articles; the CSV and state-file parts are plain Python.

WeChat Official Accounts (微信公众号) are where a lot of Chinese media, brands and analysts publish long-form posts. There is no public web search page you can scrape the way you would for Reddit or X, so people usually either refresh individual accounts on their phone or give up on topic discovery entirely.

In this tutorial we'll write a small Python script that:

searches WeChat Official Account articles by topic with POST /v1/search and platforms: ["wechat"],
narrows the window with since / until (Beijing calendar days),
pages through results with page / limit and has_more,
writes hits to a CSV you can open in a spreadsheet or feed into a notebook.

A short section at the end contrasts topic search with monitoring a specific account by mp_id (the account-scoped pair GET /v1/raw/wechat/account_search + GET /v1/raw/wechat/articles).

Setup

You need Python 3.9+ and requests:

pip install requests
export QOVIA_KEY=sk-qovia-...   # key from the Qovia console

Get a key from Qovia. Requests authenticate with Authorization: Bearer <key> (an x-api-key header also works). The search reference lists every parameter with a real response example.

Step 1: a POST helper

Successful calls return HTTP 200 with {"request_id", "data", "meta"}. The payload you care about is in data; billing lives in meta (meta.charged_usd, meta.billed, meta.empty). Empty results are still 200, with meta.empty: true, and are not charged. Errors are {"error": {"code", "message", ...}} and are never charged. A 429 carries a Retry-After header — wait and retry once:

import os
import time
import requests

BASE = "https://api.qovia.ai"
HEADERS = {
    "Authorization": f"Bearer {os.environ['QOVIA_KEY']}",
    "Content-Type": "application/json",
}

spent = 0.0


def post(path, body):
    """POST JSON; honour Retry-After once; accumulate meta.charged_usd."""
    global spent
    for attempt in range(2):
        r = requests.post(
            f"{BASE}{path}",
            headers=HEADERS,
            json=body,
            timeout=120,  # a search can take a few seconds
        )
        if r.status_code == 429 and attempt == 0:
            time.sleep(int(r.headers.get("Retry-After", "10")))
            continue
        r.raise_for_status()
        payload = r.json()
        spent += float((payload.get("meta") or {}).get("charged_usd") or 0)
        return payload.get("data") or {}

Unknown body fields are rejected with 400 invalid_params, so only send documented keys.

Step 2: WeChat-only topic search

POST /v1/search is the cross-domain search endpoint. Pass platforms: ["wechat"] to restrict hits to Official Account articles. query is required (1–200 characters). Space-separated terms are ANDed — for example 人工智能教育 means both words must appear. Put words you want out of the results in exclude_words.

def search_wechat(query, since=None, until=None, page=1, limit=20, **extra):
    body = {
        "query": query,
        "platforms": ["wechat"],
        "page": page,
        "limit": limit,  # 10 or 20 only
    }
    if since:
        body["since"] = since  # YYYY-MM-DD, Beijing time, inclusive
    if until:
        body["until"] = until  # YYYY-MM-DD, Beijing time, inclusive
    body.update({k: v for k, v in extra.items() if v is not None})
    return post("/v1/search", body)


data = search_wechat("新能源汽车")
print("total hits:", data.get("total"))
print("by_platform:", data.get("by_platform"))
for item in data.get("items", []):
    author = (item.get("author") or {}).get("name")
    print(item.get("published_at"), author, item.get("title"))
    print(" ", item.get("url"))

Each item in data.items is a search hit with fields such as id, platform (here "wechat"), site, title, text, url, published_at, type, sentiment, and author (name, sometimes followers / avatar). data.total is the hit count; data.by_platform breaks that count down per platform (useful when you later widen platforms beyond WeChat). page, limit and has_more tell you where you are in the result set.

This is the search envelope for /v1/search. Other Qovia endpoints (for example the raw WeChat account/article routes below) return their own data shapes — don't assume every endpoint looks like this one.

Step 3: date window (`since` / `until`)

Dates are whole calendar days in Beijing time (+08:00). Defaults are roughly the last 7 days (until defaults to today; since defaults to 6 days before until). The span from since to until may be at most 90 days, and until cannot be later than today.

from datetime import date, timedelta

today = date.today()  # if your machine is Asia/Singapore or UTC+8, this is fine
until = today.isoformat()
since = (today - timedelta(days=13)).isoformat()  # two weeks inclusive

data = search_wechat(
    "大模型 应用",
    since=since,
    until=until,
    exclude_words="招聘 广告",
    match=["title", "content"],  # default also includes author
)
print(f"{since} → {until}: {data.get('total')} hits, page {data.get('page')}")

Useful optional filters from the same request body:

Field	What it does
`exclude_words`	Drop results that contain these words
`author`	Only this Official Account display name
`match`	Where terms must hit: `title`, `content`, `author`, `media_text`, `audio` (default `title`+`content`+`author`)
`types`	`original` / `repost` / `comment`
`sentiment`	`negative` / `neutral` / `positive`

For brand or topic monitoring you usually want match=["title", "content"] so an account name that happens to contain your keyword doesn't pollute the list.

Step 4: pagination

limit is 10 or 20 (default 20). page starts at 1 and goes up to 50. Keep requesting the next page while has_more is true, and stop early if you only need a sample:

def collect_wechat(query, since=None, until=None, max_pages=5, limit=20, **extra):
    items = []
    for page in range(1, max_pages + 1):
        data = search_wechat(
            query, since=since, until=until, page=page, limit=limit, **extra
        )
        batch = data.get("items") or []
        items.extend(batch)
        if not data.get("has_more") or not batch:
            break
        time.sleep(0.5)  # stay under the account rate limit
    return items


hits = collect_wechat("人工智能 教育", since=since, until=until, max_pages=3)
print(len(hits), "articles; spent so far $", round(spent, 4))

Rate limit is 60 requests per minute per account (600 after the first top-up). Honour Retry-After on 429 — the helper already does that once.

Step 5: write a CSV

import csv
from pathlib import Path


def to_csv(items, path="wechat-topic-hits.csv"):
    path = Path(path)
    with path.open("w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(
            f,
            fieldnames=[
                "id",
                "published_at",
                "author",
                "title",
                "url",
                "sentiment",
                "type",
                "text",
            ],
        )
        w.writeheader()
        for it in items:
            w.writerow(
                {
                    "id": it.get("id"),
                    "published_at": it.get("published_at"),
                    "author": (it.get("author") or {}).get("name"),
                    "title": it.get("title"),
                    "url": it.get("url"),
                    "sentiment": it.get("sentiment"),
                    "type": it.get("type"),
                    "text": (it.get("text") or "")[:500],
                }
            )
    return path


path = to_csv(hits)
print("wrote", path, "rows=", len(hits), "charged_usd≈", round(spent, 4))

Run it:

python search_wechat_topic.py

You get a CSV of WeChat Official Account articles matching your topic in the date window, plus a running spent total from meta.charged_usd. Non-empty /v1/search calls are billed (documented at $0.005 per call); empty answers are free.

Topic search vs monitoring a specific account

These are two different jobs:

Goal	Endpoint(s)	You need
Discover articles about a topic across many 公众号	`POST /v1/search` with `platforms: ["wechat"]`	A `query` (and optional `since` / `until`)
Follow one account's publishing history	`GET /v1/raw/wechat/account_search` → `GET /v1/raw/wechat/articles`	The account's `mp_id` (starts with `gh_` or `wxid_`)

Topic search answers "who is writing about 新能源汽车 this week?". Account monitoring answers "what did 央视新闻 publish since yesterday?".

If you already know the account name, resolve it once and poll its article list:

def get(path, **params):
    """Same envelope rules as post(), for the raw WeChat GETs."""
    global spent
    params = {k: v for k, v in params.items() if v is not None}
    for attempt in range(2):
        r = requests.get(f"{BASE}{path}", headers=HEADERS, params=params, timeout=60)
        if r.status_code == 429 and attempt == 0:
            time.sleep(int(r.headers.get("Retry-After", "10")))
            continue
        r.raise_for_status()
        payload = r.json()
        spent += float((payload.get("meta") or {}).get("charged_usd") or 0)
        return payload.get("data") or {}


accounts = get("/v1/raw/wechat/account_search", keyword="央视新闻", kind="all")
for acc in accounts.get("items", []):
    print(acc["mp_id"], acc["name"], acc.get("type"), acc.get("verified_by"))

# pick the right mp_id, then:
page = get("/v1/raw/wechat/articles", mp_id="gh_example")
for a in page.get("items", []):
    print(a["published_at"], a["title"], a["url"])

account_search returns one page (not paginated): mp_id, name, wechat_id, avatar, description, verified, verified_by, type. kind is subscription (default), service, or all. articles returns that account's history newest-first; pass the previous page's cursor back unchanged and stop when has_more is false. Each article item has title, url, published_at, cover, digest, author, ip_region, is_original, is_paid, and in the documented example also reads_label and likes.

Those raw routes are the right tool when you care about a fixed watchlist. For "find every 公众号 article on this topic", stay on /v1/search.

Putting it together

A minimal daily job: search yesterday's Beijing day for your keywords, append new URLs to a state file, write only fresh rows to CSV.

import json
from pathlib import Path

SEEN = Path("seen-urls.json")


def load_seen():
    return set(json.loads(SEEN.read_text())) if SEEN.exists() else set()


def save_seen(seen):
    SEEN.write_text(json.dumps(sorted(seen), ensure_ascii=False))


def daily(query, keywords_extra=None):
    yesterday = (date.today() - timedelta(days=1)).isoformat()
    seen = load_seen()
    fresh = []
    for q in [query, *(keywords_extra or [])]:
        for it in collect_wechat(q, since=yesterday, until=yesterday, max_pages=3):
            url = it.get("url")
            if url and url not in seen:
                seen.add(url)
                fresh.append(it)
    save_seen(seen)
    if fresh:
        to_csv(fresh, path=f"wechat-{yesterday}.csv")
    print(f"{yesterday}: {len(fresh)} new; spent ${spent:.4f}")
    return fresh


if __name__ == "__main__":
    import sys
    daily(sys.argv[1] if len(sys.argv) > 1 else "人工智能")

Cron it once a day (Asia/Singapore or Asia/Shanghai — both sit on UTC+8, same calendar day as Beijing for these date fields).

Limits worth knowing

/v1/search spans at most 90 Beijing days; default window is about the last 7.
Max page 50 × 20 results = 1,000 hits per query window — treat it as a ranked sample, not a full crawl of WeChat.
Empty results and errors are not charged; only successful non-empty answers debit meta.charged_usd.
Account rate limit: 60 req/min (600 after first top-up). Honour Retry-After.
New accounts get trial calls before prepaid balance — see the billing notes.

Going further

Widen platforms to ["wechat", "weibo", "toutiao", "news"] when you want the same topic across Chinese social + news in one call; read data.by_platform to see where volume sits.
Combine both modes: use topic search to discover which accounts keep showing up, then add their mp_ids to an account-monitor watchlist.
Ask an assistant instead: the same endpoints are on the MCP server at https://api.qovia.ai/mcp (OAuth or the same API key as a Bearer token). See the MCP docs.

Full parameter lists live in the API reference, and there's a single-file dump at qovia.ai/llms-full.txt if you want to drop it into a model's context.

This article was written with AI assistance and reviewed against Qovia's OpenAPI. I work on Qovia.

DEV Community: api

677 of 785 Telegram Reaction Chips Counted Zero. The Custom Emoji and the Paid Stars Were Fine

Three ways to draw a reaction, one of which broke everything

What reactionsTotal still is not

views is a running counter, and my own history proves it

postId is not a post count — it is a message counter, and albums eat it

Every failure answers HTTP 200

Calling it

Live Score Alerts: API Trust Boundaries for Priority Sports Notifications

The constraint that changed the build

How should a realtime sports score API prioritize notifications across trust boundaries?

The smallest implementation I trust

What does recovery cost across a real notification workload?

The trade-off I would accept

References

Cara menemukan PDF gratis dari sebuah makalah ilmiah lewat DOI-nya

Minta semua lokasi dari OpenAlex, bukan hanya penanda open access

Periksa lima byte pertama

Sisa rantainya

Abstrak datang sebagai inverted index

Streaming Feels Faster, but Your Model Isn't: A Wire-Level Timestamp FAQ

The claim that keeps coming back

Two runs, one clock

FAQ: four claims, four corrections

1. "Streaming makes generation faster."

2. "Time to first byte is a model latency number."

3. "If chunks are arriving, nothing is buffering."

4. "Streaming is the safer default on a free tier."

Cross-check in ten seconds

Decision table

Where this bites on a free tier

What this test does not tell you

Who should skip this

The model that survives the clock

Track new jobs at any Greenhouse, Lever or Ashby company daily with Python

Why the usual approach hurts

What I built

Step 1: get an Apify token

Step 2: one call with curl

Step 3: the same thing in Python, saved to CSV

Step 4: run it daily and get only new jobs

Not on Apify? Same data as a plain REST API

Honest limits

Price

Wrap-up

Hydra-PSIE Daily Metrics Report: 2026-11-05

Starea sistemului și metrici cheie

Jurnal de reconfigurare și evenimente (ultimele 24h)

Despre acest ecosistem

Pagini-cheie

Three ways a calendar tells you what changed

How do Google Calendar sync tokens work?

How does Outlook delta sync work?

How do you sync an iCloud or CalDAV calendar?

What about calendars with no change feed?

How calsdk handles it

FAQ

What happens when a Google Calendar sync token expires?

Can I use timeMin with a Google sync token?

How does Microsoft Graph report deleted events in a delta query?

Does iCloud support incremental sync?

When should I save the sync cursor?

Finding a free slot across Google, Outlook and iCloud

How does Google Calendar report busy time?

Why not use Outlook's getSchedule?

How do you get busy time from iCloud and CalDAV?

What catches people out on every provider?

How do you merge busy times without double booking?

How calsdk does it

FAQ

Does Outlook have a free/busy API?

Does iCloud support free/busy queries?

Should tentative and declined events count as busy?

What should happen when one calendar can't be read?

How many calendars can one Google freeBusy request check?

One standup, three calendar APIs

How does Google Calendar store a recurring event?

How does Outlook store a recurring event?

How do iCloud and CalDAV store a recurring event?

How do you put one API over all three?

What `reactionsTotal` still is not

`views` is a running counter, and my own history proves it

`postId` is not a post count — it is a message counter, and albums eat it

Step 3: date window (`since` / `until`)