DEV Community

ushiro
ushiro

Posted on

The iCalendar Spec Says 75 Octets, Not 75 Characters

I run AI Change Watch, a small independent project that
crawls what 15 AI vendors publish about their own models — deprecation tables, lifecycle pages, pricing
and SDK releases — and records every time one of them changes.

One of the things it publishes is a subscribable calendar: every announced model shutdown, as .ics,
so a date the vendor moves updates in your calendar instead of in a changelog you forgot to read.

The English feed worked immediately. The Japanese one was rejected.

Same code. Same events. The only difference was the language of the text inside.

The line that RFC 5545 actually specifies

Section 3.1:

Lines of text SHOULD NOT be longer than 75 octets, excluding the line break.

Not 75 characters. 75 octets.

For ASCII those are the same number, which is exactly why this survives every test you are likely to
write. SUMMARY:gpt-4-32k shutdown (OpenAI) is 36 characters and 36 bytes. Nothing to notice.

Then the same field comes back translated:

SUMMARY:code-davinci-001 提供終了(OpenAI)
Enter fullscreen mode Exit fullscreen mode

That is 30 characters. It is 49 octets — Japanese runs about 3 bytes per character in UTF-8. A
folder that counts characters looks at 30, decides no fold is needed, and emits a line that is legal by
its own arithmetic and illegal by the spec's.

Longer titles cross 75 octets while still well under 75 characters, and that is the line the parser
rejects.

Why counting characters isn't the only bug

The obvious fix — count bytes instead — introduces a second one if you write it the obvious way.

// Still broken.
const bytes = Buffer.from(line, 'utf8');
for (let i = 0; i < bytes.length; i += 75) {
  out.push(bytes.subarray(i, i + 75).toString('utf8'));
}
Enter fullscreen mode Exit fullscreen mode

Slicing a UTF-8 buffer at a fixed offset cuts through the middle of a character. Byte 75 lands in
the second of the three bytes that make up 終, and you emit half a codepoint on one line and the other
half on the next. Some parsers replace it with U+FFFD, some abort the file.

So the fold has to be counted in bytes but taken at character boundaries:

export function foldLine(line) {
  const enc = new TextEncoder();
  if (enc.encode(line).length <= 75) return line;

  const out = [];
  let cur = '', curBytes = 0;
  // The first line gets 75 octets; every continuation gets 74, because the leading
  // space that marks it as a continuation counts toward the limit too.
  let limit = 75;

  for (const ch of line) {          // iterating a string yields whole codepoints
    const n = enc.encode(ch).length;
    if (curBytes + n > limit) {
      out.push(cur);
      cur = ''; curBytes = 0; limit = 74;
    }
    cur += ch;
    curBytes += n;
  }
  if (cur) out.push(cur);

  return out.map((l, i) => (i === 0 ? l : ` ${l}`)).join('\r\n');
}
Enter fullscreen mode Exit fullscreen mode

Three things in there are easy to leave out and each one produces a file that mostly works:

for (const ch of line), not line[i]. Indexing a JS string walks UTF-16 code units, so an emoji
or any astral-plane character gets split at the surrogate pair. Iterating with for...of yields whole
codepoints.

The limit drops from 75 to 74 after the first line. The continuation marker is a single leading
space and it counts. Keep the limit at 75 and every folded line is one octet over — which is the same
bug you just fixed, only harder to see because it only shows up on lines long enough to fold.

\r\n, not \n. RFC 5545 wants CRLF. Plenty of parsers tolerate bare LF, right up until one
doesn't.

The other one: your UID is not a display string

An .ics feed served in ten languages raises a question the spec answers but does not warn you about:
what is the UID?

If you build it from the localized summary, the same shutdown gets a different UID in every language.
Subscribe to two of them and your calendar shows two entries for one event, forever, with no way to
tell they are the same thing.

So the UID has to be keyed on the entity, never the presentation:

UID:shutdown-o1-preview@aichangewatch.com
UID:shutdown-davinci-002@aichangewatch.com
Enter fullscreen mode Exit fullscreen mode

Model slug, no locale. I checked the two live feeds while writing this:

en feed:  262 UIDs
ja feed:  262 UIDs
shared:   262            ← identical sets
Enter fullscreen mode Exit fullscreen mode

Which means a person subscribed to both gets one entry per shutdown, not two. The second subscription
overwrites the first rather than duplicating it. That is the correct failure mode — a collision beats
a double-booking, because a duplicate calendar entry is something the user has to notice and clean up by
hand.

It also means you cannot use the UID to carry the language. If you need that, it belongs in the calendar
name (X-WR-CALNAME), not the identity.

What I check now

The whole class is "a spec said octets and I read characters", and it is worth one assertion in a test
rather than a careful reading:

const body = renderCalendar(events);
for (const line of body.split('\r\n')) {
  expect(Buffer.byteLength(line, 'utf8')).toBeLessThanOrEqual(75);
}
Enter fullscreen mode Exit fullscreen mode

Run it against the localized feed, not the English one. The English feed cannot fail this test,
which is precisely why it is not the one to run it on. Mine currently reports:

en feed:  6,545 lines,   0 over 75 octets,  506 continuation lines
ja feed:  6,606 lines,   0 over 75 octets,  567 continuation lines
Enter fullscreen mode Exit fullscreen mode

The 61 extra lines in the Japanese feed are the folds the English one does not need. That gap is the
bug, made visible: same events, same code, more lines — because the same sentences take more bytes.


The tracker this came out of is at aichangewatch.com — the
shutdown calendar it generates is at
/deprecations/calendar.ics.

Top comments (0)