DEV Community

Cover image for Calendar feeds that only change when an event does
Elliot Marsh
Elliot Marsh

Posted on

Calendar feeds that only change when an event does

#python #ical #webdev #calendar

A subscribable calendar is one of the most useful things a site about events can offer. A reader adds "food festivals in Oslo" to Google, Apple or Outlook calendar once, and every edition shows up on their own calendar with a link back to the page. It also turns out to be one of the easiest feeds to get subtly wrong, because calendar clients are fussy and quiet about it: a feed that churns or breaks a rule does not throw an error, it just behaves oddly for subscribers.

We publish several hundred .ics feeds at TableJourney (one per city, per country and per theme), all regenerated every day from the same data as the pages. This post covers the rules that kept them stable.

Goal: a byte-identical file unless an event changed

Our site is static and every generated file is committed and pushed. A feed that differs every day for no reason means a commit every day, cache invalidation every day and, worse, calendar clients re-processing events that did not change. So the rule we set was simple: rebuilding the feed with no data change must produce the same bytes.

Three fields usually break that.

1. UID: derive it from what the event is

The UID must identify the same event across rebuilds. Do not use a random UUID or a database row id that can change on reimport. We build it from the page path and the edition year:

slug = r["url"].strip("/").replace("/", "-")
uid = f"{slug}-{r['start'].year}@tablejourney.com"
Enter fullscreen mode Exit fullscreen mode

The year matters: next year's edition is a different event, so it gets a different UID, and this year's stays in subscribers' calendars as history instead of being moved forward.

2. DTSTAMP: the date you checked, not the render time

RFC 5545 requires DTSTAMP, and most generators fill it with "now". That makes every build different. We use the date the entry's facts were last checked against the organiser's source, which only moves when someone actually re-verifies the event:

v = e.get("verified") or {}
stamp = str(v.get("checked_on") or r["start"].isoformat()).replace("-", "")[:8]
lines.append(f"DTSTAMP:{stamp}T000000Z")
Enter fullscreen mode Exit fullscreen mode

This is also more honest: the stamp says when the information was confirmed.

3. Write only when the bytes differ 

def write_if_changed(path, text):
    data = text.encode("utf-8")
    if path.exists() and path.read_bytes() == data:
        return False
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
    return True
Enter fullscreen mode Exit fullscreen mode

With stable UIDs and stamps, this turns several hundred feeds into a handful of changed files on a normal day.

All-day events: DTEND is exclusive

A three-day festival from 16 to 18 October is an all-day event. With VALUE=DATE, DTEND is the day after the last day, not the last day:

lines += [
    f"DTSTART;VALUE=DATE:{r['start']:%Y%m%d}",
    f"DTEND;VALUE=DATE:{r['end'] + dt.timedelta(days=1):%Y%m%d}",
]
Enter fullscreen mode Exit fullscreen mode

Get this wrong and every multi-day event loses its final day in some clients and not others.

Escaping text

Commas, semicolons, backslashes and newlines in text values must be escaped, and the order matters (backslashes first):

def esc(s):
    return (
        (s or "")
        .replace("\\", "\\\\")
        .replace(";", "\\;")
        .replace(",", "\\,")
        .replace("\r\n", "\\n")
        .replace("\n", "\\n")
    )
Enter fullscreen mode Exit fullscreen mode

Addresses are full of commas, which is why this shows up in the very first feed you test.

Line folding counts octets, not characters

Lines longer than 75 octets must be folded: a CRLF followed by a single space. The catch is that the limit is in octets, so a description with "ø" or "é" folds earlier than its character count suggests, and you must never split inside a multi-byte character:

def fold(line):
    b = line.encode("utf-8")
    if len(b) <= 75:
        return line
    out, cur = [], b""
    for ch in line:
        e = ch.encode("utf-8")
        if len(cur) + len(e) > (75 if not out else 74):
            out.append(cur.decode("utf-8"))
            cur = b""
        cur += e
    out.append(cur.decode("utf-8"))
    return "\r\n ".join(out)
Enter fullscreen mode Exit fullscreen mode

Continuation lines get 74 octets because the leading space counts. Iterating by character and measuring each character's encoded length is what keeps multi-byte letters whole.

Line endings and headers

Every line ends in CRLF, including the last. A few calendar-level properties help clients:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//TableJourney//Food festival calendar//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Food festivals in Oslo (TableJourney)
REFRESH-INTERVAL;VALUE=DURATION:P1D
X-PUBLISHED-TTL:P1D
Enter fullscreen mode Exit fullscreen mode

REFRESH-INTERVAL and X-PUBLISHED-TTL ask clients to poll daily; some honour one, some the other, some neither (Google refreshes on its own schedule).

You can check the real output, CRLFs included:

curl -s https://tablejourney.com/calendar/norway/oslo.ics | head -n 20 | cat -A
Enter fullscreen mode Exit fullscreen mode

Status, not deletion

When an organiser postpones or has not confirmed dates, keep the event and say so:

status = {"tentative": "TENTATIVE", "cancelled": "CANCELLED"}.get(e.get("event_status") or "", "CONFIRMED")
lines.append(f"STATUS:{status}")
Enter fullscreen mode Exit fullscreen mode

A cancelled event that silently disappears from a feed leaves a ghost in some subscribers' calendars. A CANCELLED status removes it properly.

Only events with a known day

An event known only as "sometime in May" is not a calendar event. We leave it out of the feeds rather than inventing a date, and the page carries it instead. Every city with an events page still gets a feed, even an empty one, so the subscribe buttons on its page never point at a 404 and the feed fills itself when a dated event arrives.

Subscribe links

The last mile is a set of buttons. Apple Calendar takes a webcal:// URL directly; Google and Outlook take the feed URL as a parameter:

https_url = "https://tablejourney.com" + feed_path
webcal = "webcal://tablejourney.com" + feed_path
google = "https://calendar.google.com/calendar/r?cid=" + quote(webcal, safe="")
outlook = "https://outlook.live.com/calendar/0/addfromweb?url=" + quote(https_url, safe="") + "&name=" + quote(label, safe="")
Enter fullscreen mode Exit fullscreen mode

One source of truth

The biggest decision was not in the feed code at all: the feeds are built from the same function that builds the calendar pages, the API answers and the open dataset. A feed that disagrees with the page it links to is worse than no feed.

All of our feeds are listed at https://tablejourney.com/calendar/ if you want to see the output in your own calendar app. What other client quirks have bitten you? I am collecting them.

Top comments (0)