<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: thechelsuk</title>
    <description>The latest articles on DEV Community by thechelsuk (@thechelsuk).</description>
    <link>https://dev.to/thechelsuk</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4216098%2F836cfe63-1b7f-41ae-9f70-54f21621becb.png</url>
      <title>DEV Community: thechelsuk</title>
      <link>https://dev.to/thechelsuk</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/thechelsuk"/>
    <language>en</language>
    <item>
      <title>How I Build 800+ Pages of Local Open Data with Jekyll, Python and a Raspberry Pi</title>
      <dc:creator>thechelsuk</dc:creator>
      <pubDate>Thu, 19 Nov 2026 21:42:32 +0000</pubDate>
      <link>https://dev.to/thechelsuk/how-i-build-800-pages-of-local-open-data-with-jekyll-python-and-a-raspberry-pi-of8</link>
      <guid>https://dev.to/thechelsuk/how-i-build-800-pages-of-local-open-data-with-jekyll-python-and-a-raspberry-pi-of8</guid>
      <description>&lt;p&gt;&lt;a href="https://cheltenham-od.uk" rel="noopener noreferrer"&gt;Cheltenham Open Data&lt;/a&gt; is a free site I run for my home town. It takes public data that's spread across council PDFs, government spreadsheets and APIs and puts it on plain, fast pages: &lt;a href="https://cheltenham-od.uk/cheltenham-fuel-prices" rel="noopener noreferrer"&gt;fuel prices&lt;/a&gt;, &lt;a href="https://cheltenham-od.uk/cheltenham-planning-applications" rel="noopener noreferrer"&gt;planning applications&lt;/a&gt;, &lt;a href="https://cheltenham-od.uk/cheltenham-flood-warnings" rel="noopener noreferrer"&gt;flood warnings&lt;/a&gt;, &lt;a href="https://cheltenham-od.uk/cheltenham-bus-data" rel="noopener noreferrer"&gt;bus disruptions&lt;/a&gt;, &lt;a href="https://cheltenham-od.uk/cheltenham-care-homes" rel="noopener noreferrer"&gt;care home ratings&lt;/a&gt; and a lot more.&lt;/p&gt;

&lt;p&gt;The sitemap now has a little over 800 URLs. I didn't write 800 pages. About 225 are hand-written Markdown, around 100 are news and analysis posts, and the rest, over 500, are generated at build time from data. There's no database, no server and no framework beyond Jekyll. This post goes through how it works, and the parts I'd do the same way again.&lt;/p&gt;

&lt;p&gt;I've written before about &lt;a href="https://thechels.uk/i-built-a-free-open-data-site-for-cheltenham" rel="noopener noreferrer"&gt;why I built it&lt;/a&gt; and &lt;a href="https://thechels.uk/cheltenham-open-data-2-0" rel="noopener noreferrer"&gt;why tools beat blog posts&lt;/a&gt; for a local site. This one is just about the technology.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Shape of It
&lt;/h2&gt;

&lt;p&gt;Every data page follows the same pipeline:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;source API / CSV / PDF
        │
        ▼
_python/&amp;lt;name&amp;gt;.py      fetch, clean, filter to Cheltenham
        │
        ▼
_data/&amp;lt;name&amp;gt;.json      committed to git
        │
        ▼
_layouts/&amp;lt;group&amp;gt;/&amp;lt;name&amp;gt;.html   Liquid turns it into tables, maps, charts
        │
        ▼
_pages/&amp;lt;group&amp;gt;/&amp;lt;name&amp;gt;.md       front matter + the words a person reads
        │
        ▼
GitHub Pages
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's it. Git is the database. Each fetcher writes a JSON file, a push triggers a build, and the build is the deploy.&lt;/p&gt;

&lt;p&gt;There are about 90 Python fetchers and nearly 150 JSON data files. Sources include ONS and Nomis, HM Land Registry, the Bus Open Data Service, National Rail, National Highways, the Environment Agency, Ofcom, the Department for Education, the CQC, the Food Standards Agency, the GOV.UK fuel price scheme and OpenStreetMap.&lt;/p&gt;

&lt;h2&gt;
  
  
  Fetchers: Boring on Purpose
&lt;/h2&gt;

&lt;p&gt;Each fetcher is a standalone script that writes one JSON file with the same envelope: the data plus &lt;code&gt;generated_at&lt;/code&gt;, &lt;code&gt;source&lt;/code&gt;, &lt;code&gt;source_url&lt;/code&gt; and &lt;code&gt;licence&lt;/code&gt;. That envelope means every page can print an honest "About This Data" line at the bottom without anyone having to remember to write one.&lt;/p&gt;

&lt;p&gt;The most useful helper in the whole repo is ten lines long. &lt;code&gt;write_json&lt;/code&gt; refuses to write a file if the only thing that changed is the run time stamp:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;write_json&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="s2"&gt;"""Write payload as JSON, unless the file already holds the same data and
    only the run time stamps would change. Leaving the file alone means a run
    that finds nothing new makes no commit."""&lt;/span&gt;
    &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pathlib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;exists&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;old&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read_text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nc"&gt;ValueError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;old&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;_without_stamps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;old&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nf"&gt;_without_stamps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;
    &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;mkdir&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;parents&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;exist_ok&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write_text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;indent&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ensure_ascii&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;False&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;True&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That one decision pays off three times:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;A fetcher that runs every five minutes and finds nothing new makes no commit, so the git history stays readable.&lt;/li&gt;
&lt;li&gt;The "Last updated" date on each page can come straight from git (more on that below), and it's the day the data actually changed, not the day a cron job last ran.&lt;/li&gt;
&lt;li&gt;No pointless rebuilds.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Every fetcher also runs through a small wrapper that records whether it worked in a status file, and a &lt;code&gt;sources.json&lt;/code&gt; registry lists every source, its licence, its data files and how often each one should change. A pytest test fails if a data file isn't in the registry, so nothing gets added quietly without a licence attached.&lt;/p&gt;

&lt;h2&gt;
  
  
  Generating Hundreds of Pages Without Writing Them
&lt;/h2&gt;

&lt;p&gt;Jekyll has a little-used feature that does most of the heavy lifting here: &lt;strong&gt;generator plugins&lt;/strong&gt;. A plugin in &lt;code&gt;_plugins/&lt;/code&gt; can read &lt;code&gt;site.data&lt;/code&gt; and add pages to the site before rendering, with no front-matter file on disk.&lt;/p&gt;

&lt;p&gt;The site has three of these:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Care homes&lt;/strong&gt;: the monthly CQC fetcher gives each care home and home care service a slug, and a plugin makes &lt;a href="https://cheltenham-od.uk/cheltenham-care-homes" rel="noopener noreferrer"&gt;a page for each one&lt;/a&gt;. A newly registered home gets its page on the next build.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Schools&lt;/strong&gt;: the same pattern for &lt;a href="https://cheltenham-od.uk/cheltenham-schools" rel="noopener noreferrer"&gt;Ofsted inspection history&lt;/a&gt;, one page per school.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cheltenham Town FC&lt;/strong&gt;: a &lt;a href="https://cheltenham-od.uk/cheltenham-sports/cheltenham-town-fc" rel="noopener noreferrer"&gt;league history&lt;/a&gt; fetcher feeds pages for every season since 2000-01, every opponent and every referee. That alone is around 380 pages.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The pattern is roughly:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight ruby"&gt;&lt;code&gt;&lt;span class="k"&gt;module&lt;/span&gt; &lt;span class="nc"&gt;CareHomePages&lt;/span&gt;
  &lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Generator&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="nc"&gt;Jekyll&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nc"&gt;Generator&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;site&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
      &lt;span class="n"&gt;site&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"cqc"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"locations"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;to_a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;each&lt;/span&gt; &lt;span class="k"&gt;do&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="n"&gt;home&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;
        &lt;span class="n"&gt;site&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pages&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&amp;lt;&lt;/span&gt; &lt;span class="nc"&gt;CareHomePage&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;site&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;home&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
      &lt;span class="k"&gt;end&lt;/span&gt;
    &lt;span class="k"&gt;end&lt;/span&gt;
  &lt;span class="k"&gt;end&lt;/span&gt;
&lt;span class="k"&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The plugins also handle the bits that are easy to get wrong at scale. Titles and meta descriptions are built from a list of options and the longest one that fits the 60 or 160 character limit wins, so a short name gets a rich title and a long one still fits in search results.&lt;/p&gt;

&lt;p&gt;Because the site builds with &lt;code&gt;bundle exec jekyll build&lt;/code&gt; in GitHub Actions rather than the old locked-down GitHub Pages builder, custom plugins are fine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Plugins That Keep Pages Honest
&lt;/h2&gt;

&lt;p&gt;A few smaller plugins run after rendering and stop the generated HTML drifting from what's on screen:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;FAQ schema&lt;/strong&gt;: every page writes its FAQs the same way, a &lt;code&gt;## Frequently Asked Questions&lt;/code&gt; heading, then &lt;code&gt;###&lt;/code&gt; questions with bullet answers. A plugin reads the &lt;em&gt;rendered&lt;/em&gt; HTML, finds those pairs and writes the matching &lt;code&gt;FAQPage&lt;/code&gt; JSON-LD into the &lt;code&gt;&amp;lt;head&amp;gt;&lt;/code&gt;. The structured data can't disagree with the page because it's made from the page.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Data freshness&lt;/strong&gt;: one &lt;code&gt;git log&lt;/code&gt; pass dates every &lt;code&gt;_data&lt;/code&gt; file by its last commit. Each page shows "Last updated …" from that, and the sitemap's &lt;code&gt;&amp;lt;lastmod&amp;gt;&lt;/code&gt; is the later of the page's own last change and that of any data it shows. The deploy checks out full history (&lt;code&gt;fetch-depth: 0&lt;/code&gt; with &lt;code&gt;filter: blob:none&lt;/code&gt; to keep it fast), because on a shallow clone every file would get the same date.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Table tidying&lt;/strong&gt;: Kramdown's Markdown tables come out bare, so a plugin wraps them for sideways scrolling on phones, turns right-aligned columns into a &lt;code&gt;number&lt;/code&gt; class, and stops postcodes and dates from wrapping mid-value.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Navigation&lt;/strong&gt;: the menu, category pages, breadcrumbs, "In this section" rows and the A-Z footer all come from one YAML file per category. Adding a page to the nav means adding a few lines of YAML, and everything else follows.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Maps and the Front End
&lt;/h2&gt;

&lt;p&gt;Maps use &lt;a href="https://leafletjs.com" rel="noopener noreferrer"&gt;Leaflet&lt;/a&gt; 1.9.4 with OpenStreetMap tiles, each page with a small script of its own, about 80 of them in all, following the same IIFE shape. The &lt;a href="https://cheltenham-od.uk/cheltenham-crime-data/map" rel="noopener noreferrer"&gt;crime map&lt;/a&gt; shades small areas by crimes per 1,000 residents, and the &lt;a href="https://cheltenham-od.uk/cheltenham-schools/catchment-areas/primary" rel="noopener noreferrer"&gt;school catchment maps&lt;/a&gt; compute Voronoi regions in the browser.&lt;/p&gt;

&lt;p&gt;There's no framework and no build step for the JavaScript. Sass is compiled by Jekyll. There are no tracking cookies and no ad tech, just anonymised, cookie-free page counts.&lt;/p&gt;

&lt;p&gt;The same data also feeds free, self-updating &lt;a href="https://cheltenham-od.uk/widgets" rel="noopener noreferrer"&gt;data widgets&lt;/a&gt; that anyone can drop into their own site. They're just more Jekyll pages, built from the same JSON on every deploy.&lt;/p&gt;

&lt;h2&gt;
  
  
  Scheduling: Why a Raspberry Pi Runs the Show
&lt;/h2&gt;

&lt;p&gt;This is the part I'd tell anyone starting something similar.&lt;/p&gt;

&lt;p&gt;GitHub Actions' &lt;code&gt;schedule:&lt;/code&gt; trigger is best effort. In practice my "hourly" job was running once or twice a day, and five-minute jobs were nowhere near five minutes, with nothing in the UI to say so. For a site where &lt;a href="https://cheltenham-od.uk/cheltenham-fuel-prices" rel="noopener noreferrer"&gt;fuel prices&lt;/a&gt; and &lt;a href="https://cheltenham-od.uk/cheltenham-power-cuts" rel="noopener noreferrer"&gt;power cuts&lt;/a&gt; should be current, that's no good.&lt;/p&gt;

&lt;p&gt;So no workflow in the repo has a &lt;code&gt;schedule:&lt;/code&gt; any more. A Raspberry Pi at home does two jobs instead:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;It runs the fast feeds itself.&lt;/strong&gt; Bus disruptions, power cuts, &lt;a href="https://cheltenham-od.uk/cheltenham-train-times" rel="noopener noreferrer"&gt;train times&lt;/a&gt;, unplanned road closures, weather warnings and fuel prices run from the Pi every five minutes or hourly. Planning applications run there too, because the council's planning portal blocks GitHub's IP ranges.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It starts every other workflow on time,&lt;/strong&gt; using &lt;code&gt;workflow_dispatch&lt;/code&gt; from its crontab.
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;*/5 * * * *  flock -n /tmp/chelt-od.lock $PI/run.sh every-5-minutes
3 * * * *    flock -w 300 /tmp/chelt-od.lock $PI/run.sh hourly
36 */2 * * * $PI/pi/dispatch.sh refresh-every-2-hours.yml
23 4 * * MON $PI/pi/dispatch.sh refresh-weekly.yml
17 0 2 * *   $PI/pi/dispatch.sh refresh-monthly.yml
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each run pulls &lt;code&gt;main&lt;/code&gt;, runs the scripts in a manifest file, commits anything that changed and pushes, which triggers the Pages deploy. &lt;code&gt;flock&lt;/code&gt; stops two runs pulling or pushing at the same time. The manifests are plain text files read after the pull, so adding a script to the Pi is just a commit. There's no SSH-ing in.&lt;/p&gt;

&lt;p&gt;One small trick: &lt;code&gt;run.sh&lt;/code&gt; puts its whole body in a function called on the last line. Bash reads scripts as it runs them, so without that, a &lt;code&gt;git pull&lt;/code&gt; that changed &lt;code&gt;run.sh&lt;/code&gt; partway through could run half the old file and half the new one.&lt;/p&gt;

&lt;h3&gt;
  
  
  A Cloudflare Worker as the Watchdog
&lt;/h3&gt;

&lt;p&gt;A Pi on home Wi-Fi will go quiet sometimes. After each good run it posts a heartbeat to a &lt;a href="https://developers.cloudflare.com/workers/" rel="noopener noreferrer"&gt;Cloudflare Worker&lt;/a&gt;. Every five minutes the Worker checks the last check-in, and after two missed runs it takes over:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;it starts a fallback workflow that runs the five-minute feeds in Actions&lt;/li&gt;
&lt;li&gt;it starts any scheduled workflow that fell due while the Pi was down, unless GitHub shows it already ran&lt;/li&gt;
&lt;li&gt;it emails me and turns a Hue light in the house orange, checking GitHub's status page first so a GitHub outage isn't blamed on the Pi&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The next heartbeat hands control back. The Worker keeps its own copy of the schedule, and a test fails if it doesn't match the Pi's crontab, so the two can't drift apart.&lt;/p&gt;

&lt;h2&gt;
  
  
  Tests as Guard Rails
&lt;/h2&gt;

&lt;p&gt;There are over 70 pytest files. Most are ordinary fetcher tests, but the ones I value most check the &lt;em&gt;structure&lt;/em&gt; of the project:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;every data file is in the sources registry, with a licence&lt;/li&gt;
&lt;li&gt;every Pi script is in exactly one manifest&lt;/li&gt;
&lt;li&gt;no workflow has a GitHub &lt;code&gt;schedule:&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;the watchdog's schedule matches the crontab&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;None of these are clever. They exist because each one was a mistake I made once.&lt;/p&gt;

&lt;p&gt;CI also runs super-linter, CodeQL, dependency review and a weekly link checker, which matters on a site with thousands of outbound links to official sources.&lt;/p&gt;

&lt;h2&gt;
  
  
  Getting Found
&lt;/h2&gt;

&lt;p&gt;With 800 pages, search engines need help knowing what changed. After each deploy, a workflow compares sitemap &lt;code&gt;&amp;lt;lastmod&amp;gt;&lt;/code&gt; dates against the commit of the last successful deploy and sends only the new or changed URLs to &lt;a href="https://www.indexnow.org" rel="noopener noreferrer"&gt;IndexNow&lt;/a&gt;. Because &lt;code&gt;&amp;lt;lastmod&amp;gt;&lt;/code&gt; comes from real data changes, that list stays short and accurate.&lt;/p&gt;

&lt;p&gt;The repository is public on &lt;a href="https://github.com/thechelsuk/uk.cheltenham-od" rel="noopener noreferrer"&gt;GitHub&lt;/a&gt; and archived on Zenodo with a DOI, so researchers can cite it, and some have.&lt;/p&gt;

&lt;h2&gt;
  
  
  What It Costs
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Hosting: GitHub Pages, free&lt;/li&gt;
&lt;li&gt;Builds and most fetchers: GitHub Actions, free for a public repo&lt;/li&gt;
&lt;li&gt;Watchdog: Cloudflare Workers free tier&lt;/li&gt;
&lt;li&gt;Scheduling and fast feeds: a Raspberry Pi's electricity - most of this is solar-powered&lt;/li&gt;
&lt;li&gt;the domain name for less than $5 a year.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The real cost is time. To cover that, the plan is a small number of local businesses each &lt;a href="https://cheltenham-od.uk/sponsor" rel="noopener noreferrer"&gt;sponsoring one section&lt;/a&gt;, with a single "supported by" card and nothing that flashes, aiming at businesses who want to be seen in their community and support a growing source that quite frankly beats the local council's own site.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I'd Tell You If You're Building Something Similar
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Git makes a fine database for slow-changing public data.&lt;/strong&gt; You get history, diffs and rollback for free, and dates from &lt;code&gt;git log&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Don't write a file that hasn't changed.&lt;/strong&gt; It keeps commits, builds and "last updated" dates honest.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Don't trust a scheduler that won't tell you it's late.&lt;/strong&gt; A Raspberry Pi and a free Worker beat it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Generate pages from data, but write the important ones by hand.&lt;/strong&gt; The generated pages are useful long-tail; the hand-written intros are why people come back.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Test the structure, not just the code.&lt;/strong&gt; Most of my bugs were a file in the wrong list.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you live in or around Cheltenham, have a look at &lt;a href="https://cheltenham-od.uk" rel="noopener noreferrer"&gt;the site&lt;/a&gt; or sign up to &lt;a href="https://cheltenham-od.uk/newsletter" rel="noopener noreferrer"&gt;The Cheltenham Week Ahead&lt;/a&gt;, a free Monday email. If you're building something like it for your own town, I'd love to hear about it - maybe a register/webring. I write about this and other side projects on &lt;a href="https://thechels.uk" rel="noopener noreferrer"&gt;my blog&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>showdev</category>
      <category>raspberrypi</category>
      <category>jekyll</category>
      <category>python</category>
    </item>
    <item>
      <title>Why a 1Password `.env` Mount Made VS Code Hang</title>
      <dc:creator>thechelsuk</dc:creator>
      <pubDate>Thu, 19 Nov 2026 21:20:00 +0000</pubDate>
      <link>https://dev.to/thechelsuk/why-a-1password-env-mount-made-vs-code-hang-2knm</link>
      <guid>https://dev.to/thechelsuk/why-a-1password-env-mount-made-vs-code-hang-2knm</guid>
      <description>&lt;p&gt;I use 1Password Environments to provide project secrets through a locally mounted &lt;code&gt;.env&lt;/code&gt;&lt;br&gt;
file. In one repository, enabling the mount made VS Code's Copilot Chat and Source Control&lt;br&gt;
views very slow or appear to hang resulting in the inability to push code.&lt;br&gt;
The same setup worked fine in other repositories.&lt;/p&gt;

&lt;p&gt;The problem was not the secret values. An empty 1Password-mounted &lt;code&gt;.env&lt;/code&gt; caused the same&lt;br&gt;
behaviour, and disabling the mount for this repository made VS Code responsive again.&lt;/p&gt;
&lt;h2&gt;
  
  
  What the Logs Showed
&lt;/h2&gt;

&lt;p&gt;VS Code reported an error reading the mounted file:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Unable to read file &lt;code&gt;.../.env&lt;/code&gt; (Unavailable (FileSystemError): An unknown error occurred.)&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Around the same time, VS Code reported that its extension host was briefly unresponsive.&lt;br&gt;
An extension-host profile showed the Python extensions responding to an environment-file&lt;br&gt;
change: &lt;code&gt;ms-python.vscode-python-envs&lt;/code&gt; was handling a file event, while &lt;code&gt;ms-python.python&lt;/code&gt;&lt;br&gt;
called &lt;code&gt;getEnvFile&lt;/code&gt; and &lt;code&gt;getEnvironmentVariablesSync&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The Python extension's default &lt;code&gt;python.envFile&lt;/code&gt; is &lt;code&gt;${workspaceFolder}/.env&lt;/code&gt;. It was&lt;br&gt;
therefore trying to use the mounted path as a source of Python environment variables.&lt;br&gt;
1Password's local &lt;code&gt;.env&lt;/code&gt; mounts use a Unix named pipe rather than an ordinary file. The&lt;br&gt;
failed VS Code read and the Python extension activity pointed to an interaction between&lt;br&gt;
that default lookup and the mounted path.&lt;/p&gt;

&lt;p&gt;The profile was only a short CPU sample, so it did not prove what caused every delay in&lt;br&gt;
Copilot or Source Control. The practical confirmation came from changing the Python&lt;br&gt;
extension's env-file path: VS Code stopped hanging with the 1Password mount enabled.&lt;/p&gt;
&lt;h2&gt;
  
  
  The Fix
&lt;/h2&gt;

&lt;p&gt;I configured the Python extension to use a separate, ordinary empty file instead of the&lt;br&gt;
mounted &lt;code&gt;.env&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;In the workspace's &lt;code&gt;.vscode/settings.json&lt;/code&gt;, add this setting inside the existing JSON&lt;br&gt;
object. Keep any settings already in the file, separating entries with commas:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"python.envFile"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"${workspaceFolder}/.vscode/python.env"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then, from the repository root, create the empty file:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;touch .vscode/python.env
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After reloading VS Code and enabling the 1Password mount again, the hangs stopped. The&lt;br&gt;
mounted &lt;code&gt;.env&lt;/code&gt; remained available to the repository; this change stopped the Python&lt;br&gt;
extension from using that path as its environment-variable file.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Trade-Off
&lt;/h2&gt;

&lt;p&gt;Python processes launched by the VS Code Python extension no longer automatically receive&lt;br&gt;
variables from the mounted &lt;code&gt;.env&lt;/code&gt; through the &lt;code&gt;python.envFile&lt;/code&gt; setting. Scripts that load&lt;br&gt;
&lt;code&gt;.env&lt;/code&gt; themselves are unaffected by this setting, as are Git, Copilot and shell sessions.&lt;br&gt;
If a debug or test configuration depends on VS Code injecting values from &lt;code&gt;.env&lt;/code&gt;, provide&lt;br&gt;
those values another way for that launch. I didn't have any such configurations and&lt;br&gt;
&lt;code&gt;python.env&lt;/code&gt; remains empty.&lt;/p&gt;

&lt;p&gt;In this repository, &lt;code&gt;.vscode/&lt;/code&gt; is Git-ignored, so the setting and helper file are local to&lt;br&gt;
the checkout and do not need to be committed. Other repositories may have a different&lt;br&gt;
trigger; this fix addresses the Python extension's default &lt;code&gt;.env&lt;/code&gt; lookup in this workspace.&lt;/p&gt;

&lt;h2&gt;
  
  
  Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://www.1password.dev/environments/local-env-file" rel="noopener noreferrer"&gt;1Password: Access secrets through local &lt;code&gt;.env&lt;/code&gt; files&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://code.visualstudio.com/docs/python/environments" rel="noopener noreferrer"&gt;VS Code: Python environments&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://thechels.uk/1password-env-hang-in-vscode.md" rel="noopener noreferrer"&gt;originally published on thechels.uk&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>1password</category>
      <category>vscode</category>
      <category>env</category>
      <category>python</category>
    </item>
  </channel>
</rss>
