DEV Community

Cover image for From X/Twitter Bookmarks to a Searchable Static Site
Bramandia G Adam
Bramandia G Adam Subscriber

Posted on

From X/Twitter Bookmarks to a Searchable Static Site

#javascript #powershell #cloudflare #webdev

I wanted my X bookmarks to feel like a personal archive. The saved posts lived inside one account, the useful parts were hard to search, and the workflow offered no public reading surface.

This project turns that archive into a static site. A Windows machine reads the active Firefox session twice each day. Field Theory downloads the bookmark data into a local JSONL cache. A PowerShell runner creates export files and pushes them into a GitHub repository. Cloudflare Pages builds the frontend from that repository and serves it at signals.asynchronope.my.id.

Signals

The site stays static. Its browser code fetches JSON files. The request path contains static files served by Pages.

The project has two repositories:

  • Field Theory CLI, which owns the local sync and export pipeline.
  • my-twitter-bookmarks, a private repository that owns the static frontend and the generated fieldtheory/ snapshot.

The local checkout used while writing this article contains both projects. The frontend checkout sits under .tmp-my-twitter-bookmarks.

The whole flow 

Firefox session on Windows
        |
        | 07:00 and 19:00 local time
        v
Windows Task Scheduler
        |
        v
sync-bookmarks-to-github.ps1
        |
        +--> node bin/ft.mjs sync
        |        |
        |        v
        |    ~/.ft-bookmarks/bookmarks.jsonl
        |
        +--> node bin/ft.mjs site-export
        |        |
        |        v
        |    fieldtheory/site/*.json
        |
        +--> git commit and git push
                 |
                 v
        my-twitter-bookmarks repository
                 |
                 v
        Cloudflare Pages build: npm run build
                 |
                 v
        dist/index.html + dist/data/*.json
                 |
                 v
        signals.asynchronope.my.id
Enter fullscreen mode Exit fullscreen mode

The local machine owns the browser session and the source cache. GitHub carries the generated snapshot. Cloudflare Pages serves the public files.

Repository layout

The Field Theory repository has the parts below.

fieldtheory/
  bin/
    ft.mjs
  src/
    cli.ts
    graphql-bookmarks.ts
    bookmarks-db.ts
    bookmarks-site-export.ts
    paths.ts
  scripts/
    automation/
      setup-bookmark-sync.ps1
      sync-bookmarks-to-github.ps1
      export_bookmarks_xlsx.py
  output/
  tests/
Enter fullscreen mode Exit fullscreen mode

The separate frontend repository has this shape:

my-twitter-bookmarks/
  fieldtheory/
    site/
      manifest.json
      search-index.json
      topics.json
      stats.json
      bookmarks.index.json
      bookmarks.full.jsonl
  web/
    index.html
    app.js
    styles.css
    build.mjs
    _headers
  package.json
Enter fullscreen mode Exit fullscreen mode

fieldtheory/site is the handoff point. The exporter writes the data there. The static build copies that directory into dist/data.

How Field Theory gets the bookmarks

The CLI entry point is bin/ft.mjs.

#!/usr/bin/env node
import { buildCli } from '../dist/cli.js';

await buildCli().parseAsync(process.argv);
Enter fullscreen mode Exit fullscreen mode

The production repository builds TypeScript into dist before the CLI runs. The scheduled runner calls the local entry point with Node:

node D:\Repo\fieldtheory\bin\ft.mjs sync --yes --browser firefox
Enter fullscreen mode Exit fullscreen mode

Field Theory reads cookies from the configured Firefox profile, uses the active X session for the bookmark request, and writes records into the data directory.

The default data location comes from src/paths.ts:

import path from 'node:path';
import os from 'node:os';
import fs from 'node:fs';

export function dataDir(): string {
  const override = process.env.FT_DATA_DIR;
  if (override) return override;
  return path.join(os.homedir(), '.ft-bookmarks');
}

export function twitterBookmarksCachePath(): string {
  return path.join(dataDir(), 'bookmarks.jsonl');
}

export function twitterBookmarksMetaPath(): string {
  return path.join(dataDir(), 'bookmarks-meta.json');
}

export function twitterBookmarksIndexPath(): string {
  return path.join(dataDir(), 'bookmarks.db');
}
Enter fullscreen mode Exit fullscreen mode

The resulting local directory looks like this:

C:\Users\your-user\.ft-bookmarks\
  bookmarks.jsonl
  bookmarks-meta.json
  bookmarks.db
  site\
Enter fullscreen mode Exit fullscreen mode

The JSONL file stores one bookmark record per line. SQLite gives the CLI its local full-text search index. The website uses the generated JSON files, so the database stays on the source machine.

The 7 AM and 7 PM schedule

The schedule lives in setup-bookmark-sync.ps1.

These parameters define the local times:

param(
  [string]$PrimaryMorningTime = '07:00',
  [string]$PrimaryEveningTime = '19:00'
)
Enter fullscreen mode Exit fullscreen mode

The installer turns those values into daily triggers:

$morningAt = [datetime]::Today.Add([timespan]::Parse($PrimaryMorningTime))
$eveningAt = [datetime]::Today.Add([timespan]::Parse($PrimaryEveningTime))

$primaryTriggerMorning = New-ScheduledTaskTrigger -Daily -At $morningAt
$primaryTriggerEvening = New-ScheduledTaskTrigger -Daily -At $eveningAt
$primaryTriggers = @($primaryTriggerMorning, $primaryTriggerEvening)

$recoveryTrigger = New-ScheduledTaskTrigger -AtLogOn -User $env:USERNAME
Enter fullscreen mode Exit fullscreen mode

The primary task runs twice per day. The recovery task runs when the user logs in. Its action receives -SkipIfRecentSuccessHours 6, which prevents a login from starting a second sync shortly after a successful scheduled run.

The task settings live in the same installer:

$settings = New-ScheduledTaskSettingsSet `
  -StartWhenAvailable `
  -WakeToRun `
  -AllowStartIfOnBatteries `
  -DontStopIfGoingOnBatteries `
  -RunOnlyIfNetworkAvailable `
  -ExecutionTimeLimit (New-TimeSpan -Hours 2) `
  -MultipleInstances IgnoreNew `
  -RestartCount 3 `
  -RestartInterval (New-TimeSpan -Minutes 30)
Enter fullscreen mode Exit fullscreen mode

The task action points to the runner script:

$syncScriptPath = Join-Path $SourceRepoPath 'scripts\automation\sync-bookmarks-to-github.ps1'

$primaryAction = New-ScheduledTaskAction `
  -Execute $powershellExe `
  -Argument (New-TaskActionArgument `
    -ScriptPath $syncScriptPath `
    -ConfigPath $configPath `
    -TriggerReason 'scheduled')

$recoveryAction = New-ScheduledTaskAction `
  -Execute $powershellExe `
  -Argument (New-TaskActionArgument `
    -ScriptPath $syncScriptPath `
    -ConfigPath $configPath `
    -TriggerReason 'logon-recovery' `
    -SkipIfRecentSuccessHours 6)
Enter fullscreen mode Exit fullscreen mode

The registration names are:

Register-BookmarkTask `
  -TaskPath '\FieldTheory\' `
  -TaskName 'Bookmark Sync' `
  -Triggers $primaryTriggers `
  -Action $primaryAction `
  -Settings $settings `
  -Principal $principal `
  -Description 'Sync Field Theory bookmarks to GitHub at 07:00 and 19:00 local time.'

Register-BookmarkTask `
  -TaskPath '\FieldTheory\' `
  -TaskName 'Bookmark Sync Recovery' `
  -Triggers @($recoveryTrigger) `
  -Action $recoveryAction `
  -Settings $settings `
  -Principal $principal `
  -Description 'Catch up Field Theory bookmark sync after logon if the last successful run is stale.'
Enter fullscreen mode Exit fullscreen mode

Run the installer from an elevated PowerShell window:

powershell -ExecutionPolicy Bypass `
  -File D:\Repo\fieldtheory\scripts\automation\setup-bookmark-sync.ps1 `
  -SourceRepoPath D:\Repo\fieldtheory `
  -FirefoxProfileDir 'C:\Users\your-user\AppData\Roaming\Mozilla\Firefox\Profiles\your-profile' `
  -RepoUrl 'https://github.com/your-account/my-twitter-bookmarks.git' `
  -RunNow
Enter fullscreen mode Exit fullscreen mode

The installer writes its configuration here:

C:\Users\your-user\.ft-bookmarks\automation\github-sync\bookmark-sync.config.psd1
Enter fullscreen mode Exit fullscreen mode

A safe configuration file contains machine paths and repository settings. Browser cookies remain inside the Firefox profile.

@{
  SourceRepoPath = 'D:\Repo\fieldtheory'
  DataDir = 'C:\Users\your-user\.ft-bookmarks'
  RepoUrl = 'https://github.com/your-account/my-twitter-bookmarks.git'
  RepoBranch = 'main'
  StateDir = 'C:\Users\your-user\.ft-bookmarks\automation\github-sync'
  Browser = 'firefox'
  FirefoxProfileDir = 'C:\Users\your-user\AppData\Roaming\Mozilla\Firefox\Profiles\your-profile'
  ManagedRoot = 'fieldtheory'
  CommitHeartbeat = $true
}
Enter fullscreen mode Exit fullscreen mode

Check the tasks with:

Get-ScheduledTask -TaskPath '\FieldTheory\' |
  Select-Object TaskPath, TaskName, State
Enter fullscreen mode Exit fullscreen mode

Inspect a task’s next run time with:

Get-ScheduledTaskInfo -TaskPath '\FieldTheory\' -TaskName 'Bookmark Sync' |
  Select-Object LastRunTime, NextRunTime, LastTaskResult
Enter fullscreen mode Exit fullscreen mode

StartWhenAvailable lets Windows catch up after a missed start. A powered-off machine pauses the scheduled process. The logon task gives the next session a recovery path.

The runner that creates the archive snapshot

The production runner lives in sync-bookmarks-to-github.ps1. It has a lock file, retry handling, status output, optional Telegram messages, CSV export, XLSX export, and website export.

The central command array is short:

$ftArguments = @(
  (Join-Path $config.SourceRepoPath 'bin/ft.mjs'),
  'sync',
  '--yes',
  '--browser', $config.Browser
)

if (-not [string]::IsNullOrWhiteSpace([string]$config.FirefoxProfileDir)) {
  $ftArguments += @('--firefox-profile-dir', [string]$config.FirefoxProfileDir)
}

$syncResult = Invoke-WithRetry -Label 'ft sync' -MaxAttempts 2 -DelaySeconds @(60) -RetryPatterns @(
  '429',
  '5\d\d',
  'ETIMEDOUT',
  'ECONN',
  'temporar',
  'rate limited',
  'Could not read .*Cookies database'
) -Operation {
  $result = Invoke-LoggedProcess `
    -FilePath 'node' `
    -Arguments $ftArguments `
    -WorkingDirectory $config.SourceRepoPath

  if ($result.ExitCode -ne 0) {
    throw "ft sync failed.`n$($result.StdErr)$($result.StdOut)"
  }

  return $result
}
Enter fullscreen mode Exit fullscreen mode

The runner then creates the portable exports:

$csvOutputPath = Join-Path $stagingRoot 'exports\bookmarks-full-latest.csv'

Invoke-WithRetry -Label 'ft export csv' -MaxAttempts 2 -DelaySeconds @(30) -Operation {
  $result = Invoke-LoggedProcess -FilePath 'node' -Arguments @(
    (Join-Path $config.SourceRepoPath 'bin/ft.mjs'),
    'export',
    '--format', 'csv',
    '--output', $csvOutputPath
  ) -WorkingDirectory $config.SourceRepoPath

  if ($result.ExitCode -ne 0) {
    throw "ft export failed.`n$($result.StdErr)$($result.StdOut)"
  }

  return $result
}

$siteOutputPath = Join-Path $stagingRoot 'site'

Invoke-WithRetry -Label 'ft site-export' -MaxAttempts 2 -DelaySeconds @(30) -Operation {
  $result = Invoke-LoggedProcess -FilePath 'node' -Arguments @(
    (Join-Path $config.SourceRepoPath 'bin/ft.mjs'),
    'site-export',
    '--output', $siteOutputPath
  ) -WorkingDirectory $config.SourceRepoPath

  if ($result.ExitCode -ne 0) {
    throw "ft site-export failed.`n$($result.StdErr)$($result.StdOut)"
  }

  return $result
}
Enter fullscreen mode Exit fullscreen mode

The generated managed tree looks like this:

fieldtheory/
  raw/
    bookmarks.jsonl
    bookmarks-meta.json
  exports/
    bookmarks-full-latest.csv
    bookmarks-full-latest.xlsx
  site/
    manifest.json
    bookmarks.index.json
    bookmarks.full.jsonl
    search-index.json
    topics.json
    stats.json
  status/
    last-sync.json
  README.md
Enter fullscreen mode Exit fullscreen mode

The remote update uses a temporary clone. The runner replaces the managed root, creates a commit, and pushes the configured branch:

$managedRootInClone = Join-Path $clonePath $config.ManagedRoot

Remove-PathIfExists -Path $managedRootInClone
Copy-Item -LiteralPath $stagingRoot -Destination $managedRootInClone -Recurse -Force

$addResult = Invoke-LoggedProcess `
  -FilePath 'git' `
  -Arguments @('add', '--all', $config.ManagedRoot) `
  -WorkingDirectory $clonePath

$commitMessage = if ($addedCount -gt 0) {
  "Sync bookmarks ($addedCount new) $($timestamp.ToString('yyyy-MM-dd HH:mm zzz'))"
} else {
  "Heartbeat bookmark sync $($timestamp.ToString('yyyy-MM-dd HH:mm zzz'))"
}

$commitResult = Invoke-LoggedProcess `
  -FilePath 'git' `
  -Arguments @('commit', '-m', $commitMessage) `
  -WorkingDirectory $clonePath

$pushResult = Invoke-LoggedProcess `
  -FilePath 'git' `
  -Arguments @('push', '-u', 'origin', $config.RepoBranch) `
  -WorkingDirectory $clonePath
Enter fullscreen mode Exit fullscreen mode

The GitHub repository receives the generated site data through this path. Git authentication belongs to the Windows machine. A GitHub token can live in the credential manager or in an SSH configuration, with the repository URL chosen to match that setup.

Keeping the frontend repository private

The repository can stay private. Cloudflare Pages supports both private and public GitHub repositories through its Git integration, then publishes the contents of the configured output directory. Visitors receive the generated dist files through the Pages domain. They receive the deployed site, while the Git history, source tree, and local sync scripts remain inside the private repository. See the Cloudflare Pages Git integration guide.

The public boundary sits at dist. Every field copied into dist/data becomes readable by a visitor who knows the file URL. Review the generated JSON before the first deployment and decide which bookmark fields belong in a public archive.

For a personal archive, place Cloudflare Access in front of the Pages project or publish the site behind an authenticated route. A public custom domain works well for a deliberately public reading archive. The article can describe either version with the same source layout.

The website export format

The CLI command is registered in src/cli.ts:

program
  .command('site-export')
  .description('Generate website-ready cleaned JSON artifacts from the raw bookmark cache')
  .option('--output <dir>', 'Write site artifacts to a directory')
  .action(safe(async (options) => {
    if (!requireData()) return;

    const result = await exportSiteDataset({
      output: options.output ? String(options.output) : path.join(dataDir(), 'site'),
    });

    console.log(`Exported ${result.exported} site bookmarks -> ${result.outputDir}`);
  }));
Enter fullscreen mode Exit fullscreen mode

The implementation lives in src/bookmarks-site-export.ts. It reads bookmarks.jsonl, removes duplicate tweet IDs, normalizes author fields, extracts links and media, creates topic buckets, and writes the artifact set.

One detail matters for dates. X's bookmark timeline response includes each post's created_at value and an opaque sortIndex used for timeline order. It does not include the moment when the account saved the post. The exporter keeps sortIndex as bookmarkOrder, uses postedAt for the post date, and uses syncedAt as capturedAt when the archive observed the record. The frontend labels that field Captured, so the public page does not turn an ordering token into a false calendar date.

The main transformation has this shape:

export async function exportSiteDataset(options: SiteExportOptions = {}): Promise<SiteExportResult> {
  if (!options.output) {
    throw new Error('Site export requires an output directory.');
  }

  const outputDir = path.resolve(options.output);
  const records = await readJsonLines<BookmarkRecord>(twitterBookmarksCachePath());
  const dataset = buildSiteDataset(records, { generatedAt: options.generatedAt });

  await ensureDir(outputDir);

  const files = {
    manifest: path.join(outputDir, 'manifest.json'),
    index: path.join(outputDir, 'bookmarks.index.json'),
    full: path.join(outputDir, 'bookmarks.full.jsonl'),
    search: path.join(outputDir, 'search-index.json'),
    topics: path.join(outputDir, 'topics.json'),
    stats: path.join(outputDir, 'stats.json'),
  };

  await writeJson(files.manifest, dataset.manifest);
  await writeJson(files.index, dataset.index);
  await writeJsonLines(files.full, dataset.bookmarks);
  await writeJson(files.search, dataset.search);
  await writeJson(files.topics, dataset.topics);
  await writeJson(files.stats, dataset.stats);

  return {
    exported: dataset.bookmarks.length,
    outputDir,
    files,
  };
}
Enter fullscreen mode Exit fullscreen mode

The manifest gives the frontend a small discovery layer:

{
  "schemaVersion": 1,
  "generatedAt": "2026-08-14T12:00:00.000Z",
  "source": "fieldtheory-cli",
  "bookmarkCount": 1842,
  "artifacts": {
    "index": "bookmarks.index.json",
    "full": "bookmarks.full.jsonl",
    "search": "search-index.json",
    "topics": "topics.json",
    "stats": "stats.json"
  }
}
Enter fullscreen mode Exit fullscreen mode

The browser loads search-index.json, topics.json, and stats.json. The full JSONL archive remains available for deeper tooling and future builds.

The static frontend

The frontend repository has a tiny package file:

{
  "name": "my-twitter-bookmarks-web",
  "version": "1.0.0",
  "private": true,
  "description": "Searchable public view over a private Field Theory bookmark archive.",
  "scripts": {
    "build": "node web/build.mjs"
  },
  "engines": {
    "node": ">=20"
  }
}
Enter fullscreen mode Exit fullscreen mode

The build script creates dist, copies the browser files, and places the generated archive under dist/data:

import { cp, mkdir, rm } from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

const webDir = path.dirname(fileURLToPath(import.meta.url));
const repoDir = path.resolve(webDir, '..');
const distDir = path.join(repoDir, 'dist');

await rm(distDir, { recursive: true, force: true });
await mkdir(distDir, { recursive: true });

for (const filename of ['index.html', 'styles.css', 'app.js', '_headers']) {
  await cp(path.join(webDir, filename), path.join(distDir, filename));
}

await cp(path.join(repoDir, 'fieldtheory', 'site'), path.join(distDir, 'data'), { recursive: true });

console.log(`Built bookmark frontend: ${distDir}`);
Enter fullscreen mode Exit fullscreen mode

The _headers file controls cache behavior for the changing manifest and the data files:

/data/manifest.json
  Cache-Control: no-cache

/data/*
  Cache-Control: public, max-age=300
Enter fullscreen mode Exit fullscreen mode

The page shell is plain HTML. It gives the browser code stable elements for the archive status, filters, result grid, and detail drawer.

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <meta name="theme-color" content="#f6f4ef" />
    <meta name="description" content="A living index of saved ideas, tools, research, and opportunities." />
    <title>Saved Signals</title>
    <link rel="stylesheet" href="./styles.css" />
  </head>
  <body>
    <div class="site-shell">
      <header class="topbar">
        <a class="brand" href="./" aria-label="Saved Signals home">
          <span class="brand-mark">SS</span>
          <span>
            <strong>Saved Signals</strong>
            <small>an indexed bookmark archive</small>
          </span>
        </a>
        <div class="sync-pill" id="sync-status" data-state="loading">
          <span class="status-dot" aria-hidden="true"></span>
          <span>Loading archive</span>
        </div>
      </header>

      <main>
        <section class="intro" aria-labelledby="page-title">
          <div class="intro-copy">
            <p class="eyebrow">Field notes from the scroll</p>
            <h1 id="page-title">Everything I saved because it made me stop.</h1>
            <p class="intro-text">A searchable window into the ideas, tools, people, and opportunities I keep coming back to.</p>
          </div>
          <div class="intro-aside">
            <p class="sync-label">Latest archive</p>
            <strong id="sync-date">Loading</strong>
            <a href="https://signals.asynchronope.my.id/" target="_blank" rel="noreferrer">Open deployed archive</a>
          </div>
        </section>

        <section class="stat-strip" aria-label="Archive statistics">
          <div class="stat-block">
            <span class="stat-value" id="total-count">-</span>
            <span class="stat-label">bookmarks</span>
          </div>
          <div class="stat-block">
            <span class="stat-value" id="author-count">-</span>
            <span class="stat-label">voices</span>
          </div>
          <div class="stat-block">
            <span class="stat-value" id="topic-count">-</span>
            <span class="stat-label">signals</span>
          </div>
          <div class="stat-block stat-note">
            <span class="stat-label">Archive span</span>
            <strong id="date-range">Loading</strong>
          </div>
        </section>

        <section class="workspace" aria-label="Bookmark explorer">
          <aside class="sidebar">
            <div class="sidebar-section search-section">
              <label class="search-label" for="search-input">Search the archive</label>
              <div class="search-box">
                <span class="search-icon" aria-hidden="true">/</span>
                <input id="search-input" type="search" placeholder="Try: agent memory, fellowship, design" autocomplete="off" />
                <button id="clear-search" class="icon-button" type="button" aria-label="Clear search" title="Clear search">x</button>
              </div>
              <p class="search-hint">Press <kbd>/</kbd> to focus</p>
            </div>

            <div class="sidebar-section">
              <label class="field-label" for="topic-select">Signal</label>
              <select id="topic-select"><option value="all">All signals</option></select>
            </div>

            <div class="sidebar-section">
              <label class="field-label" for="year-select">Posted year</label>
              <select id="year-select"><option value="all">All years</option></select>
            </div>

            <div class="sidebar-section">
              <label class="field-label" for="sort-select">Sort by</label>
              <select id="sort-select">
                <option value="saved">Latest in X order</option>
                <option value="posted">Recently posted</option>
                <option value="author">Author name</option>
              </select>
            </div>

            <div class="sidebar-section topic-section">
              <div class="section-heading">
                <span class="field-label">Most present</span>
                <span class="section-count" id="topic-list-count">-</span>
              </div>
              <div class="topic-list" id="topic-list"></div>
            </div>
          </aside>

          <section class="results-panel" aria-live="polite">
            <div class="results-toolbar">
              <div>
                <p class="eyebrow">Your index</p>
                <h2 id="results-title">All bookmarks</h2>
              </div>
              <span class="results-count" id="results-count">Loading</span>
            </div>
            <div class="active-filters" id="active-filters" hidden></div>
            <div class="results-grid" id="results-grid"></div>
            <div class="empty-state" id="empty-state" hidden>
              <span class="empty-mark">?</span>
              <h3>Nothing matched that search.</h3>
              <p>Try a broader phrase or clear one of the filters.</p>
            </div>
            <div class="error-state" id="error-state" hidden>
              <span class="empty-mark">!</span>
              <h3>The archive could not be loaded.</h3>
              <p id="error-message">Check the latest deployment and try again.</p>
            </div>
            <button class="load-more" id="load-more" type="button" hidden>Load more bookmarks</button>
          </section>
        </section>
      </main>

      <footer class="footer">
        <span>Synced from Field Theory twice daily.</span>
        <span>Built for slow browsing.</span>
      </footer>
    </div>

    <div class="drawer-backdrop" id="drawer-backdrop" hidden></div>
    <aside class="detail-drawer" id="detail-drawer" aria-label="Bookmark detail" aria-hidden="true" hidden>
      <div class="drawer-header">
        <span class="eyebrow">Bookmark detail</span>
        <button class="icon-button drawer-close" id="drawer-close" type="button" aria-label="Close detail">x</button>
      </div>
      <div id="drawer-content"></div>
    </aside>

    <script type="module" src="./app.js"></script>
  </body>
</html>
Enter fullscreen mode Exit fullscreen mode

The browser code keeps its state in memory. It fetches the manifest, finds the current artifact names, loads the search data, and renders cards from that array.

const DATA_BASE = './data/';
const PAGE_SIZE = 36;

const state = {
  bookmarks: [],
  topics: [],
  stats: {},
  query: '',
  topic: 'all',
  year: 'all',
  sort: 'saved',
  visible: PAGE_SIZE,
  activeId: null
};

const $ = (selector) => document.querySelector(selector);
const nf = new Intl.NumberFormat('en-US');
const els = {
  syncStatus: $('#sync-status'), syncDate: $('#sync-date'), totalCount: $('#total-count'), authorCount: $('#author-count'),
  topicCount: $('#topic-count'), dateRange: $('#date-range'), search: $('#search-input'), clearSearch: $('#clear-search'),
  topic: $('#topic-select'), year: $('#year-select'), sort: $('#sort-select'), topicList: $('#topic-list'),
  topicListCount: $('#topic-list-count'), resultsTitle: $('#results-title'), resultsCount: $('#results-count'),
  activeFilters: $('#active-filters'), grid: $('#results-grid'), empty: $('#empty-state'), error: $('#error-state'),
  errorMessage: $('#error-message'), loadMore: $('#load-more'), backdrop: $('#drawer-backdrop'), drawer: $('#detail-drawer'),
  drawerContent: $('#drawer-content'), drawerClose: $('#drawer-close')
};

function escapeHtml(value) {
  return String(value ?? '').replace(/[&<>"']/g, (character) => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' }[character]));
}

function safeUrl(value) {
  try {
    const url = new URL(String(value || ''));
    return url.protocol === 'https:' || url.protocol === 'http:' ? url.href : '';
  } catch {
    return '';
  }
}

function formatDate(value, fallback = 'Undated') {
  if (!value) return fallback;
  const date = new Date(value);
  return Number.isNaN(date.getTime()) ? fallback : new Intl.DateTimeFormat('en', { day: '2-digit', month: 'short', year: 'numeric' }).format(date);
}

function compactDate(value) {
  if (!value) return '';
  const date = new Date(value);
  return Number.isNaN(date.getTime()) ? '' : new Intl.DateTimeFormat('en', { month: 'short', year: 'numeric' }).format(date);
}

function capturedDate(bookmark) {
  return bookmark.capturedAt || bookmark.postedAt || '';
}

function compareSaved(a, b) {
  const aOrder = String(a.bookmarkOrder || '');
  const bOrder = String(b.bookmarkOrder || '');
  if (/^\d+$/.test(aOrder) && /^\d+$/.test(bOrder) && aOrder !== bOrder) {
    return BigInt(aOrder) > BigInt(bOrder) ? -1 : 1;
  }
  return String(capturedDate(b)).localeCompare(String(capturedDate(a)));
}

function firstLink(bookmark) {
  return (Array.isArray(bookmark.links) ? bookmark.links : [])
    .map((link) => typeof link === 'string' ? link : link?.url)
    .map(safeUrl)
    .find(Boolean) || '';
}

function authorLabel(bookmark) {
  const author = bookmark.author || {};
  return author.name && author.handle ? author.name + '  @' + author.handle : author.handle ? '@' + author.handle : author.name || 'Unknown voice';
}

function topicLabel(topic) {
  return topic.label || topic.name || topic.id || 'General';
}

function topicId(topic) {
  return topic.id || topic.name || topic.label || 'general';
}

function searchBlob(bookmark) {
  return [
    bookmark.title, bookmark.excerpt, bookmark.text, bookmark.author?.handle, bookmark.author?.name, bookmark.primaryTopic,
    ...(bookmark.topics || []), ...(bookmark.tags || []),
    ...(bookmark.links || []).map((link) => typeof link === 'string' ? link : link?.url || ''),
    bookmark.quotedTweet?.text, bookmark.quotedTweet?.author?.handle
  ].join(' ').toLowerCase();
}

function score(bookmark, query) {
  if (!query) return 0;
  const terms = query.toLowerCase().split(/\s+/).filter(Boolean);
  const title = String(bookmark.title || '').toLowerCase();
  const author = String(bookmark.author?.handle || '').toLowerCase();
  const blob = searchBlob(bookmark);

  return terms.reduce((total, term) => {
    if (!blob.includes(term)) return -100000;
    return total + (title.includes(term) ? 12 : 0) + (author.includes(term) ? 8 : 0) + (blob.split(term).length - 1);
  }, 0);
}

function getResults() {
  const query = state.query.trim();
  const results = state.bookmarks.filter((bookmark) => {
    const matchesTopic = state.topic === 'all' || bookmark.primaryTopic === state.topic || (bookmark.topics || []).includes(state.topic);
    const matchesYear = state.year === 'all' || String(bookmark.year || '') === state.year;
    return matchesTopic && matchesYear && (!query || score(bookmark, query) > -100000);
  });

  results.sort((a, b) => {
    if (query) {
      const difference = score(b, query) - score(a, query);
      if (difference) return difference;
    }

    if (state.sort === 'author') return authorLabel(a).localeCompare(authorLabel(b));

    if (state.sort === 'saved') return compareSaved(a, b);
    return String(b.postedAt || '').localeCompare(String(a.postedAt || ''));
  });

  return results;
}

function renderCard(bookmark) {
  const link = firstLink(bookmark);
  const topic = bookmark.primaryTopic || (bookmark.topics || [])[0] || 'saved';
  const host = link ? new URL(link).hostname.replace(/^www\./, '') : 'Open detail';

  return '<button class="bookmark-card" type="button" data-id="' + escapeHtml(bookmark.id) + '">' +
    '<span class="card-top"><span class="card-topic">' + escapeHtml(topic) + '</span><span class="card-year">' + escapeHtml(bookmark.year || '') + '</span></span>' +
    '<span class="card-author">' + escapeHtml(authorLabel(bookmark)) + '</span>' +
    '<span class="card-title">' + escapeHtml(bookmark.title || bookmark.excerpt || 'Saved signal') + '</span>' +
    '<span class="card-excerpt">' + escapeHtml(bookmark.excerpt || bookmark.text || '') + '</span>' +
    '<span class="card-bottom"><span>' + escapeHtml(compactDate(bookmark.capturedAt || bookmark.postedAt)) + '</span><span class="card-link">' + escapeHtml(host) + '</span></span>' +
  '</button>';
}

function renderStats() {
  const stats = state.stats || {};
  els.totalCount.textContent = nf.format(stats.bookmarkCount || stats.totalBookmarks || state.bookmarks.length);
  els.authorCount.textContent = nf.format(stats.authorCount || stats.uniqueAuthors || new Set(state.bookmarks.map((item) => item.author?.handle).filter(Boolean)).size);
  els.topicCount.textContent = nf.format(state.topics.length);

  const dates = state.bookmarks.map((item) => item.postedAt || item.capturedAt).filter(Boolean).sort();
  els.dateRange.textContent = dates.length ? new Date(dates[0]).getFullYear() + ' - ' + new Date(dates[dates.length - 1]).getFullYear() : 'No dates';
}

function renderOptions() {
  const years = [...new Set(state.bookmarks.map((item) => item.year).filter(Boolean))].sort((a, b) => Number(b) - Number(a));
  els.year.innerHTML = '<option value="all">All years</option>' + years.map((year) => '<option value="' + escapeHtml(year) + '">' + escapeHtml(year) + '</option>').join('');
  els.year.value = state.year;
  els.topic.innerHTML = '<option value="all">All signals</option>' + state.topics.map((topic) => '<option value="' + escapeHtml(topicId(topic)) + '">' + escapeHtml(topicLabel(topic)) + '  (' + nf.format(topic.count || 0) + ')</option>').join('');
  els.topic.value = state.topic;
  els.topicListCount.textContent = nf.format(state.topics.length);
  els.topicList.innerHTML = state.topics.slice(0, 12).map((topic) => '<button class="topic-button' + (state.topic === topicId(topic) ? ' is-active' : '') + '" type="button" data-topic="' + escapeHtml(topicId(topic)) + '"><span>' + escapeHtml(topicLabel(topic)) + '</span><span>' + nf.format(topic.count || 0) + '</span></button>').join('');
}

function renderResults() {
  const results = getResults();
  const shown = results.slice(0, state.visible);
  const selectedTopic = state.topics.find((topic) => topicId(topic) === state.topic);

  els.resultsTitle.textContent = state.query ? 'Search results' : selectedTopic ? topicLabel(selectedTopic) : 'All bookmarks';
  els.resultsCount.textContent = nf.format(results.length) + (results.length === 1 ? ' result' : ' results');
  els.grid.innerHTML = shown.map(renderCard).join('');
  els.empty.hidden = results.length !== 0;
  els.loadMore.hidden = shown.length >= results.length || results.length === 0;
}

async function fetchJson(file) {
  const response = await fetch(DATA_BASE + file, { cache: 'no-store' });
  if (!response.ok) throw new Error(file + ' returned HTTP ' + response.status);
  return response.json();
}

async function init() {
  try {
    const manifest = await fetchJson('manifest.json');
    const [bookmarks, topics, stats] = await Promise.all([
      fetchJson(manifest.artifacts?.search || 'search-index.json'),
      fetchJson(manifest.artifacts?.topics || 'topics.json'),
      fetchJson(manifest.artifacts?.stats || 'stats.json')
    ]);

    state.bookmarks = Array.isArray(bookmarks) ? bookmarks : [];
    state.topics = Array.isArray(topics) ? topics : [];
    state.stats = stats || {};
    els.syncStatus.dataset.state = 'ready';
    els.syncStatus.lastElementChild.textContent = 'Archive ready';
    els.syncDate.textContent = formatDate(manifest.generatedAt, 'Unknown');
    renderStats();
    renderOptions();
    renderResults();
  } catch (error) {
    els.syncStatus.dataset.state = 'error';
    els.syncStatus.lastElementChild.textContent = 'Archive unavailable';
    els.errorMessage.textContent = error.message;
    els.error.hidden = false;
  }
}

els.search.addEventListener('input', (event) => {
  state.query = event.target.value;
  state.visible = PAGE_SIZE;
  renderResults();
});

els.clearSearch.addEventListener('click', () => {
  els.search.value = '';
  state.query = '';
  renderResults();
  els.search.focus();
});

els.topic.addEventListener('change', (event) => {
  state.topic = event.target.value;
  state.visible = PAGE_SIZE;
  renderOptions();
  renderResults();
});

els.year.addEventListener('change', (event) => {
  state.year = event.target.value;
  state.visible = PAGE_SIZE;
  renderResults();
});

els.sort.addEventListener('change', (event) => {
  state.sort = event.target.value;
  state.visible = PAGE_SIZE;
  renderResults();
});

els.topicList.addEventListener('click', (event) => {
  const button = event.target.closest('[data-topic]');
  if (!button) return;
  state.topic = button.dataset.topic;
  state.visible = PAGE_SIZE;
  renderOptions();
  renderResults();
});

els.grid.addEventListener('click', (event) => {
  const card = event.target.closest('[data-id]');
  if (!card) return;
  const bookmark = state.bookmarks.find((item) => String(item.id) === String(card.dataset.id));
  if (!bookmark) return;
  const external = safeUrl(bookmark.url) || firstLink(bookmark);
  els.drawerContent.innerHTML = '<h2 class="drawer-title">' + escapeHtml(bookmark.title || 'Saved signal') + '</h2>' +
    '<p class="drawer-author">' + escapeHtml(authorLabel(bookmark)) + '</p>' +
    '<div class="drawer-text">' + escapeHtml(bookmark.text || bookmark.excerpt || '') + '</div>' +
    '<div class="drawer-meta"><span>Captured ' + escapeHtml(formatDate(bookmark.capturedAt || bookmark.postedAt)) + '</span><span>Posted ' + escapeHtml(formatDate(bookmark.postedAt)) + '</span></div>' +
    (external ? '<div class="drawer-actions"><a href="' + escapeHtml(external) + '" target="_blank" rel="noreferrer">Open source</a></div>' : '');
  els.backdrop.hidden = false;
  els.drawer.hidden = false;
  els.drawer.setAttribute('aria-hidden', 'false');
});

els.loadMore.addEventListener('click', () => {
  state.visible += PAGE_SIZE;
  renderResults();
});

function closeDrawer() {
  els.backdrop.hidden = true;
  els.drawer.hidden = true;
  els.drawer.setAttribute('aria-hidden', 'true');
}

els.backdrop.addEventListener('click', closeDrawer);
els.drawerClose.addEventListener('click', closeDrawer);

document.addEventListener('keydown', (event) => {
  if (event.key === '/' && document.activeElement !== els.search && !['INPUT', 'SELECT', 'TEXTAREA'].includes(document.activeElement?.tagName)) {
    event.preventDefault();
    els.search.focus();
  }
  if (event.key === 'Escape' && !els.drawer.hidden) closeDrawer();
});

init();
Enter fullscreen mode Exit fullscreen mode

The search index is small enough for the browser to hold in memory. Each query scores title matches and author matches, then checks the wider text blob. The page renders 36 cards at a time, which keeps the initial DOM light for a large archive.

The stylesheet gives the archive its paper-like surface and its responsive grid. The full production stylesheet lives in web/styles.css inside the private frontend repository. This compact version shows the layout primitives:

:root {
  --paper: #f5f1e9;
  --ink: #1f2622;
  --muted: #69716b;
  --line: #d5cec1;
  --card: #fcfaf5;
  --accent: #c54d31;
  --serif: Georgia, serif;
  --sans: "Segoe UI", sans-serif;
}

* { box-sizing: border-box; }
html { background: var(--paper); color: var(--ink); }
body { margin: 0; min-width: 320px; font-family: var(--sans); background: var(--paper); }
button, input, select { font: inherit; }
a { color: inherit; }

.site-shell { width: min(1480px, 100%); margin: 0 auto; padding: 0 38px 34px; }
.topbar { min-height: 84px; display: flex; align-items: center; justify-content: space-between; border-bottom: 1px solid var(--line); }
.brand { display: inline-flex; gap: 12px; align-items: center; text-decoration: none; }
.brand-mark { display: grid; place-items: center; width: 34px; height: 34px; border: 1px solid var(--ink); border-radius: 50%; }
.intro { display: grid; grid-template-columns: minmax(0, 1fr) 270px; gap: 40px; padding: 78px 0 64px; }
h1 { max-width: 760px; margin: 0 0 20px; font: 500 clamp(46px, 7vw, 94px)/0.94 var(--serif); letter-spacing: -0.055em; }
.stat-strip { display: grid; grid-template-columns: repeat(3, 1fr) 2fr; border-top: 1px solid var(--ink); border-bottom: 1px solid var(--line); }
.stat-block { min-height: 92px; padding: 18px 24px 16px 0; border-right: 1px solid var(--line); }
.workspace { display: grid; grid-template-columns: 230px minmax(0, 1fr); gap: 50px; padding-top: 52px; }
.results-grid { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 14px; }
.bookmark-card { min-height: 270px; display: flex; flex-direction: column; padding: 20px; border: 1px solid var(--line); background: var(--card); cursor: pointer; text-align: left; }
.card-title { margin: 0 0 10px; font: 500 24px/1.04 var(--serif); }
.card-bottom { display: flex; justify-content: space-between; gap: 10px; margin-top: auto; padding-top: 20px; color: var(--muted); font-size: 10px; }
.detail-drawer { position: fixed; z-index: 6; top: 0; right: 0; width: min(560px, 100%); height: 100vh; overflow: auto; padding: 30px 34px 50px; background: var(--card); }

@media (max-width: 760px) {
  .site-shell { padding: 0 18px 24px; }
  .intro { display: block; padding: 54px 0 42px; }
  .workspace { display: block; padding-top: 34px; }
  .results-grid { grid-template-columns: 1fr; }
}
Enter fullscreen mode Exit fullscreen mode

The live repository keeps the complete version with topic buttons, drawers, mobile rules, typography, and hover states.

Cloudflare Pages setup

The frontend repository README records these build values:

Production branch: main
Build command: npm run build
Build output directory: dist
Root directory: repository root
Enter fullscreen mode Exit fullscreen mode

In Cloudflare, create a Pages project from the GitHub repository. Choose the main branch. Add the build command and the output directory above. The project’s build invokes web/build.mjs, which copies the data snapshot into the deployable directory.

Cloudflare’s current static HTML guide describes the same model: the build command runs during deployment and the output directory becomes the published site. See the Cloudflare Pages static HTML guide.

The custom domain setup lives under Workers & Pages, the Pages project, and Custom domains. Add signals.asynchronope.my.id there. Cloudflare then connects the subdomain to the Pages project. The custom domains documentation describes the current dashboard path and the CNAME behavior.

For a subdomain, the DNS record points at the Pages hostname:

Type: CNAME
Name: signals
Target: your-project.pages.dev
Enter fullscreen mode Exit fullscreen mode

The domain association happens inside the Pages project before the DNS record is tested. This gives Cloudflare the context needed to activate the custom domain and issue the certificate.

Run the frontend locally

Clone the frontend repository and build it:

git clone https://github.com/your-account/my-twitter-bookmarks.git
Set-Location my-twitter-bookmarks
npm install
npm run build
Enter fullscreen mode Exit fullscreen mode

Serve dist with any static server. Node gives a convenient option:

npx serve dist
Enter fullscreen mode Exit fullscreen mode

Open the local URL printed by the server. The browser should request these files:

/data/manifest.json
/data/search-index.json
/data/topics.json
/data/stats.json
Enter fullscreen mode Exit fullscreen mode

A direct check helps isolate a build problem from a browser problem:

Get-Content .\dist\data\manifest.json
Enter fullscreen mode Exit fullscreen mode

The generatedAt value shows the time recorded by the exporter. The bookmark count comes from the same dataset that the cards display.

Run one sync by hand

The schedule can wait while the pipeline is tested manually:

powershell -ExecutionPolicy Bypass `
  -File D:\Repo\fieldtheory\scripts\automation\sync-bookmarks-to-github.ps1 `
  -ConfigPath 'C:\Users\your-user\.ft-bookmarks\automation\github-sync\bookmark-sync.config.psd1' `
  -TriggerReason 'manual-test'
Enter fullscreen mode Exit fullscreen mode

Watch the log directory:

Get-ChildItem 'C:\Users\your-user\.ft-bookmarks\automation\github-sync\logs' |
  Sort-Object LastWriteTime -Descending |
  Select-Object -First 1 |
  Get-Content
Enter fullscreen mode Exit fullscreen mode

The run should leave a new commit in the frontend repository. Cloudflare Pages will receive that commit through its Git integration and build dist.

A useful failure map

The task stays idle

Inspect the task metadata:

Get-ScheduledTask -TaskPath '\FieldTheory\' -TaskName 'Bookmark Sync'
Get-ScheduledTaskInfo -TaskPath '\FieldTheory\' -TaskName 'Bookmark Sync'
Enter fullscreen mode Exit fullscreen mode

The task account needs permission to read the Firefox profile and execute Node. The machine needs a network connection for the bookmark request and the Git push.

Cookie access fails

Check the Firefox profile path in the config. Firefox should have an active X session in that profile. A Firefox profile can change after a migration or a fresh installation, so inspect the profile directory when this error appears.

Get-ChildItem 'C:\Users\your-user\AppData\Roaming\Mozilla\Firefox\Profiles'
Enter fullscreen mode Exit fullscreen mode

The JSON files update while the site stays stale

Open the deployed data/manifest.json and compare its generatedAt value with the local file. The site sets no-cache on the manifest and a five-minute cache on the other data files. A fresh deployment should show a new manifest immediately after Pages finishes.

The Pages build fails

Run the same build locally:

Set-Location D:\Repo\my-twitter-bookmarks
npm run build
Get-ChildItem .\dist\data
Enter fullscreen mode Exit fullscreen mode

fieldtheory/site must exist before web/build.mjs runs. The source repository needs the generated snapshot commit before Pages can copy it into dist/data.

The public archive exposes more than intended

Review the generated files before the first public deployment. The exporter selects fields for the website. The public repository should contain the chosen bookmark fields and generated metadata. Browser cookies, OAuth tokens, and the local SQLite file belong on the source machine.

Why the split works

Field Theory understands X session sync, local storage, deduplication, classification, and export formats. The frontend understands reading, filtering, and browsing static JSON. Each side has a small boundary made of files.

That boundary makes the archive easy to inspect. A sync run can be tested from PowerShell. A site build can be tested from the frontend repository. A browser request can be opened directly against data/manifest.json.

The site can keep its visual language for years while the dataset changes twice a day. The exporter can add a new field while the browser keeps reading the existing ones. A new consumer can read bookmarks.full.jsonl without joining the frontend bundle.

Complete source files

The production files are longer than the excerpts in this article because they include retries, lock handling, Telegram notifications, XLSX generation, topic rules, detailed normalization, and the full responsive stylesheet. The frontend repository can remain private while the article carries the implementation pattern and the complete public-facing page code.

Use these paths as the canonical source inside the private frontend repository:

The important operational detail sits in the handoff:

Firefox -> ft sync -> bookmarks.jsonl -> ft site-export -> fieldtheory/site -> git push -> Pages build -> signals.asynchronope.my.id
Enter fullscreen mode Exit fullscreen mode

Top comments (0)