Most of the builders I'd love to have on Boat Build Log already wrote their log somewhere else. They posted it to Facebook, one update at a time, for years. Asking a retired man to retype ten years of posts into my site is exactly why he never signs up.
So I wrote an importer. This post is about the part that turned out to matter: taking messy, harvested posts, turning them into clean dated entries, and loading them with an Artisan command you can run ten times without making a mess.
The case study is Jim Norman's Jersey Girl, a 20 ft mini houseboat he built from 2015 on, posting every step to a public Facebook Page. I read that public Page slowly and imported it. It's now Jersey Girl on the site: 191 entries and 568 photos, March 2015 to August 2025.
| Piece | Version |
|---|---|
| Laravel | 13 |
| PHP | 8.3+ |
| MySQL | 8 |
| Images | Imagick |
Split the harvest from the import
The first version of this was for a different builder whose posts were friends-only. The only way in was Facebook's own "Download your information" archive, so I wrote bbl:parse-fb-export to read that zip. Then Jersey Girl came along, a public Page with no archive, and the harvest looked nothing like the export.
That's why the import is its own command and knows nothing about Facebook. Every harvest just has to produce the same JSON. Here's the contract, straight from the command's docblock:
[{ "date": "2024-05-14", // required, YYYY-MM-DD
"title": "Frames on the strongback",
"text": "markdown body", // required (unless there are photos)
"hours": 6.5,
"phase": "framing", // a phases.slug, optional
"source_url": "https://...", // attribution; a real link or null
"source_ref": "fb-post:171...", // stable identity, used to dedupe
"photos": ["media/0001.jpg"], // paths relative to --media
"videos": [{"provider":"youtube","id":"..."}],
"cover": true }]
A data export, a forum thread, a WordPress blog (that adapter came three days later), or me typing it in by hand. Nothing past the JSON cares which.
The command itself is small to call:
protected $signature = 'bbl:import-log
{--user= : id, handle or email of the account that will own the build}
{--json= : path to the normalised entries JSON}
{--media= : directory the photo paths in that JSON are relative to}
{--build= : import into this existing build id instead of creating one}
{--title= : title for the build being created}
{--dry-run : validate and report, write nothing}';
The Laravel Artisan docs cover the signature syntax.
The date is the whole point
A build log is a chronology. If the dates are wrong, it's just a photo dump. So the importer has one rule I'd defend above all the others: logged_on comes from the post date, and nothing ever overwrites it.
That's the opposite of the normal upload path on the site. When a builder uploads photos by hand, I prefer the photo's EXIF date over the date they typed, because you shoot on Saturday and post on Sunday. For an import that reasoning flips. Facebook strips EXIF on upload, and the EXIF that survives a re-encode is more often the day somebody's scanner ran than the day of the work. So the import stores the photo with no taken date at all:
'taken_at' => null, // the post date is the truth here; stale EXIF would show under the photo as a lie
'sort_order' => $order++,
And a missing date never becomes today(). A hundred entries all dated today is the worst outcome I can think of, because it looks fine until someone reads it. Preflight refuses it:
if ($date === '') {
$problems[] = "{$n}: no date. Resolve it in the harvest; this will not guess.";
continue;
}
if (strtotime($date) > strtotime('today 23:59:59')) {
$problems[] = "{$n}: {$date} is in the future.";
continue;
}
Preflight is a separate pass over every row before anything is written. If entry 90 of 100 is bad, I want to know before entry 1 lands, not clean up half a log by hand.
Getting dates out of Jim's own words
Jim made this easier than most. Nearly every update starts the same way: "Houseboat update, Thursday, August 28, 2025 -- Jersey Girl in her new coat." So the normaliser, a Python script on my machine, reads the date out of the first 250 characters of the text, not from any timestamp.
He's a human typing, though, so it's not that clean. Some posts say "Sept. 29" with no year. One says "November 27, 2001", fourteen years before he started. The rules ended up as:
- A year outside 2015 to 2026 is thrown away and treated as missing.
- If he named a weekday, trust the weekday over the year he typed. People fat-finger the year far more often than the day of the week.
- A date with no year borrows one from the nearest older dated post, bumping it if the month wraps past December.
- Anything still undated goes to
held.jsonfor me to look at. It never gets a guess.
The weekday check is the bit I'm happiest with:
if year and wd: # Jim sometimes typed the wrong year; trust the weekday he named
want = WEEKDAYS.index(wd.group(1).lower())
for y in (year, year - 1, year + 1):
if dt.date(y, mon, day).weekday() == want:
year = y
break
The same script guesses a phase (framing, glassing, paint and so on) by scoring each post against a small vocabulary and taking the best match, not the first. "Deck" turns up in nearly every post, so first match put half the log in the deck phase.
Idempotency, because you will run it again
I knew I'd re-run this. I'd fix a date, fix a title, re-harvest a batch. Every one of those runs has to update what's there, not append a second copy of the log.
The export has no permalinks, so I couldn't key on URL. Instead every row carries a source_ref, and the column is unique per build:
$table->string('source_ref', 191)->nullable()->after('source_url');
$table->unique(['build_id', 'source_ref']); // per build: the same post can go into two builds
For Jersey Girl it's the Facebook photo id of the post's first photo (fb-jg:1697983821144093). For the export it's the timestamp plus a short hash of the content. The import looks that up first and falls back to the URL:
if ($row['source_ref']) {
$entry = Entry::where('build_id', $build->id)
->where('source_ref', $row['source_ref'])->first();
} elseif ($row['source_url']) {
$entry = Entry::where('build_id', $build->id)
->where('source_url', $row['source_url'])->first();
}
Photos follow a stricter rule: they're only attached to an entry that has none. A re-run must not double the album, and if the builder has since reordered or captioned his photos by hand, an import re-run shouldn't throw that away.
if ($entry->media()->exists() || (! $row['photos'] && ! $row['videos'])) {
return;
}
Preflight also rejects a file where the same key appears twice. Otherwise the second row silently overwrites the first and the log comes out short with nothing said about it.
Two more decisions I'd make again. Attribution lives in entries.source_url, a column, never appended to the body. The entries table has a FULLTEXT index on title and body, and three hundred Facebook URLs in bodies would pollute every search on the site. And the command never publishes anything. The build is created as a draft, because the words belong to the builder and he hasn't seen them yet.
Photos are untrusted input
The photo paths in that JSON were written by a script that ran against a page I don't control. So preflight treats them as hostile. A ../../ in there would otherwise be read off disk and uploaded into a public build:
if ($rel === '' || str_contains($rel, '..') || preg_match('#^([a-zA-Z]:|[/\\\\])#', $rel)) {
$problems[] = "{$n} ({$date}): photo path “{$rel}” must be relative and inside --media.";
continue;
}
Each photo is then copied to a temp file and run through the same ImageProcessor the normal upload uses, so an imported photo gets the same web, thumbnail and print sizes as one a builder uploads. The harvest folder is the only copy of those files, so it stays untouched.
What went wrong
Plenty.
Reshares. Jim reshared his own older posts as memories, so the same text turned up twice with different photo ids. Different ids meant the dedupe let them both through. The fix was a second key on the normalised text (lowercased, letters and digits only, first 150 characters), and a reshare is skipped, never a second entry. A couple had already landed, so I deleted those by hand. Then the final re-derive pass only updates entries that already exist on the build, so a deleted reshare can't come back.
Photo-only posts. About a hundred of the Page's posts were a single photo with no caption. Each would have been an empty entry. Now a photo with no real caption joins the post before it.
Mojibake on the export path. Facebook's archive escapes UTF-8 one byte at a time, so "é" arrives as "é" and every apostrophe in years of writing comes out as line noise. The fix walks it back through latin-1, guarded twice so it can't mangle text that was already right:
if (preg_match('/[^\x{0000}-\x{00FF}]/u', $s)) {
return $s; // a character above U+00FF can't be this kind of mojibake
}
$bytes = @mb_convert_encoding($s, 'ISO-8859-1', 'UTF-8');
if (! is_string($bytes) || ! mb_check_encoding($bytes, 'UTF-8')) {
return $s; // a genuine "café" stays a café
}
I lost a harvest. I'd read the whole Page once, kept it in memory, and lost it before it was saved. After that I harvested in small batches, saved each one to disk, and ran the import after every batch. Slower, but nothing was ever more than one batch from safe. A handful of items didn't make it in, like a Christmas greeting and a watercolor-class post, and I'm fine with that.
The launch isn't in the log. The Page has no posts with photos between 2020 and 2023, so the first launch in May 2021 isn't there, and I don't know the day. I'd rather leave it blank than invent it.
Would I do it again?
Yes. The harvest is throwaway and source-specific. The import is boring, strict and safe to re-run, which is what you want from the half that writes to production. The dates are the only thing I'd really fight for, and preflight is where that fight happens.
If you've got a build log sitting on Facebook or an old blog, have a look at what it would look like on Boat Build Log. Feel free to borrow the posts.json shape for your own importers too.
Top comments (0)