DEV Community: John Sheehy

Cleaning duplicate photos inside Lightroom Classic and Capture One catalogs without breaking them

John Sheehy — Wed, 04 Nov 2026 12:42:21 +0000

Disclosure: I'm the developer of Folio Refresh, the Mac app this post is about.

Most duplicate photo finders work on folders. Point them at a drive, they hash files, they move copies to the Trash. That is fine until the photos live in a Lightroom Classic or Capture One catalog. Delete a file behind Lightroom's back and the catalog shows a missing-photo badge; delete the wrong copy and you lose the edits, ratings and collections attached to it.

When I built Folio Refresh I wanted it to clean duplicates inside those catalogs, so here is how it does that without corrupting them.

Both catalogs are SQLite

A Lightroom Classic catalog (.lrcat) is a SQLite database. A Capture One catalog (.cocatalog) is a package with a SQLite database (.cocatalogdb) inside. Neither schema is documented, and both change between versions. That shapes every rule below.

Rule 1: read-only, always, while scanning

Scanning only ever opens the catalog read-only.

Lightroom: SQLITE_OPEN_READONLY. SQLite guarantees a read-only connection cannot write the database or its log, so this is safe even while Lightroom has the catalog open; it is just another reader.
Capture One: its database keeps no write-ahead log, so while Capture One is open it is read with ordinary shared locks, and while it is closed it is opened immutable.

Before reading anything, the app checks that every table and column it uses exists. If a new Lightroom or Capture One version renames something, the catalog is left alone instead of being read wrongly.

Rule 2: the catalog decides what is worth keeping

A folder can't tell you that one copy of a photo has two hours of Develop adjustments on it. The catalog can. When Folio Refresh groups duplicates, it reads what each copy carries:

Lightroom: pick flag, star rating, Develop adjustments, collections (including the Quick Collection), and virtual copies.
Capture One: rating, colour tag, adjustments, albums the person made, keywords, and variants.

A copy someone worked on is the one suggested to keep. A photo with a virtual copy (Lightroom) or a second variant (Capture One) is never offered for removal at all, because that "duplicate" is really another edit of the same file.

Rule 3: never delete. Mark, and let the app delete

Folio Refresh never deletes or moves a file inside a catalog. It changes exactly one thing, the same thing a person would change by hand:

Lightroom Classic: the duplicate's master image is marked Rejected, the same as pressing X. That is one column:

  UPDATE Adobe_images SET pick = -1.0
  WHERE id_local = ? AND masterImage IS NULL;

The person then runs Lightroom's own Delete Rejected Photos, which removes the photos from the catalog and the disk the way Lightroom expects.

Capture One: there is no Rejected flag, so duplicates are tagged Red, writing the colour tag to the same metadata rows Capture One writes when you press the Red tag yourself. The person filters by Red and deletes with Capture One's own Delete.

Marking frees no space by itself, and that is the point: the destructive step stays in the app that owns the catalog.

Rule 4: only write when the app is closed

Writing to a SQLite file another process has open is how catalogs get corrupted. So before any write:

Lightroom: no .lrcat-lock beside the catalog, and Lightroom Classic not running.
Capture One: not running, and no writelock file inside the package.

If either is open, nothing is written; the app asks you to quit it first.

Rule 5: back up with SQLite itself, then change everything in one transaction

Copying the .lrcat file with FileManager can miss pages still in the log. Instead the backup uses SQLite's online backup API, so the copy is a complete, consistent database:

guard let backup = sqlite3_backup_init(destination, "main", source, "main") else { throw ... }
let step = sqlite3_backup_step(backup, -1)   // copy every page
sqlite3_backup_finish(backup)

It lands in a Folio Refresh Backups folder beside the catalog, named with the date and never overwriting an earlier one. If the backup fails, nothing is changed.

Then all the marks go in one transaction (BEGIN IMMEDIATE … COMMIT, with ROLLBACK on any error), so a catalog is never left half-marked.

Rule 6: re-check right before writing, and keep an undo

Between review and cleanup, files can change. Right before marking, each duplicate must still be:

the same file the person reviewed, at the same path in the catalog,
not already Rejected (or Red, or in Capture One's Trash),
free of virtual copies or extra variants,

and at least one copy its group keeps must still be there, unchanged. Anything that fails is skipped with a reason.

Every mark is recorded with the flag it replaced, so Undo can put the old pick flag or tag back (again only while the app is closed), and the dated backup is always there as a last resort.

Rule 7: only write to versions you have tested

Reading works on any catalog whose columns check out. Writing is refused on a catalog version the app hasn't been tested against (Adobe_variablesTable.Adobe_DBVersion for Lightroom, ZVERSIONINFO.ZVERSION for Capture One). An undocumented schema is a moving target, and a refusal is cheaper than a broken catalog.

The rest of the app, briefly

Exact copies are found by SHA-256 of the file contents, so renamed copies match.
Similar shots use Apple Vision image feature prints, compared on the Mac, at three levels from near-exact to looser edits.
Bursts are grouped by capture time, with a keeper suggested by sharpness and Vision's aesthetics score.
Apple Photos duplicates go into a review album; deleting happens in Photos, which asks first.
Nothing is uploaded and there is no account.

If you keep a Lightroom or Capture One catalog and have opinions about how a tool should touch it, I'd like to hear them in the comments. Folio Refresh is free to scan and to clean your first 100 photos, then $14.99 once: caddora.ie/folio.

Flutter + ML Kit: the missing dependency that reads a Japanese menu as romaji

John Sheehy — Wed, 04 Nov 2026 12:10:26 +0000

SofiLingo is the offline travel translator I make in Flutter, for iPhone and Android. Its camera mode reads menus, signs and tickets with Google's ML Kit text recognition, then translates them on the device.

One part of that setup fails without a single error: on iOS, on Android, in flutter analyze, in a Dart test or in a build log. If you use google_mlkit_text_recognition with anything other than Latin script, it is worth five minutes.

ML Kit's recognizer is per script

ML Kit ships five on-device text recognizers: Latin, Chinese, Devanagari, Japanese and Korean. You pick one when you create the recognizer:

final recognizer = TextRecognizer(script: TextRecognitionScript.japanese);

The wrong one doesn't fail. It returns confident nonsense. Point the Latin model at a Japanese menu and you get stray romaji fragments, which look like a bad photo rather than a bug. So the app picks the recognizer from the source language:

static const _scripts = <TranslateLanguage, TextRecognitionScript>{
  TranslateLanguage.chinese: TextRecognitionScript.chinese,
  TranslateLanguage.japanese: TextRecognitionScript.japanese,
  TranslateLanguage.korean: TextRecognitionScript.korean,
  TranslateLanguage.hindi: TextRecognitionScript.devanagiri,
  TranslateLanguage.marathi: TextRecognitionScript.devanagiri,
};

That part is in the docs. The trap is underneath it.

The plugin ships Latin only

The Flutter plugin's text recognition pod contains the Latin model and nothing else. Chinese, Japanese, Korean and Devanagari are separate native dependencies, and the plugin does not pull them in for you.

Leave them out and here is what happens.

iOS: nothing visible. The plugin detects which scripts were compiled in with __has_include, and its switch has a bare default: that returns the Latin recognizer for any script it wasn't built with. Your code asks for Japanese, gets Latin, and the Japanese menu comes back as romaji. No exception, no log line.

Android: the plugin declares those libraries compileOnly, so they are on its compile path but absent from your APK. The first frame throws NoClassDefFoundError. In a camera preview that processes frames in a loop, that throw usually lands in a per-frame catch (_), and you get a preview that never finds any text.

Two platforms, two different failures, neither of them loud.

The fix: declare the scripts yourself

iOS, in ios/Podfile inside the Runner target (about 1.2 to 1.6 MB each):

pod 'GoogleMLKit/TextRecognitionChinese', '~> 9.0.0'
pod 'GoogleMLKit/TextRecognitionDevanagari', '~> 9.0.0'
pod 'GoogleMLKit/TextRecognitionJapanese', '~> 9.0.0'
pod 'GoogleMLKit/TextRecognitionKorean', '~> 9.0.0'

Android, in android/app/build.gradle.kts:

dependencies {
    implementation("com.google.mlkit:text-recognition-chinese:16.0.1")
    implementation("com.google.mlkit:text-recognition-devanagari:16.0.1")
    implementation("com.google.mlkit:text-recognition-japanese:16.0.1")
    implementation("com.google.mlkit:text-recognition-korean:16.0.1")
}

Match the versions to the ML Kit version your plugin release uses.

Guard it with a test

Nothing in the toolchain will notice if someone tidies those lines away later, so a plain Dart test reads the two build files and fails if a script goes missing:

const scripts = ['Chinese', 'Devanagari', 'Japanese', 'Korean'];

test('the iOS pods are declared', () {
  final podfile = File('ios/Podfile').readAsStringSync();
  for (final script in scripts) {
    expect(podfile, contains('GoogleMLKit/TextRecognition$script'),
        reason: '$script has no pod, so it silently reads as Latin');
  }
});

test('the Android dependencies are declared', () {
  final gradle = File('android/app/build.gradle.kts').readAsStringSync();
  for (final script in scripts) {
    expect(gradle, contains('text-recognition-${script.toLowerCase()}'),
        reason: '$script is not in the APK, so it throws at the first frame');
  }
});

It is a blunt test, but it is the only place this failure can be caught before a user points their phone at a menu.

One more: list your Latin languages, don't default to them

A related trap: if the language-to-script lookup falls back to Latin for anything it doesn't know, every new language silently claims the camera works for it. Russian through the Latin recognizer returns nothing at all, again looking like a bad photograph. SofiLingo lists its Latin-script languages explicitly, and any language in neither list has no on-device path, so it goes to cloud OCR instead (Cyrillic, Greek, Hebrew, Arabic script, Thai and every Indic script except Devanagari).

SofiLingo translates menus, signs, speech and text in 58 languages, with typing that works offline, no account, and translations that stay on your phone. It's free with ads, and a one-off Holiday pass removes them for 14 days. sofilingo.com

What fills up a developer's Mac, and what is safe to clear?

John Sheehy — Fri, 23 Oct 2026 11:52:16 +0000

A developer's Mac fills up differently from anyone else's. Photos and downloads are rarely the problem; build systems are. Every tool you use keeps its own cache, every project keeps its own dependencies, and almost none of them ever clean up after themselves.

The good news is that nearly all of it regenerates. The useful question is not "what can I delete" but "what does deleting it cost me" — a slower next build, a re-download, or something that does not come back.

Xcode and the simulators

Usually the largest single block of space on an Apple developer's Mac.

	What it is	Clearing it costs
DerivedData	Build output and indexes, one folder per project	A full rebuild and re-index — how to clear it
iOS Device Support	Symbol files copied from each device you have plugged in	A few minutes the next time that device connects
SwiftUI Previews	Simulators created for Xcode's canvas	Previews rebuild themselves
Simulator runtimes	One per iOS or watchOS version you have downloaded	Several gigabytes to download again — removing old runtimes
Simulators	Each simulated device, with its apps and data	The apps and data inside it
Archives	Every build you distributed	Not regenerable — they hold the symbols you need to read crash reports

Archives are the one item here that is yours rather than a cache. Move old ones to another drive rather than deleting them.

Package managers

Every package manager keeps a download cache, separate from the projects that use it. All of these are re-downloaded on the next install:

npm — ~/.npm/_cacache. npm cache clean --force.
Yarn and pnpm — their own caches and stores.
CocoaPods — ~/Library/Caches/CocoaPods.
Gradle — ~/.gradle/caches, plus downloaded Gradle versions.
Homebrew — old bottles and downloads. How to clean the Homebrew cache.

Inside your projects

Each project carries dependencies and build output of its own: node_modules, Pods, Flutter's .dart_tool, .next, dist, Rust's target, Swift's .build. All rebuildable, but the project will not build until you reinstall — so clear the projects you are not working on this week. Deleting node_modules on a Mac covers finding them all.

Android

The emulators' disk images and the SDK's extras — the NDK, system images and the emulator itself — are only needed to build Android apps on this Mac. Each can be reinstalled from Android Studio, at the cost of a large download.

AI coding tools

Coding agents and desktop AI apps leave build trees, runtimes, update downloads and session history behind. Most of it is cache. Session history is not: it is a record of your conversations, and deleting it cannot be undone. Where AI tools keep their data has the folders.

Things that are running, not stored

Not every developer problem is disk space. A simulator left booted, or a log watcher a tool started and forgot, keeps costing CPU and memory long after you have finished with it — which shows up as a hot Mac and a fan that will not stop, and as swap written to disk.

How RefreshYourMac handles this

RefreshYourMac was built on a developer's Mac, and these are the places it looks hardest. It measures each of the above — see RefreshYourMac for developers — as its own category among its 45, sized separately and rated for what clearing it costs:

Safe to Remove — caches that regenerate: DerivedData, iOS Device Support, SwiftUI Previews and the npm, Yarn, pnpm, CocoaPods, Gradle and Homebrew caches.
Review — rebuildable but not free: simulator runtimes and simulators, Android emulators and SDK extras, and your projects' build folders, each with a note on how to restore it.
Move to Backup — Xcode archives and AI tool session history, which hold things that do not come back. There is no bulk clean here; you pick items one at a time, or move them to another drive.

Its Running Now list shows booted simulators and old log watchers, with what each is costing in CPU and memory, and stops the ones you choose.

Nothing is deleted without a confirmation, and a scheduled clean only ever takes Safe to Remove categories — skipping Xcode's while Xcode is open. The reasoning behind each rating is on how it decides what is safe.