DEV Community

OnePassApps
OnePassApps

Posted on

Find Duplicate Files in the Browser: SHA-256 Groups, Not Uploads

Duplicate files are rarely duplicates by name. They are duplicates by content — the same photo saved three times with three different names, the same export saved again by an older tool.

So the useful question is not "do these two files look the same" but "do these two files hash the same". That is a question you can answer exactly, and you can answer it without handing your files to anybody.

I work on OnePassApps, and the File Studio in our browser app has a Find duplicates tab that does precisely this. It hashes the content of the files you pick, groups the ones that match, and leaves every file where it is.

What the panel says

Open the File Studio and pick the Find duplicates tab. The empty panel is small and specific:

01
Source
0 files · 0 B. Only the files you choose are compared.
02
Comparison
Files
0 of 500
Total size
0 B of 2 GiB
Compared by
SHA-256 of the content
Enter fullscreen mode Exit fullscreen mode

Two details there matter more than they look.

"Only the files you choose are compared." This is not a folder scan. There is a drop area labelled FILES TO COMPARE with the hint "Drop files to compare, or browse" and a Browse files button. You pick the set; the tool never wanders off to look for more.

"Compared by SHA-256 of the content." The comparison key is a cryptographic hash of the bytes, not the file name and not the file size. Below the panel the page also states the promise plainly: "Groups of identical files are listed on the stage. Nothing is deleted." and "Nothing leaves this tab."

Running it on real files

I made four small text files on disk. Three of them were byte-identical copies of the same 49-byte file. The fourth was a deliberate trap: the same 49 bytes, with one character changed ("line three" → "line threE"). Same size. Different content.

Dropping all four in and pressing Find exact duplicates gave:

03
Result
1 group
Found 1 exact duplicate group among 4 selected files.
No files were moved, changed or deleted.
Compared in 0.4 s
Enter fullscreen mode Exit fullscreen mode

And then the group:

01
Group 1 · 49 B
3 identical files
dup-alpha.txt
File 1
49 B
dup-alpha-copy.txt
File 2
49 B
dup-alpha-third.txt
File 3
49 B
SHA-256 d5b9ce22116b32aa79d891084f76fde67f61529314def063ef3f7af23d0e2d70
Enter fullscreen mode Exit fullscreen mode

Three files in the group. The fourth — the same-size impostor — is not in it, and it does not appear anywhere else in the result either. It simply is not a duplicate, and the tool says so by leaving it out.

The counters moved with the selection: "4 files · 196 B. Only the files you choose are compared." in the Source step, "Files 4 of 500" and "Total size 196 B of 2 GiB" in the Comparison step.

Checking the hash independently

The panel prints the SHA-256 of the group. That is a claim you can check, and you should — a hash in a UI is just text until something else agrees with it.

On the same machine, sha256sum on the original files:

$ sha256sum dup-alpha.txt dup-alpha-copy.txt dup-alpha-third.txt dup-same-size.txt
d5b9ce22116b32aa79d891084f76fde67f61529314def063ef3f7af23d0e2d70  dup-alpha.txt
d5b9ce22116b32aa79d891084f76fde67f61529314def063ef3f7af23d0e2d70  dup-alpha-copy.txt
d5b9ce22116b32aa79d891084f76fde67f61529314def063ef3f7af23d0e2d70  dup-alpha-third.txt
dabc02b6c22f74154286fd98887f78566174121a07f13b26730813bddb82c0d2  dup-same-size.txt
Enter fullscreen mode Exit fullscreen mode

Same digest for the three copies, a different one for the impostor — and the digest the browser panel printed for the group is byte-for-byte the one sha256sum reports. The tool is not inventing a label for "these look alike"; it is reporting a real hash that another tool agrees with.

Why hash on content and not on name or size

Two reasons, both practical.

Size collisions are common, hash collisions are not. Photos exported from the same camera at the same settings often land on identical byte counts without being the same image. SHA-256 gives you a 256-bit digest, so "same hash" is a far stronger statement than "same size".

Names lie. photo-final-v2.jpg and photo-final-FINAL-ok.jpg are the same picture. A name-based comparison misses that; a content-based one does not. In my run the three copies had three different names and were still grouped together.

The cost is honest and worth stating: hashing every byte of every file is real work, so the panel caps the job at "Up to 500 files and 2 GiB in total". Files that big are for the archive case, not for a quick check.

What it does not do

Worth being clear about the edges, because tools like this are easy to over-sell:

  • Nothing is deleted, moved or renamed. The result says so directly: "No files were moved, changed or deleted." You get a list of what is duplicated; cleaning up is your decision.
  • It is exact-match only. A file that differs by one byte is not a duplicate, as the impostor showed. There is no fuzzy or near-duplicate mode here.
  • The comparison is limited to the files you select. It is not a disk-wide sweep, and it does not read directories you have not opened.
  • It reports, it does not act. There is a Copy list control to take the group listing away with you, and a Compare again control to re-run the comparison. That is the whole action set.

Where it runs

The File Studio processes in the browser tab. The page states "Processing stays in this browser." and "Nothing leaves this tab." — the hashing happens locally, so the files whose contents you may not want uploaded stay on your machine.

The Standard plan is free right now at $0 / month with 100,000 credits every 30 days, no end date set and no card required. There is no automatic paid subscription.

If you want to try it, the File Studio is at app.onepassapps.com/studios/files — or read through the app guides first: app.onepassapps.com/guides.

One thing I would still like to see: a "near-duplicate" mode — same image re-encoded at a different quality, so the bytes differ but the picture does not. That is a different problem from this one, and this tab does not attempt it.

Top comments (0)