DEV Community

OnePassApps
OnePassApps

Posted on

Remove Duplicate Rows in a CSV in the Browser, Without Uploading the File

Full disclosure: I work on OnePassApps.

A CSV export arrives with the same customer three times, or two signups for the same email that differ by a trailing space. Finding those rows by eye in a spreadsheet is slow, and the usual shortcut - uploading the export to a duplicate-finder website - means handing a file with names and emails in it to a server you do not control.

OnePassApps does this inside the browser tab instead. The Data Studio has a Duplicates tool, and the page says exactly how it decides that two rows are the same: A row is a duplicate when every cell matches an earlier row.

What the Duplicates tool does

The panel runs in three steps - 01 Source, 02 Options, 03 Result - and the source box accepts more than CSV: Drop a data file, or browse CSV, TSV, JSON or YAML - up to 20 MiB - or paste rows in Source.

  • Delimiter sits in the Options step and defaults to Auto detect. The tool finds the separator itself, and the result panel reports which one it used.
  • The rule is strict: a duplicate is a row where every cell matches an earlier row. A partial match - same email, different name - is not a duplicate, and the tool will not treat it as one.
  • The result is a cleaned CSV you download. The result step is described as Rows, columns and any parsing issues appear here after the tool runs, and the preview shows the cleaned rows with a Download CSV button.

The page is plain about where the work happens: Nothing leaves this tab. and Processing stays in this browser.

A real run

I pasted a CSV with four rows into the source box, where one row was an exact repeat of another, and pressed Remove duplicate rows:

  • Removed 1 duplicate row(s). and Duplicates removed 1
  • 3 rows · 3 columns · no parsing issues
  • Delimiter found Comma
  • Cleaned in 571 ms

The CSV preview then showed the three surviving rows - Ada, Grace and Linus - with the duplicated Ada row gone, and a Download CSV button next to it. The original file on disk is untouched; the cleaned copy is built locally in the tab.

Why "every cell matches" is the right rule

Duplicate detection is one of those jobs where a looser rule quietly does damage. Rows that match on the one field you care about, and differ on another, are usually not duplicates - they are two different records that happen to share an email or a postcode. Matching on every cell keeps a row that was changed in any way, which is the safer default when the file behind it is a customer list.

The tool then shows you the counts - how many rows went in, how many came out, how many were dropped - so you can check the arithmetic before you overwrite anything.

Where a browser-local run is the right call

  • The file has names, emails or IDs in it. Those are the rows you do not want landing on a third party's disk, and this tool never sends the file anywhere.
  • You want to look before you overwrite. The preview shows the surviving rows in the tab; you download a new CSV and keep the original.
  • The clean-up is a one-off. A nightly dedupe across millions of rows belongs in your own pipeline, not in a tab.

The same Studio covers the work around it: Viewer, Sort / filter, Columns, Merge, Split, To JSON, Formatter, Validator, To CSV and YAML. So you can inspect, reshape and re-export the same data without moving it between tools.

Standard is currently free during a promotional period: $0 per month, 100,000 credits every 30 days, no card required. The Studios run in the browser tab and use no credits, so export your CSV before you close the tab.

Try it on your own file: open Data Studio.

If you want the wider picture on how these browser-local tools work and what they do not do, the OnePassApps guides cover it.

OnePassApps Team - we build app.onepassapps.com, one membership for nine browser-local and hosted apps.

Top comments (0)