If you've ever worked with exported spreadsheets, customer lists, or CSV files, you've probably encountered messy data: duplicate records, inconsistent formatting, missing values, or columns that need restructuring.
I wanted to create a tool that could handle these common problems without requiring users to install software, write scripts, or upload their datasets to a server.
That's how I started building DataForge.
Try DataForge: https://dataforged.netlify.app
What does DataForge do?
DataForge is a free, browser-based data-cleaning application supporting CSV, Excel (.xlsx), and JSON files.
Some of its main features include:
- Importing and previewing datasets.
- Removing duplicate records.
- Cleaning inconsistent text and missing values.
- Renaming, splitting, merging, and reorganizing columns.
- Converting column data types.
- Creating validation rules to identify problematic records.
- Saving reusable cleaning workflows locally.
- Exporting cleaned data to CSV, XLSX, or JSON.
It also includes sample data, so visitors can explore its features without uploading their own files.
Why I chose browser-based processing
Privacy was one of my main design priorities.
Many datasets contain information that people don't want to send to an external server, especially customer records or internal business exports.
DataForge processes datasets locally in the browser rather than uploading them to a backend.
This also made it possible to build the application without requiring user accounts or a hosted database.
However, browser-based processing has its own challenges, particularly when handling large files.
The technical challenges
1. Processing large datasets
Large CSV and Excel files can make browser applications unresponsive if processing happens entirely on the main thread.
To address this, I worked on chunked imports, background workers, and virtualized data previews.
The goal was to keep the interface responsive while users worked with larger datasets.
2. Building a reliable undo and redo system
Data cleaning often involves experimentation.
Users may remove duplicates, convert columns, or apply transformations, only to realize that they need to reverse a previous action.
DataForge includes undo and redo functionality so users can explore changes more safely.
3. Making workflows reusable
Repeating the same cleaning operations manually can become tedious.
I added reusable recipes, saved validation rules, and workflows that can be applied to compatible datasets.
4. Keeping everything local
The application uses browser storage for saved configurations and supports offline functionality after the necessary application resources have been cached.
This introduces additional considerations around storage limits, cache updates, and browser compatibility.
Testing and reliability
I spent a significant part of development working on testing, accessibility, and performance rather than adding features alone.
The final release passed:
- 782 unit tests.
- 268 browser tests across desktop Chromium, mobile Chromium, Firefox, and WebKit.
- Automated security and deployment checks.
Testing also helped identify problems with malformed files, worker failures, memory usage, and accessibility.
Of course, automated tests don't replace feedback from real users.
What I learned
One of the biggest lessons from this project was that implementing features is only part of building a usable product.
Handling unexpected inputs, keeping the interface understandable, preventing data loss, and making failures recoverable are just as important.
I also learned that a browser-only application can offer substantial functionality without needing a complicated backend.
I'd appreciate your feedback
DataForge is now available for anyone to try:
https://dataforged.netlify.app
It's free and doesn't require an account.
If you work with CSV files, spreadsheets, or JSON datasets, I'd especially appreciate feedback on these questions:
- Is the interface understandable when you first open it?
- Can you complete a basic cleaning task without much guidance?
- Which feature would make DataForge more useful for your work?
You can use the built-in sample data instead of real or confidential files.
Thanks for reading! I'm looking forward to hearing what other developers and data users think.
Top comments (0)