You have built your own CSV importer, but as your organization, your product and its requirements grow, you ask yourself if switching from your homegrown solution to a pre-built library could be beneficial.
As you grow and onboard more and more clients, your importer increases in complexity and chances are, you can’t afford to keep up with the cost, talent, and time it’ll take to build and maintain your own CSV import solution. Data import cases will increase in complexity, requirements especially performance requirements will rise accordingly and at some point, a data importer without smart features just won’t be able to do the job successfully on a long-term basis.
In general, deciding on build vs buy is crucial, to begin with:
The focus of your developer resources should always be on the product’s core features. The implementation time and the maintenance of secondary/support functionalities such as importer, login, sign up, etc. shall be reduced as far as possible.
However, even for companies that decided to build a CSV importer in-house at some point, replacing the homegrown importer at a later point is worth considering.
Here are some signs that should get you thinking about switching:
New requirements on security & compliance when tapping into new markets
Increased implementation and maintenance efforts for advanced as well as smart data mapping and cleaning features
High implementation services for customers
Limitations in adoption and customer onboardings due to long development time
There is a risk of brain drain in case a key developer leaves your team or company
In general, software libraries that offer 24/7 support with a quick response time (24h), are well maintained, and frequently updated (especially in regards to major version releases of common frontend frameworks) can be a valuable solution to replace your homegrown importer.
To provide some numbers and a potential guideline that can support in deciding to make or buy, we created some scenarios covering different levels of import complexities:
Is an off-the-shelf CSV importer worth the investment?
Deciding to buy instead of build or replace a homegrown component that already took internal resources should always be driven by a positive ROI calculation considering the investment for setup but also long-term maintenance of the solution.
As the import cases and their complexity can vary significantly, it is important to distinguish and consider different use cases separately.
To provide some reference data and support decision making we will look at four different import scenarios, including their requirements, and discuss their pros and cons:
1. Simple CSV importer for internal use only
As a first case, we consider a basic and straightforward data import case. The data is imported by internal teams only, the target data model is rather basic, containing around 5 – 10 columns, no complex validations, and the column mapping is conducted using a simple dropdown view without automation via AI or any other algorithm.
This case serves the minimum requirements, usually has no specific requirements regarding styling or UI, and is implemented quite fast.
Experienced software engineers estimate around 2 months of development time, requiring a team of 3-4 engineers working on it.
The simplicity that allows the fast implementation of the importer, unfortunately, comes with some drawbacks that need to be addressed as they will significantly affect the use and manual effort related to data imports:
Simple and static importers require effortful preparation of the data, including manual mapping and cleaning with every file imported.
The performance of the importer is often neglected in this simple data import case and can lead to errors and failure of the importer as soon as larger datasets are imported.
Maintenance of static importers is high as with every change in the target data schema, the importer has to be adapted, requiring continuous developer resources.
2. Advanced CSV Importer
The second scenario we consider is a more advanced data importer that can be provided to non-technical, external uses as for customer self-service imports.
To ensure a smooth experience for your clients, the advanced CSV importer should fulfill some additional requirements and offer features such as:
Advanced UI that guides non-tech users through the workflow seamlessly
Basic data mapping enabled by fuzzy matching
Data cleaning functions and data validation features
The efforts to develop a CSV importer as described above can be estimated at around 3 months of development time for an Engineering team of 3-4 developers.
Even though this more advanced CSV importer can enable self-service imports for your clients and takes manual efforts from internal teams due to smart features and data validation and cleaning functions, however, there are drawbacks that should be considered making a decision of in-house development of this advanced importer:
Maintaining high performance with the upload of large data is something most homegrown importers struggle with.
Maintenance requirements of the data importer provided to clients for self-service are high and continuous developer resources are needed to update the importer with changing target data schemas should be considered with the decision to keep an advanced CSV importer in-house.
3. AI-assisted, high-performance CSV importer
The third scenario we are looking at is an AI-assisted, high-performance importer that is optimized for large data, is characterized by a simple and advanced UI and includes AI-assisted smart features for automated data mapping, validation, and cleaning.
Being able to provide a CSV import solution equipped with all features mentioned above will not only provide a seamless data onboarding experience to your clients but will also gain significant time savings for your internal teams.
The development time of importers on this level can be estimated at around 8 months (considering a development team of 3-4 developers) and more to be added, as additional features will be required with a growing customer base and increased import complexity over time.
4. nuvo Importer
Implementing a plug-and-play import library is considered the fourth scenario in this article. The nuvo Importer is a component that is embedded in the application’s front-end using one of the following frontend frameworks: React, Angular, Vue.js, and plain JavaScript.
The component enables you to guide your users through our data onboarding workflow, including uploading and selecting a file, choosing the preferred sheet and the header row, matching the imported columns to the columns of your target data model, and cleaning the imported data.
Getting started usually takes a couple of minutes only and provides you with a data importer offering:
- Fully custom styling
- Compliance to the highest security and data privacy standards
- AI-supported column matching
- Advanced data validation and cleaning as well as 24/7 customer support, maintenance, and continuous feature and performance improvements
To conclude, make vs buy or maintaining homegrown vs replacing with an off-the-shelf solution is a decision that has to be considered carefully and depends on the requirements and resources available.
As both variables can develop and vary over time, reconsidering a decision that was once made, can often be worth the investment.
Top comments (0)