Every time I built data-intensive systems in .NET, handling bulk data ingestion always felt fragmented. Different databases required completely different boilerplate, performance quirks, and error-handling strategies.
So I spent the week solving it cleanly from first principles.
I’m excited to introduce FastIngest: a modular, high-throughput bulk ingestion framework for .NET. Published completely open source under the permissive MIT license.
Instead of writing provider-specific bulk copy hacks for every new project, FastIngest gives developers a unified, highly optimized API designed for heavy write loads and streaming pipelines.
📦 Available today on NuGet (v0.3.2):
Core Pipeline Engine: FastIngest.Core
Relational Providers: SQL Server, PostgreSQL, MySQL, SQLite
Document & Cloud Stores: Azure Cosmos DB, MongoDB
Search & Indexing: Elasticsearch
Extensions: Pipeline utilities & dependency injection helpers
Without an official announcement, the suite organically crossed 1,600+ downloads across 9 packages in its first 48 hours.
The project is free for both commercial and non-commercial use:
📖 Documentation: https://lnkd.in/gVKAs-Qt
💻 GitHub (MIT License): https://lnkd.in/gRgqzF3S
📦 NuGet: Search FastIngest or find the packages under aebibtech
If you are building data migration tools, event processors, or high-throughput write services in .NET, I’d love for you to take it for a spin. Feedback, issues, and PRs are welcome!
Top comments (1)
The fragmentation problem is the right thing to solve: each provider's bulk copy path has its own API, quirks and error semantics, so one streaming abstraction over them removes real duplicated boilerplate. Keeping the providers modular should also keep the core pipeline free of provider-specific hacks.