Too often we see development teams struggle to finish a project. Here is a nice open-source framework I wrote to help with your upcoming microservices (or legacy!) project: the pipeline framework.
https://github.com/mbarcia/CSV-Payments-PoC/tree/main/pipeline-framework
Welcome to the Pipeline Framework - a robust, scalable, and maintainable solution for processing data through a series of steps with built-in benefits for high-throughput, distributed systems.
- Step-based Processing: Each business logic operation is encapsulated in a step that implements a specific interface.
- Reactive Programming: Steps use Mutiny reactive streams for non-blocking I/O operations.
- Type Safety: Steps are strongly typed with clear input and output types that chain together.
- Configuration Management: Steps can be configured globally or individually for retry logic, concurrency, and more.
- Observability: Built-in metrics, tracing, and logging for monitoring and debugging.
It doesn't ask for much, only that you translate the business case into a "pipeline" model with each step having a defined input/output. You only write the business logic for each step and the framework will do all the Kubernetes heavy lifting for you (messaging, auto-persist, error handling, etc.)
I'm in the process of making it an independent set of JARs on Maven central. At the moment it is included in the "CSV Payments Processing" system which is its "reference implementation". Reach out to me in the comments if you would like to know more. Cheers.
Top comments (1)
Great framework! Are you planning to support event-driven setups or tools like Kafka for better scalability?