⚙️ Designing a Data Bus for Uneven Logistics Workloads

Logistics systems rarely receive data at a perfectly steady rate. Evrone encountered this challenge while working with a logistics company whose B2B integrations relied on Apache NiFi.

The database received very different workloads. One customer could generate thousands of small events, while another could send a huge JSON document. Because processing was synchronous, background operations could consume database resources and affect other systems, including 1C.

Evrone's solution introduced an intermediate Python service and Kafka as a controlled processing layer.

🔄 Architecture

The new flow works in several stages:

1. Apache NiFi prepares incoming data.
2. Kafka receives messages through the appropriate topic.
3. Python retrieves messages in batches.
4. Business rules determine how each batch should be processed.
5. The database receives data at a controlled rate.

This approach reduced the impact of workload spikes and created a clearer scaling path. The client can also configure how frequently the service retrieves messages from Kafka.

🧩 Extending the Architecture

After completing the data-bus project, Evrone helped integrate another customer. The team created mapping scripts that converted external data into the logistics company's required format. NiFi continued to provide the ETL pipeline for validation, transformation, and database loading.

The same architecture later supported a public shipment tracking service.

Evrone used Python and Django for the backend because the client's existing ETL ecosystem already relied on Python. React powered the frontend, while a separate database isolated tracking operations from the company's core systems.

The tracking interface included:

• tabular shipment information;
• expandable lists;
• modal windows;
• automatically generated TypeScript types from YAML through swagger-typescript-api.

👥 Delivery

The first project phase lasted nine months and involved a DevOps Lead, two Python developers, two DevOps engineers, a React developer, and a Delivery Manager.

Evrone is now preparing an authenticated version of the tracker with customer accounts, legal-entity data, documents, timestamps, and detailed shipment history.

🚚 How Evrone Turned Unpredictable ETL Traffic into a Scalable Flow.