From Stream to Database: Processing Market Data with Spring Boot, Redis, and Flyway

Hello everyone!

In my last post, we saw how our Python service collects B3 data and publishes it to Kafka. Today, we take a crucial step: consuming this data and making it useful for our brokerage ecosystem.

I’ll introduce the Broker Asset API, the Java microservice responsible for managing the asset catalog, keeping prices updated, and serving this information with ultra-low latency.

---

🎯 MVP Focus (Minimum Viable Product)

Before diving into the code, a quick disclaimer: we are building the foundation. At this stage, the goal is to ensure the end-to-end flow works seamlessly.

The focus is on delivering core value: making data available and performant. In the future, we will revisit this service to add unit tests, refine exception handling, and increase resilience.

---

🏗️ The Pillars of the Asset API

For this MVP, I focused on four main implementation points:

1. Database Evolution with Flyway

To ensure our MySQL schema is versioned and reproducible, I used Flyway. We created the assets table to store the ticker, name, current price, and asset status.

• Technical Highlight: Using an index on the ticker field ensures that symbol-based queries are extremely fast.

2. Kafka Consumer: Real-Time Reactivity

The API "listens" to the trading-assets-market-data-v1 topic. As soon as the Python service publishes a new price, our consumer captures the message and triggers the update flow.

• Ordering Guarantee: Since we use the ticker as the Kafka key, we process updates in the exact order they were generated.

    @KafkaListener(
            topics = "trading-assets-market-data-v1",
            groupId = "broker-asset-api"
    )
    public void consume(AssetMarketDataDTO dto) {
        log.info("Received market data for: {}", dto.getTicker());

        assetService.updateAsset(dto);
    }

3. Service Layer: Hybrid Persistence (SQL + Redis)

The AssetService is where the business logic resides. Upon receiving a new price, it performs two operations:

1. MySQL Persistence: Updates the asset record (or creates a new one) to ensure data consistency.

1. Cache Update (Redis): The price is sent to a Redis cache with the key market:price:{ticker}.
2. This allows other system components to query prices instantly without overloading the relational database.

4. Controller: Serving the Information

We created REST endpoints so that other services or the front-end can query the asset catalog:

• GET /api/v1/assets: Lists all available active assets.
• GET /api/v1/assets/{ticker}: Returns details and the updated price for a specific asset.

---

🛠️ What’s Next?

The Asset microservice is the foundation that ensures the system knows "what" is being traded and "for how much." Since we are following an MVP strategy, the focus was on establishing this data contract and basic persistence.

But an asset alone doesn't make a brokerage. It needs to belong to someone.

In the next post, we’ll increase the complexity and talk about the Broker Wallet API. We will explore:

• Managing user financial balances.
• Linking asset custody to customer wallets.
• How the system reflects market variations in the investor's equity.

What do you think of this step-by-step approach to building an ecosystem? Let me know your thoughts in the comments!

---

🔎 About the series

⬅️ Previous Post: Tooling Tips.

📘​ Series Index: Series Guide.

---

Links:

• LinkedIn