DEV Community

Juan Andrés Echeverri Saldariaga
Juan Andrés Echeverri Saldariaga

Posted on

Building an Art Auction System in Rust: Microservices, Event-Driven Architecture, and Semantic Search

#rust #microservices #eventdriven #apacheiggy

Building an Art Auction System in Rust

1. Introduction

In 2025, after years of building backends in various languages, I made a decision: write my startup's entire auction system in Rust. It wasn't a trend — it was a necessity.

Why Rust for an auction system?

A real-time auction system has very specific requirements:

  • Tens of thousands of concurrent WebSocket connections during live auction events.
  • Sub-50ms bid latency — nobody wants to lose a piece of art due to network lag.
  • Guaranteed correctness in the bidding engine: you cannot outbid yourself, increments must be valid, proxy bidding must be mathematically exact.
  • Safe concurrency — no data races, no forgotten locks, no memory leaks.

Rust with Tokio gave us all of that: memory safety without a GC, zero-cost abstractions for modeling complex domains with enums, and an async runtime that scales linearly.

The domain problem

Traditional art auctions are physical events: a room, a gavel, a curator certifying each piece's authenticity. Digitizing this process requires modeling:

  1. Inventory: artwork with a lifecycle (creation → curatorship → authentication → auction → sale).
  2. Curatorship: experts who verify authenticity and issue certificates.
  3. Auctions: events with start/end dates, multiple lots, reserve prices.
  4. Bids: a real-time engine with increments, proxy bidding, and outbid detection.
  5. Public catalog: an optimized view for thousands of concurrent users.
  6. Semantic search: finding artwork by description in multiple languages.
  7. Live notifications: bids, outbids, auction starts and ends.

Tech stack at a glance

Category Technology
Language Rust (edition 2024)
Async Runtime Tokio
Sync Communication gRPC (tonic) + protobuf
Async Messaging Apache Iggy (persistent streaming)
Databases PostgreSQL, ScyllaDB, Dragonfly, Qdrant
File Storage LocalStack S3
Auth JWT (jsonwebtoken) + Argon2
Real-Time WebSocket (Axum) + Iggy pub/sub
Observability Datadog

2. General Architecture

diagram 01 architecture general hosted at ImgBB — ImgBB

Image diagram 01 architecture general hosted on ImgBB

favicon ibb.co

The system follows a microservice architecture with 7 core services plus a WebSocket Gateway, all written in Rust with Tokio. Synchronous communication happens via gRPC through a unified Gateway exposing gRPC-Web to the frontend. Asynchronous communication uses Apache Iggy as a persistent message broker.

Key architectural patterns

1. Hexagonal Architecture — Each microservice follows the ports and adapters pattern. The domain sits at the center, pure Rust with no external dependencies. Interfaces (trait) define the ports, while infrastructure (PostgreSQL, Iggy, gRPC) provides swappable adapters.

2. Domain-Driven Event Design — Domain events are the primary contract between services. inventory-ms, auction-ms, and bid-ms publish events; auction-view-ms and multilingual-embedding-ms consume them to keep their data models synchronized.

3. Transactional Outbox — To guarantee reliable event delivery without two-phase commit, we combine the event write in the same SQL transaction as the business operation.

4. CQRS — auction-view-ms is a pure read model. It accepts no commands, only consumes events and maintains denormalized data in ScyllaDB for lightning-fast queries.

gRPC port map

diagram 12 sync communication hosted at ImgBB — ImgBB

Image diagram 12 sync communication hosted on ImgBB

favicon ibb.co

Each service exposes its own gRPC port. The Gateway acts as a reverse proxy, authenticating every request via JWT and verifying permissions before forwarding to the appropriate service:

// Real gateway code: route → required permission mapping
permission_map.insert(
    "/user_ms.UserService/GetUserById",
    Some("user")
);
permission_map.insert(
    "/auction_ms.AuctionService/CreateAuction",
    Some("admin")
);
permission_map.insert(
    "/inventory_ms.InventoryService/CreateItem",
    Some("curator")
);
Enter fullscreen mode Exit fullscreen mode
Role Typical Permissions
user Browse auctions, place bids, own profile
curator Inventory CRUD, curatorship, sources, categories
admin Create/manage auctions, manage users

3. The Heart of the Domain: The Inventory State Machine

The inventory is the most complex domain in the system. Each artwork goes through a lifecycle that we model as a state machine using a Rust enum.

ItemStatus: 12 states, explicit transitions

diagram 02 item status machine hosted at ImgBB — ImgBB

Image diagram 02 item status machine hosted on ImgBB

favicon ibb.co 
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub enum ItemStatus {
    Draft,
    UnderReview,
    Authenticated,
    NotApproved,
    ReadyForAuction,
    InAuction,
    AuctionCancelled,
    AuctionEnded,
    SoldPendingPayment,
    PaymentReceived,
    ReturnedToConsignor,
    Delivered,
}
Enter fullscreen mode Exit fullscreen mode

What's interesting here is how Rust lets us model valid transitions explicitly:

impl Item {
    pub fn transition_status(&mut self, new_status: ItemStatus) -> Result<(), String> {
        match (&self.status, &new_status) {
            | (ItemStatus::Draft, ItemStatus::UnderReview) => {}
            | (ItemStatus::UnderReview, ItemStatus::Authenticated) => {}
            | (ItemStatus::UnderReview, ItemStatus::NotApproved) => {}
            | (ItemStatus::Authenticated, ItemStatus::ReadyForAuction) => {}
            | (ItemStatus::ReadyForAuction, ItemStatus::InAuction) => {}
            | (ItemStatus::InAuction, ItemStatus::AuctionEnded) => {}
            | (ItemStatus::InAuction, ItemStatus::AuctionCancelled) => {}
            | (ItemStatus::InAuction, ItemStatus::SoldPendingPayment) => {}
            | (ItemStatus::SoldPendingPayment, ItemStatus::PaymentReceived) => {}
            | (ItemStatus::SoldPendingPayment, ItemStatus::ReturnedToConsignor) => {}
            | (ItemStatus::SoldPendingPayment, ItemStatus::ReadyForAuction) => {}
            | (ItemStatus::PaymentReceived, ItemStatus::Delivered) => {}
            | (from, to) => {
                return Err(format!(
                    "Cannot transition from {:?} to {:?}", from, to
                ));
            }
        }
        self.status = new_status;
        self.updated_at = Utc::now();
        Ok(())
    }
}
Enter fullscreen mode Exit fullscreen mode

The Rust compiler forces us to cover every combination. If we add a new state, the compiler demands we define its valid transitions. This eliminates an entire category of domain bugs.

The curatorship flow

A curator (user with curator permission) reviews each piece and issues a verdict:

pub enum AuthenticityStatus {
    Pending,        // Awaiting review
    UnderReview,    // Curatorship in progress
    Authentic,      // Authenticated → triggers transition to ReadyForAuction
    Inconclusive,   // Could not determine authenticity
}
Enter fullscreen mode Exit fullscreen mode

When a curator marks a piece as Authentic, a CuratorshipUpdated event is published. The same inventory-ms consumes that event (via Iggy) and completes the transition: UnderReview → Authenticated → ReadyForAuction.

This self-consumption of events might seem odd at first, but it's by design: it makes the flow reactive and decoupled. In the future, authentication could come from an external AI service or a third party, and inventory-ms would react the same way.

4. The Bidding Engine: The Most Complex Business Logic

The bidding engine lives in bid-ms and is, without doubt, the most delicate component in the system. It implements proxy bidding (similar to eBay), with precise increment rules, reserve price handling, and outbid detection.

BidPlace: the root aggregate 

pub struct BidPlace {
    previous_bid: Option<Bid>,    // Current winning bid
    current_bid: Option<Bid>,     // New incoming bid
    max_bid: Option<Bid>,         // Previous winner's max bid
    bid_table_registry: BidTableRegistry,
    reserve_price: Money,         // Lot reserve price
    bids_to_store: Vec<Bid>,      // Bids generated for persistence
    events_to_publish: Vec<BidEvent>,  // Events to publish
    outbids: HashMap<Uuid, Uuid>, // Outbid users → user_id: bid_id
}
Enter fullscreen mode Exit fullscreen mode

BidPlace is not a persisted entity. It's a temporary aggregate built in memory during the bidding process. It runs all validations, generates the necessary proxy bids, and produces the events and records to persist.

BidTable: progressive increments 

pub struct BidTable {
    pub bid_increment_ranges: Vec<(f64, f64, f64)>,
    pub currency: Currency,
}

impl BidTable {
    pub fn new(currency: Currency) -> Self {
        let bid_increment_ranges = match currency {
            | Currency::USD => vec![
                (0.0, 500.0, 5.00),
                (500.0, 2000.0, 10.00),
                (2000.0, 5000.0, 25.00),
                (5000.0, 10000.0, 50.00),
                (10000.0, f64::INFINITY, 100.00),
            ],
            | Currency::COP => vec![
                (0.0, 500000.0, 10000.00),
                (500000.0, 10000000.0, 50000.00),
                (10000000.0, 50000000.0, 100000.00),
                (50000000.0, 100000000.0, 500000.00),
                (100000000.0, f64::INFINITY, 1000000.00),
            ],
        };
        // ...
    }
}
Enter fullscreen mode Exit fullscreen mode
Currency Range Increment
USD $0 – $500 $5
$500 – $2,000 $10
$2,000 – $5,000 $25
$5,000 – $10,000 $50
$10,000+ $100

How a bid works step by step

diagram 05 bid flow hosted at ImgBB — ImgBB

Image diagram 05 bid flow hosted on ImgBB

favicon ibb.co

The engine rules are:

  1. You cannot outbid yourself: If you're the current winner and want to bid higher, you must place a multiple of the base increment.
  2. Proxy Bidding: If you bid $5,000 on a lot with a current bid of $100, the system records your maximum as $5,000 but only displays a bid of $105 (the next valid increment). If someone else bids, the system automatically counter-bids on your behalf up to your maximum.
  3. Reserve Price: If the reserve price hasn't been met, the system generates a SystemProxy bid at the reserve value.
  4. Outbid Detection: When someone is outbid, it's recorded in the outbids table for push notifications.

Here's the most interesting competitive flow: when there's an active max_bid:

fn handle_bid_below_max(
    &mut self, current_bid: Bid, max_bid: &Bid,
) -> Result<(), BidPlaceError> {
    // The new bidder (current_bid.user_id) bids less than the
    // current winner's max_bid. The system generates a system
    // proxy bid at the next valid increment for the max_bid owner.

    let next_amount = bid_table.next_valid_increment(
        current_bid.money.amount as f64
    ) as i32;

    let proxy_bid = self.create_bid(
        max_bid.user_id,     // The max_bid owner stays in the lead
        next_amount,         // At the next increment
        &current_bid,
        false,               // Not a max bid
        BidSource::SystemProxy,
    );

    self.current_bid = Some(proxy_bid);
    // ... store + publish events
}
Enter fullscreen mode Exit fullscreen mode

This design means the frontend only needs to send { amount: 5000, lot_id: X } to place a max bid. The server handles all the proxy bidding math.

5. Event-Driven Architecture with Apache Iggy

Apache Iggy is the message streaming system I chose. It's persistent (messages are stored on disk), Rust-native, and much lighter than Kafka for our use case.

Topics and event flow

diagram 06 event map hosted at ImgBB — ImgBB

Image diagram 06 event map hosted on ImgBB

favicon ibb.co
Topic Producer Consumers
inventory.events inventory-ms inventory-ms (self-consume), auction-view-ms, multilingual-embedding-ms
auction.events auction-ms inventory-ms, auction-view-ms, websocket-gateway
lot.events auction-ms inventory-ms, auction-view-ms, multilingual-embedding-ms, websocket-gateway
bid.events bid-ms auction-ms, auction-view-ms, websocket-gateway

Transactional Outbox Pattern

diagram 10 outbox pattern hosted at ImgBB — ImgBB

Image diagram 10 outbox pattern hosted on ImgBB

favicon ibb.co

The Transactional Outbox pattern is essential for guaranteeing reliable event delivery. The flow is:

  1. In the same SQL transaction that executes the business operation, we insert a row into the outbox_events table with the event serialized as JSON.
  2. A background process (OutboxProcessor) runs every 5 seconds, looks for unpublished events, and sends them to Iggy.
  3. If publishing fails, it retries every 30 seconds up to a configurable maximum. 
// Fragment from IggyOutboxPublisher
#[async_trait]
impl OutboxPublisherPort for IggyOutboxPublisher {
    async fn publish(&self, event: &OutboxEvent) -> ApplicationResult<()> {
        let payload = serde_json::to_string(&event.payload)?;

        self.event_publisher
            .publish_event(
                self.tenant_id,
                &self.topic,
                &event.event_type,
                &payload,
            )
            .await?;

        self.repository
            .mark_as_published(event.id, self.publisher_id())
            .await?;

        Ok(())
    }
}
Enter fullscreen mode Exit fullscreen mode

Why not publish directly to Iggy inside the transaction? Because Iggy doesn't support distributed transactions. The outbox pattern gives us exactly-once delivery semantics without two-phase commit.

Real example: inventory-ms self-consumption

When a curator updates a curatorship to Authentic:

  1. inventory-ms persists the change + outbox_event in the same TX.
  2. OutboxProcessor publishes CuratorshipUpdated to inventory.events.
  3. The same inventory-ms consumes that event and executes the state transition: UnderReview → Authenticated → ReadyForAuction.
  4. It publishes ItemUpdated, which is consumed by auction-view-ms to update the public catalog.

This self-consumption pattern is intentional: it separates command logic (receiving the gRPC request) from reaction logic (completing secondary transitions).

6. CQRS: The Read Model with ScyllaDB

A common mistake in microservices is sharing databases. We take separation to the extreme: each service has its own independent database, and data flows between services exclusively through events.

diagram 13 database architecture hosted at ImgBB — ImgBB

Image diagram 13 database architecture hosted on ImgBB

favicon ibb.co

auction-view-ms is our pure read model. It accepts no commands, exposes no write endpoints. It only:

  1. Consumes events from 4 Iggy topics (auction.events, inventory.events, lot.events, bid.events).
  2. Denormalizes data to optimize public catalog queries.
  3. Stores in ScyllaDB (wide-column database, optimized for reads) and Dragonfly (in-memory cache). 
// Auction event consumer in auction-view-ms
pub async fn start_auction_events_consumer(
    &self, tenant_id: u32
) -> ApplicationResult<()> {
    self.event_consumer
        .consume_events(
            tenant_id,
            AUCTION_EVENTS_TOPIC,
            AUCTION_CONSUMER_NAME,
            |payload, _message_id, _partition_id, _offset| {
                let service = self.clone();
                Box::pin(async move {
                    let event: AuctionEvent =
                        serde_json::from_str(&payload)?;
                    service.auction_management_service
                        .process_auction_event(&event)
                        .await?;
                    Ok(())
                })
            },
            None,
        )
        .await
}
Enter fullscreen mode Exit fullscreen mode

Why ScyllaDB over PostgreSQL for the read model? Because the public catalog receives queries with multiple filters (category, status, price range, artist) and must respond in <10ms under high concurrency. ScyllaDB, being wide-column, handles this access pattern far better than PostgreSQL.

Additionally, Dragonfly (a Rust-based Redis replacement) serves as cache for:

  • Current bids: the active bid for each lot is cached with a TTL.
  • Outbids: outbid user IDs are cached for instant push notifications.

7. Semantic Search with Qdrant

One of the most powerful features is multilingual semantic search. Users can search for artwork in their native language and find relevant results even when the description is in another language.

diagram 07 semantic search hosted at ImgBB — ImgBB

Image diagram 07 semantic search hosted on ImgBB

favicon ibb.co

Architecture

multilingual-embedding-ms maintains two collections in Qdrant:

Collection Content Embedding of...
lots Each lot's title + description "Oil on canvas, Italian Renaissance..."
sources Each source's name + description (artist, house) "Leonardo da Vinci, Renaissance painter..."

Embedding generation (event-driven)

When a lot or source is created or updated, the service consumes the corresponding event from Iggy and generates the embedding:

pub async fn upsert_embedding(
    &self,
    id: String,
    text: String,
    metadata: serde_json::Value,
    collection_type: CollectionType,
) -> ApplicationResult<()> {
    let embeddings = self.model
        .generate_embeddings(vec![text.clone()])
        .await?;

    let embedding_vector = embeddings.first().ok_or_else(|| {
        ApplicationError::Internal("No embedding generated".to_string())
    })?;

    let embedding = Embedding::new(
        id, text, embedding_vector, metadata, collection_type
    );

    self.repository.ensure_collection(collection_type).await?;
    self.repository.upsert_embedding(embedding).await?;

    Ok(())
}
Enter fullscreen mode Exit fullscreen mode

Search

When a user searches for "Italian Renaissance art", the service:

  1. Generates an embedding of the search text using the same model.
  2. Finds the most similar vectors in Qdrant using cosine similarity.
  3. Returns results sorted by relevance score.

The embedding model is multilingual, meaning "Italian Renaissance art" in English and "arte renacentista italiano" in Spanish produce vectors that are close together in the embedding space.

8. Real-Time with WebSocket + Iggy

Live auctions require real-time notifications. Our websocket-gateway is an independent service that:

  1. Accepts WebSocket connections with JWT authentication.
  2. Consumes 3 Iggy topics (auction.events, lot.events, bid.events).
  3. Forwards events to subscribed clients based on their preferences.

diagram 08 websocket flow hosted at ImgBB — ImgBB

Image diagram 08 websocket flow hosted on ImgBB

favicon ibb.co

Protocol messages

Client → Server:

{ "action": "subscribe",   "topics": ["auction.events", "bid.events"] }
{ "action": "unsubscribe", "topics": ["lot.events"] }
{ "action": "pong" }
Enter fullscreen mode Exit fullscreen mode

Server → Client:

{
  "topic": "bid.events",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "bid": { "user_id": "...", "lot_id": "...", "amount": 500, ... },
  "occurred_at": "2025-06-15T12:00:00Z"
}
Enter fullscreen mode Exit fullscreen mode

The WebSocket gateway handles 10k+ concurrent connections with very low memory consumption thanks to Tokio's lightweight async task model.

9. Lessons Learned and Performance

Impact metrics

Metric Before After Improvement
p99 bid latency 120ms 45ms -62.5%
RAM consumption — -20% Memory leak fixed
WebSocket connections — 10k+ concurrent Pub/sub via Iggy
Message throughput — 50k msgs/sec Event streaming

Problems solved along the way

1. Memory leak detected with Datadog

One of the biggest challenges was a slow but steady memory leak in bid-ms. Datadog showed linear RAM growth in the Rust heap. The cause: in BidPlace, we were cloning the Vec<Bid> of bids_to_store multiple times without clearing the original vector after validation. The fix was using std::mem::take() to move (not clone) the data:

let bids = std::mem::take(&mut bid_place.bids_to_store);
let events = std::mem::take(&mut bid_place.events_to_publish);
Enter fullscreen mode Exit fullscreen mode

2. PostgreSQL connection pool

Each service opens its own pool to its dedicated database. We learned the hard way that the default max_connections=10 isn't enough for traffic spikes during live auctions. Now we configure larger pools with min_connections for warm connections:

PgPoolOptions::new()
    .max_connections(50)
    .min_connections(5)
    .acquire_timeout(Duration::from_secs(30))
    .idle_timeout(Duration::from_secs(300))
    .test_before_acquire(true)
Enter fullscreen mode Exit fullscreen mode

3. Strategic caching with Dragonfly

bid-ms now caches each lot's current bid in Dragonfly immediately after processing it. This reduces PostgreSQL reads for the next bid on the same lot (the most common case). The TTL is configured to expire after the auction ends.

What I'd do differently

  • gRPC streaming: currently the gateway is request-response. For bids, server-side streaming would eliminate connection overhead.
  • Shared event schemas: proto files are duplicated across services. A shared crate with protos would be more maintainable.
  • Integration testing: we have solid unit tests in the domain, but integration tests with Iggy are difficult to set up locally.

10. Conclusion and Next Steps

Building this system took me from being a developer who "used Rust" to truly understanding how Rust's type safety and concurrency model change the way you design distributed systems.

What I learned

  • Rust enums are the best tool for modeling complex domains. The ItemStatus state machine is nearly impossible to implement incorrectly when the compiler forces you to cover every transition.
  • The Transactional Outbox pattern is indispensable in event-driven microservices. Without it, events get lost when the message broker goes down.
  • ScyllaDB + Qdrant is a powerful combination for systems that need both fast structured queries and semantic search.
  • Apache Iggy is a real Kafka alternative for Rust teams. Less operational overhead, same persistence guarantees.

Roadmap

  • Payment system: Stripe/PayPal integration for the SoldPendingPayment → PaymentReceived flow.
  • Push notifications: Firebase Cloud Messaging for outbid alerts when the app is in the background.
  • Deploy to Kubernetes: currently on a homelab with Proxmox + Talos Linux (in progress).
  • Open source: I plan to release the system's core under an MIT license.

Interested in a specific part? In upcoming articles in this series, I'll dive deeper into the bidding engine, Apache Iggy integration, Talos Linux deployment, and modeling complex domains with Rust.

juansaldarriaga.com · GitHub

#rust #microservices #architecture #eventdriven #webdev

Top comments (0)