Building an AI-Powered Bill Splitter: OCR, LLMs, and Real-time State

Doru Prodan — Sun, 18 Jan 2026 11:35:59 +0000

The Developer’s Dilemma: The Post-Lunch Math Problem

We’ve all been there. A team lunch ends, a long receipt arrives, and suddenly everyone is squinting at tiny font sizes, trying to calculate tax and tip for their specific order of truffle fries and a club sandwich. For developers, this isn't just a social annoyance; it’s a logic problem begging for an automated solution.

This is the problem space that Hackbill occupies. By leveraging AI-powered scanning and real-time synchronization, it eliminates the manual overhead of debt collection among friends. In this article, we’ll dive into the technical architecture required to build a high-performance bill-splitting engine, focusing on OCR pipelines, LLM-based data structuring, and real-time state management.

1. The OCR Pipeline: From Pixels to Raw Text

The first challenge in building a tool like Hackbill is converting a potentially blurry, low-light smartphone photo into machine-readable text. Traditional Optical Character Recognition (OCR) has come a long way, but receipts present unique challenges: variable fonts, crumpled paper, and complex layouts (columns for quantity, name, and price).

The Pre-processing Step

Before hitting an OCR engine, the image usually needs a pipeline to improve accuracy:

Grayscale Conversion: Removing color noise.
Perspective Correction: Using edge detection (like Canny) to find the receipt corners and perform a 4-point perspective transform.
Adaptive Thresholding: Handling uneven lighting across the paper.

Choosing an Engine

While Tesseract is the open-source standard, modern cloud-based solutions like AWS Textract or Google Cloud Vision provide better results for multi-column layouts because they return "blocks" and "forms" rather than just raw strings.

2. Using LLMs for Structured Data Extraction

Raw OCR output is often a mess of unstructured strings. For example, a line might read 1 BU RGER $15 .00. Traditional regex-based parsing is incredibly brittle here because every POS system formats receipts differently.

This is where Large Language Models (LLMs) like GPT-4o or Claude 3.5 Sonnet shine. Instead of writing 500 lines of regex, you can pass the raw OCR text to an LLM with a system prompt designed to return a JSON schema.

Example Implementation (Node.js)

const extractReceiptItems = async (rawText) => {
  const prompt = `
    Extract the items, quantities, and prices from this receipt text.
    Return ONLY a JSON array of objects with keys: name, quantity, price.
    Text: "${rawText}"
  `;

  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: prompt }],
    response_format: { type: "json_object" },
  });

  return JSON.parse(response.choices[0].message.content);
};

By using AI-powered scanning, Hackbill can intelligently identify which lines are items and which are metadata (like the date or the server's name), significantly reducing the "Review" phase for the user.

3. Real-time Collaboration and State Management

Once the receipt is scanned and items are extracted, the next technical hurdle is the "Share and Claim" phase. In a developer-centric view, this is a distributed state problem. If three people are looking at the same bill, we need to ensure that two people don't claim the same beer at the same time.

The Tech Stack for Live Syncing

To achieve the "see who's claiming what live" feature mentioned in the Hackbill workflow, you generally have three options:

WebSockets (Socket.io): Best for low-latency, bidirectional communication.
Server-Sent Events (SSE): Good for one-way updates (server to client).
Real-time Databases (Supabase/Firebase): The most efficient for rapid development, as they handle the pub/sub logic out of the box.

Handling Conflicts

When a user clicks "Claim," the client should send an optimistic update to the UI while the backend validates the request. If the item is already claimed by another user_id in the database, the backend rejects the transaction, and the UI rolls back.

4. The Math of Fair Tip Distribution

One of the most innovative features of the Hackbill philosophy is Fair Tip Distribution. Most people simply split the tip evenly, but that’s technically unfair. If I ordered a $5 salad and you ordered a $50 steak, an even split of a $10 tip means I’m overpaying significantly relative to my consumption.

The Algorithm

Hackbill ensures only people who claimed items pay their share of the tip. Mathematically, this is calculated as a weighted percentage:

Calculate Subtotal: Sum of all claimed items.
Calculate User Subtotal: Sum of items claimed by User A.
Calculate Proportion: User A Proportion = User A Subtotal / Total Subtotal.
Apply Tip/Tax: User A Total = User A Subtotal + (Total Tip * User A Proportion) + (Total Tax * User A Proportion).

Implementing this logic in your backend ensures that the final "Claim" amount is mathematically sound and socially frictionless.

5. Security and Privacy Considerations

As developers, we must consider the sensitivity of receipt data. Receipts often contain the last four digits of a credit card, the merchant's address, and dining habits.

Data Minimization: Only store the item names and prices. Discard the raw image after successful parsing.
Short-lived Sessions: Use unique, UUID-based URLs for sharing bills that expire after a set period (e.g., 24 hours).
Encryption: Ensure all data in transit is handled via HTTPS and PII (Personally Identifiable Information) is encrypted at rest.

Conclusion: Solving Social Friction with Code

Building a tool like Hackbill is a masterclass in combining various modern technologies—Computer Vision, Natural Language Processing, and Real-time Web Systems—to solve a mundane human problem.

For developers looking to build similar applications, the takeaway is clear: don't fight the unstructured nature of the real world with rigid regex. Embrace LLMs for data extraction, use real-time sync for collaboration, and always ensure your math handles edge cases like weighted tip distribution.

Ready to stop doing manual math? Check out Hackbill to see these technical principles in action and streamline your next group dinner.

Key Takeaways for Devs:

OCR is the start, not the end: Use LLMs to structure the messy data OCR produces.
Real-time is non-negotiable: Use WebSockets or Supabase for a seamless "claiming" experience.
Weighted Math > Simple Math: Always calculate shares based on proportional subtotal to ensure fairness.

DEV Community: Doru Prodan