Parsing SMS Transactions Is a Data Quality Problem, Not Just a Regex Problem

One of the more interesting parts of building FinLedger has been working with transaction data that doesn't arrive in a clean format.

When transaction information comes from SMS messages, the application isn't receiving a nice API response like:

{
  "amount": 500,
  "merchant": "Example Store",
  "type": "DEBIT"
}

Instead, it may receive human-readable text.

That changes the problem completely.

The input is messy

A transaction message might conceptually contain:

Your account has been debited by Rs. 500
at Example Store.

Another message from a different bank may use completely different wording.

So the parser has to identify useful information from text rather than simply reading predefined fields.

The pipeline becomes something like:

SMS
 ↓
Parser
 ↓
Extracted transaction data
 ↓
Validation
 ↓
Transaction model
 ↓
Local database

The important part is that parsing and accepting data are not the same thing.

Regex is useful, but it isn't the whole solution

Regular expressions can be useful for extracting predictable pieces of information.

For example, an amount might follow a pattern such as:

Rs. 500
INR 500.00
₹500

A parser can look for these patterns.

But extracting:

500

doesn't prove that the message represents a valid transaction.

You still need to answer:

Is this actually a financial transaction?
Is it a debit or credit?
Can the amount be trusted?
Was the transaction already processed?
Can the date be determined?

Extraction gives you candidate data.

Validation decides whether that data should enter your accounting system.

False positives are dangerous

For a finance application, an incorrect transaction can be worse than a missed transaction.

Imagine a parser incorrectly interprets a promotional SMS as:

Expense = ₹500

The application has now corrupted the user's financial history.

That's why automated parsing should be conservative.

A useful mindset is:

Unknown
   ↓
Try to parse
   ↓
Validate
   ↓
Accept only if sufficiently reliable

Don't force every message into a transaction.

Idempotency matters too

Automation creates another problem: the same message might be processed more than once.

If the application sees the same transaction twice:

SMS
 ↓
Parse
 ↓
Create transaction

and later processes it again:

Same SMS
 ↓
Parse
 ↓
Create another transaction

the user's balance can become incorrect.

So automated ingestion needs some way to recognize that a transaction has already been processed.

The general principle is:

Processing the same input twice should not accidentally create two financial events.

This is the same kind of problem you'll encounter in backend systems, payment processing, message queues, and distributed systems.

Store useful raw information

Another useful design consideration is keeping enough information to investigate parsing decisions.

Instead of only storing:

amount = 500
type = DEBIT

the system may benefit from retaining appropriate source information or metadata that allows the application to understand where the transaction came from.

Why?

Because when parsing goes wrong, you need evidence.

Without the original context, debugging becomes much harder.

Parsing should be treated as an evolving system

Bank and payment messages aren't necessarily designed as a stable API contract for your application.

Formats can differ between institutions and can change over time.

That means a parser shouldn't be treated as:

Write regex once
↓
Never touch it again

A better mental model is:

Input formats
     ↓
Detection
     ↓
Extraction
     ↓
Normalization
     ↓
Validation
     ↓
Transaction

Each stage has a responsibility.

That makes failures easier to understand.

The bigger lesson

Building an automated transaction parser has reinforced something I keep seeing across software engineering:

Real-world data is rarely as clean as the data in your database schema.

The database might expect:

amount
type
date
merchant

But the outside world gives you:

unstructured text
different formats
missing fields
unexpected wording
duplicate events

The engineering challenge is building a reliable boundary between those two worlds.

And that principle applies far beyond SMS parsing.

APIs can change.

User input can be malformed.

Third-party integrations can behave unexpectedly.

Files can have inconsistent formats.

Your application should treat external data as untrusted input and make it trustworthy before building business logic on top of it.

android #kotlin #fintech #softwareengineering #parsing #databases #architecture #jetpackcompose #buildinpublic