DEV Community: Salil Chincholikar

Building a Financial Reconciliation Platform, Not a Matching Engine

Salil Chincholikar — Wed, 24 Jun 2026 10:19:07 +0000

Most reconciliation projects start with a simple assumption.

Find two records with matching references and mark them as reconciled.

That sounds reasonable until real-world financial data arrives.

The ERP says one thing.
The payment provider says another.
The settlement report says something else.
The bank statement tells a different story.

And finance teams need answers before month-end closes.

At that point, reconciliation stops being a matching problem and becomes a workflow problem.

The Simplified View

Most discussions about reconciliation look something like this:

ERP
 ↕
Bank

The expectation is straightforward.

An invoice exists in the ERP.

A payment appears in the bank.

Match them.

Done.

Unfortunately, real financial systems are rarely that simple.

The Real World

A single customer payment may appear across multiple systems.

ERP
CRM
Payment Provider
Settlement Report
Bank Statement

Each system represents a different perspective of the same financial event.

The ERP represents what should happen.

The payment provider represents what was processed.

Settlement reports represent what is expected to settle.

The bank statement represents actual cash movement.

These views often disagree.

The goal of reconciliation is not just finding matches.

It is explaining those differences.

Payment Success Does Not Mean Settlement Success

One of the most common misconceptions is treating successful payment collection as the end of the workflow.

Consider a payment of ₹10,000.

Payment Provider:

Payment Success
₹10,000

Settlement Report:

Gross Amount      ₹10,000
Gateway Fee         ₹250
GST                  ₹45
Net Settlement     ₹9,705

Bank Statement:

₹9,705

A naive reconciliation system reports a mismatch.

A useful reconciliation system understands the relationship between gross collections, fees, taxes, and net settlements.

In financial operations, matching amounts is rarely enough.

Understanding the transaction lifecycle is far more important.

Reconciliation Is Not a Single Pass

Early reconciliation systems often assume a binary outcome.

Matched
Not Matched

In practice, financial workflows need richer states.

Matched
Force Matched
Pending
Refund
Provider Charge
Mismatch

This distinction becomes critical as transaction volume grows.

Some transactions can be matched immediately.

Others require additional context.

Others require human review.

And some become reconciled only after more data arrives.

This changes reconciliation from a one-time activity into an iterative workflow.

Why Pending Matters

One lesson that surprised me was how valuable the Pending state becomes.

Many transactions initially appear unmatched because:

Settlement reports arrive later
Exchange rates change
References are missing
External systems are delayed
Additional provider data is not yet available

Treating these transactions as failures creates operational noise.

Instead, a better approach is:

Ingest
   ↓
Match
   ↓
Pending
   ↓
Enrichment
   ↓
Reconciliation Again

Over time, many pending transactions become valid matches without manual intervention.

The ability to revisit and re-evaluate decisions turned out to be more important than improving the matching algorithm itself.

Missing References Are Normal

Most matching examples assume a common identifier exists everywhere.

Reality is messier.

ERP:

INV-123

Payment Provider:

PAY-456

Bank:

UTR-789

There is no shared key.

This forces reconciliation systems to combine multiple signals:

Amount
Date
Customer
Invoice
Settlement Batch
Payment Reference

A single field is rarely enough.

Successful reconciliation often depends on combining weak signals into a strong conclusion.

Exact Matching Is Not Enough

Another common mistake is relying entirely on exact matches.

Financial systems frequently require:

Tolerance-based matching
Many-to-one matching
Partial settlement handling
Refund reconciliation
Fee reconciliation

A robust reconciliation platform usually employs multiple matching strategies rather than a single matching rule.

The challenge is not finding one algorithm.

The challenge is deciding which algorithm should be trusted under which circumstances.

Matching Is Only Half the Problem

Most engineering effort initially goes into matching.

Eventually, a different realization emerges.

Matching is not the product.

Exception management is.

The real questions become:

Why did a match fail?
Who should investigate it?
What evidence exists?
How should it be resolved?
What audit trail should be preserved?

Without clear workflows around exceptions, even accurate matching engines become difficult to operate.

Finance teams care less about algorithm sophistication and more about explainability.

Reconciliation Ends in Accounting

Another lesson is that reconciliation does not end when records are matched.

Eventually the outcome needs to flow into downstream accounting systems.

Payment Data
      ↓
Reconciliation
      ↓
Review
      ↓
Approval
      ↓
Accounting Export

A technically correct match has little value if the result cannot be consumed by the rest of the financial workflow.

The export path is part of the architecture.

Not an afterthought.

Lessons Learned

Reconciliation Is a Workflow Problem

Matching is only one stage of the process.

Operational workflows matter just as much.

Payment Success and Settlement Success Are Different Events

Collection and settlement should be treated independently.

Pending Is a Valid State

Not every transaction can be resolved immediately.

Systems should be designed to accommodate uncertainty.

Explainability Builds Trust

Finance teams trust systems they can understand.

Opaque matching logic creates operational friction.

Data Quality Is Usually the Real Problem

Matching engines often get blamed for issues originating in source systems.

Exact Matching Scales Poorly

As transaction complexity grows, multiple matching strategies become necessary.

Final Thoughts

The hardest part of reconciliation is not matching records.

It is building a system that can explain why records did or did not match, accommodate incomplete information, support human review, and ultimately produce accounting-ready outcomes.

Once transaction volume grows, reconciliation stops being a data problem.

It becomes an operational platform.

And designing that platform is far more interesting than designing a matching engine.

Designing an Enterprise SFTP Integration Platform

Salil Chincholikar — Tue, 23 Jun 2026 20:16:25 +0000

Most engineering teams don't think about file transfer until an enterprise customer asks for it. Then suddenly it becomes a critical requirement.

A few years ago, I assumed APIs would be enough for most integrations. In reality, many enterprises still rely on secure file transfer for exchanging invoices, remittance advice, payment reports, customer data, and financial documents. For organizations operating in regulated environments, SFTP remains one of the most widely accepted integration mechanisms.

The challenge is that enterprise-grade file exchange introduces a completely different set of architectural concerns. It is not simply about moving files from one location to another. You need to think about security, auditability, reliability, scalability, tenant isolation, operational overhead, and cost.

This article walks through the design principles behind building an enterprise SFTP integration platform on AWS. The focus is on architecture and decision making rather than implementation details.

The Problem

Imagine a fintech platform serving multiple enterprise customers. Every customer has its own ERP, accounting systems, banking relationships, and operational processes.

Some customers are modern and expose APIs. Others still operate through scheduled file exchanges.

Invoices arrive as CSV files.
Payment reports arrive as Excel sheets.
Bank statements arrive as text files.
Master data is exchanged through nightly uploads.

The platform needs to support all of these workflows while maintaining:

Security
Reliability
Auditability
Scalability
Tenant isolation

The solution cannot become an operational burden for the engineering team.

Requirements

Before discussing architecture, it helps to define the requirements.

Functional Requirements

Secure file upload and download
Support for multiple enterprise customers
Automated ingestion pipelines
Validation and processing workflows
Integration with downstream business systems

Non-Functional Requirements

Strong security controls
Complete audit trail
High availability
Cost efficiency
Minimal operational overhead
Ability to onboard new customers quickly

These non-functional requirements ultimately drove most architectural decisions.

Architectural Principles

Several principles guided the design.

1. Managed Services First

Infrastructure should be maintained only when it creates business value.

Running and maintaining SFTP servers sounds simple until key rotation, patching, monitoring, scaling, backups, and disaster recovery become operational responsibilities.

Where possible, managed services reduce complexity.

2. Event-Driven Processing

File transfer and file processing should be separate concerns.

The ingestion platform should focus on securely receiving files.

Processing should happen asynchronously.

This allows the platform to scale independently as data volumes increase.

3. Tenant Isolation

Enterprise customers expect clear separation of data.

Isolation must exist not only at the application layer but throughout the entire ingestion pipeline.

Every architectural decision should preserve that boundary.

4. Auditability by Design

Financial workflows require traceability.

For every file received, the platform should answer:

Who uploaded it?
When was it uploaded?
What happened after upload?
Was processing successful?
If not, why?

Auditability should be designed into the platform rather than added later.

High-Level Architecture

At a high level, the architecture follows a straightforward pattern.

Enterprise Customer
        |
        v
  SFTP Solution
        |
        v
 Secure Storage
        |
        v
 Event Trigger
        |
        v
 Validation Layer
        |
        v
 Processing Pipeline
        |
        v
 Business Systems

The key idea is separation of concerns.

File transfer is one responsibility.

Validation is another.

Business processing is another.

This makes the platform easier to scale and evolve.

Security Considerations

Security requirements often dominate enterprise integration discussions.

Several areas deserve special attention.

Authentication

Every customer should have independently managed credentials and access controls.

Compromising one customer should not impact another.

Encryption

Data should remain encrypted both in transit and at rest.

This is generally considered table stakes for financial systems.

Access Control

Access should follow the principle of least privilege.

Only the components that require access should receive it.

Audit Logging

Every interaction should be recorded.

Audit logs frequently become the most valuable operational tool during investigations and support requests.

Processing Strategy

One mistake I have seen repeatedly is coupling file arrival directly to business processing.

Initially this feels simple.

As volume grows, it becomes fragile.

A better approach is:

Receive file
Store file
Generate processing event
Validate contents
Execute business workflow
Persist results
Notify stakeholders if required

This pattern creates natural resilience and simplifies recovery from failures.

If processing fails, the original file remains available for replay.

Cost Considerations

Cost optimization is often overlooked during initial architecture design.

The first solution that works is rarely the most economical solution.

Several principles help control costs:

Avoid permanently running infrastructure where possible
Process only when work exists
Separate storage from compute
Scale processing independently from ingestion

The most expensive architecture is usually not the one consuming the most cloud resources.

It is the one requiring constant engineering attention.

Operational simplicity has economic value.

Lessons Learned

Looking back, several lessons stand out.

Reliability Matters More Than Features

Enterprise customers care more about consistency than sophistication.

A platform that works every day is more valuable than one with dozens of advanced capabilities.

Auditability Pays Off

The teams that invest in traceability early save enormous amounts of time later.

Operational questions become much easier to answer.

Automation Reduces Support Burden

Every manual operational task eventually becomes a bottleneck.

Automate onboarding.
Automate validation.
Automate monitoring.
Automate recovery where possible.

Design for Growth Early

Many integration platforms begin with a handful of customers.

Success creates a different problem.

Architecture decisions that seem insignificant at ten customers can become painful at one hundred.

Final Thoughts

Enterprise integrations rarely receive the same attention as customer-facing features.

Yet they often sit at the center of critical business operations.

Designing an SFTP integration platform is ultimately not about file transfer.

It is about building a secure, reliable, auditable, and scalable system that organizations can trust with important business processes.

The technologies will evolve.

The architectural principles tend to remain surprisingly consistent.