Most banking backend tutorials focus on transactions your own system creates. A user taps a button, your service writes a record, and you control every part of that process from start to finish.
But in a real fintech product, a large portion of the transaction data your system deals with does not originate inside your system at all. It comes from the customer's actual bank, through an Open Banking style API. That data has already passed through at least one external step before it ever reaches you, so it cannot be treated with the same confidence as a transfer your own service generated.
This is where a lot of backends quietly get into trouble. They treat everything that lands in the database the same way, whether it came from their own logic or from an outside provider. NestJS gives you the structure to draw a clear line between the two, and to treat provider data with the caution it actually deserves.
Why external data is not the same as internal data
When your own service creates a transaction, you control every step of that process, so you can trust it completely by the time it reaches your database.
When a transaction comes from an outside provider, you are only seeing a copy of what the bank recorded, not the original event itself. That copy needs to be checked and matched against your own ledger before you treat it as fact. It can arrive late, arrive twice, or arrive with slightly different metadata than a previous version of the same event.
A useful mental model here is to treat anything coming from outside your system as unverified input, not as a trusted fact, until it has been checked against what you already know.
The duplicate entry problem
If you are polling an external provider on a schedule to pull in new transactions, duplicates are almost guaranteed to happen at some point. A network retry, an overlapping polling window, or a provider sending the same batch twice can all result in the same transaction reaching your system more than once.
The fix is to key off the bank's own transaction id, not one your own system generated, and to check for an existing record before inserting anything new.
@Injectable()
export class BankIngestionService {
constructor(
@InjectRepository(ExternalTransaction)
private readonly externalTransactionRepo: Repository<ExternalTransaction>,
) {}
async ingestTransaction(payload: ProviderTransactionDto): Promise<void> {
const existing = await this.externalTransactionRepo.findOne({
where: { providerTransactionId: payload.providerTransactionId },
});
if (existing) {
// Already recorded, safe to ignore or update metadata only
return;
}
const record = this.externalTransactionRepo.create({
providerTransactionId: payload.providerTransactionId,
amount: payload.amount,
currency: payload.currency,
rawPayload: payload,
reconciled: false,
});
await this.externalTransactionRepo.save(record);
}
}
Notice that the provider's transaction id is the thing being checked, not anything generated locally. Your own id generation has no way of knowing whether an event has already been seen by the outside world.
When the call itself fails partway through
Pulling data from an external provider is not a clean, all or nothing operation the way a database transaction inside your own system can be. A token can expire mid sync. A batch of transactions can partially load before a request times out. Your system needs to know exactly where it stopped, so the next attempt does not miss transactions or record the same one twice.
A simple way to handle this in NestJS is to track a sync checkpoint per provider connection, and only advance it once a batch has been fully and successfully processed.
@Injectable()
export class BankSyncService {
constructor(
private readonly bankIngestionService: BankIngestionService,
@InjectRepository(SyncCheckpoint)
private readonly checkpointRepo: Repository<SyncCheckpoint>,
) {}
async syncAccount(accountId: string): Promise<void> {
const checkpoint = await this.checkpointRepo.findOne({
where: { accountId },
});
const fromCursor = checkpoint?.lastCursor ?? null;
const batch = await this.fetchTransactionsFromProvider(accountId, fromCursor);
for (const transaction of batch.transactions) {
await this.bankIngestionService.ingestTransaction(transaction);
}
await this.checkpointRepo.save({
accountId,
lastCursor: batch.nextCursor,
lastSyncedAt: new Date(),
});
}
private async fetchTransactionsFromProvider(accountId: string, cursor: string | null) {
// Call to the actual Open Banking provider goes here
// Returns a batch of transactions plus the next cursor to resume from
return { transactions: [], nextCursor: cursor };
}
}
If the process fails halfway through the loop, the checkpoint has not moved yet, so the next run picks up from the same place instead of skipping ahead or starting completely over.
Applying the same discipline at a new boundary
If you have worked with NestJS on the request side, you already know the pattern. Guards protect requests coming into your system, and interceptors let you log and shape what happens as a request moves through your application.
The same mindset needs to wrap around the outbound call to the external provider too, not just your own controllers. An interceptor style wrapper around your provider calls lets you log every attempt, what came back, and whether it was successfully reconciled, without scattering that logic across every place you happen to call the provider.
@Injectable()
export class ProviderCallLogger {
private readonly logger = new Logger(ProviderCallLogger.name);
async wrap<T>(callName: string, fn: () => Promise<T>): Promise<T> {
const startedAt = Date.now();
try {
const result = await fn();
this.logger.log(`${callName} succeeded in ${Date.now() - startedAt}ms`);
return result;
} catch (error) {
this.logger.error(`${callName} failed after ${Date.now() - startedAt}ms`, error.stack);
throw error;
}
}
}
Wrapping every external call this way means that when something goes wrong three weeks from now, you have a clear record of exactly what was called, when, and what came back, instead of trying to guess after the fact.
The bigger picture
NestJS does not automatically make external data trustworthy. No framework can do that on its own. What it gives you is a clear place to put the discipline that external data needs, checkpoints as their own tracked entities, ingestion logic separated from your core business logic, and consistent logging around every outbound call.
The pattern is simple once it clicks. Anything that enters your system from outside stays unreconciled until it has been checked against your own records. NestJS's structure, services, repositories, and interceptors, gives you a natural place to enforce that instead of leaving it to scattered checks across your codebase.
If you are building a product that pulls in financial data from outside providers and want this handled properly from the start, this is exactly the kind of problem worth getting right early, since fixing it after the fact usually means untangling data that has already been trusted for too long.
I am Peace Melodi, a backend software engineer. If you want your business to scale big, comfortably handling millions of users without breaking, with strong scalability and security in place, feel free to reach out.
LinkedIn: https://www.linkedin.com/in/melodi-peace-406494368
GitHub: https://github.com/PeaceMelodi
Top comments (4)
Nice breakdown — the separation between "data my system created" and "data that arrived from outside" is the exact mental model that catches silent bugs before they compound. A few things worth adding from building bank-data ingestion in production:
Transaction mutation, not just duplication. Your dedup on
providerTransactionIdcatches the same event arriving twice, but real bank transactions change. A pending debit posts with a different amount. A merchant name gets corrected three days later. A split transaction merges back. If your idempotency check is pure insert-if-not-exists, you silently keep the stale version. Thereconciled: falseflag you have is the right instinct — but it needs to become an UPSERT keyed on the provider ID, with a field likeproviderVersionorlastSeenAtso you can detect and log when an already-stored transaction was mutated on a later sync.Cursor pagination is not universal across bank APIs. Your checkpoint pattern is clean, but the cursor abstraction has to absorb wildly different pagination models depending on the provider. Plaid gives you a true cursor. Most PSD2 AIS APIs (Tink, TrueLayer, GoCardless/Nordigen) paginate by
bookingDatewindow with next-page links, not a resumable cursor. If your checkpoint stores a genericlastCursor, you end up baking in one provider's model and breaking the abstraction when you add a second bank source. Worth making the checkpoint provider-aware from day one.The harder boundary is access itself. The ingestion code is the fun part. The part that quietly blocks most small teams is that PSD2 requires an eIDAS QWAC certificate — €3k–15k/year depending on your bank and country — before a single bank API call can be made. That's why most teams route through an aggregator rather than connecting directly. It's a gap I focus on with open-banking.io (cert-free bank data access for small players and developers), but regardless of the route you take, it's the structural thing worth knowing about before you architect the ingestion pipeline.
The interceptor-wrapping pattern at the end is underrated advice. Three months in, when a reconciliation breaks and nobody remembers which provider call returned what, that call log is the only thing that saves you.
John, this is incredibly useful, thank you for taking the time to write it out this fully.
The mutation point is the one that stands out most. I built the dedup logic around the assumption that a provider id shows up once and stays fixed, but a pending debit changing amount or a merchant name getting corrected days later breaks that assumption completely. Moving to an upsert with a providerVersion or lastSeenAt field makes a lot more sense than what I had, and it would have caught exactly the kind of silent staleness you are describing.
Good catch on the cursor assumption too. I wrote that part with a single provider in mind, and you are right that baking in one pagination model works fine until a second bank source with a completely different approach shows up. Making the checkpoint provider aware from the start avoids having to redo that abstraction later.
The certification point is genuinely new information for me, I had not factored in the cost and process behind actually getting access before any of this ingestion logic even runs. Good to know that is usually why teams route through an aggregator rather than going direct.
Really appreciate the depth here, this is the kind of comment that makes the article better after the fact.
Thanks Peace, really glad it landed well.
One addition on the
lastSeenAtpattern: the mental shift that helps is treating each synced record as "a snapshot the bank gave us at time T" rather than "the truth." When a merchant name gets corrected or a pending debit settles at a different amount, you can trace exactly when the bank's view changed — which matters a lot when reconciling against payment confirmations or accounting entries.On the provider-aware checkpoint: the simplest version is a
(provider, cursor)composite instead of a single globallast_cursor. I've seen systems use one column and then wonder why Bank B's transactions stop arriving after Bank A's cursor advances past them.And yes — the certification piece is actually why I maintain open-banking.io. The eIDAS QWAC requirement (typically €500–3000/yr, weeks of paperwork) is a genuine barrier that catches people off guard, because you need it before you can call a single bank API. Aggregators absorb that cost and complexity, which is the real value proposition once you see the bill.
John, the snapshot framing is a great way to put it, a record as a snapshot at time T rather than the truth changes how you think about the whole ingestion layer, not just the lastSeenAt field itself. It also makes tracing back to a specific point in time during reconciliation much more natural, instead of just knowing something changed without knowing exactly when.
The composite key point is a good catch too, I can see exactly how a single global cursor quietly breaks the moment a second provider is added, since one bank advancing past the other's position would silently stop pulling their data with no obvious error to point at.
Appreciate you sharing the real numbers on the certification cost as well, that context makes it much clearer why most teams end up going through an aggregator rather than trying to absorb that themselves.