DEV Community: Borino88

From Webhook to Audit Trail: Designing Reliable Integration Workflows

Borino88 — Tue, 28 Jul 2026 17:26:45 +0000

From Webhook to Audit Trail: Designing Reliable Integration Workflows

Webhooks are the connective tissue of modern SaaS ecosystems. However, consuming webhooks reliably from third-party systems like Stripe, Shopify, or GitHub is challenging. Downstream APIs fail, networks timeout, and delivery systems send duplicate payloads.

Here is how to design a resilient, fault-tolerant webhook integration hub in Java and Spring Boot.

The Resilient Integration Pipeline

A production-grade webhook consumer must isolate failures and guarantee eventual consistency:

[ Webhook Ingestion Controller ]
              │
              ▼
   (Idempotency Key Check)
              │
              ▼
    [ Redis/DB Event Log ] ➔ (Queue / RabbitMQ) ➔ [ sub-system consumers ]
                                                       │
                                                       ▼
                                            (Exponential Backoff Retry)
                                                       │
                                                       ▼
                                            [ Dead-Letter Queue (DLQ) ]

1. Idempotency Safeguards

Webhooks are generally delivered "at-least-once". To prevent executing side-effects multiple times, track event IDs using database constraints or Redis caches:

@Component
public class WebhookIdempotencyService {
    @Autowired
    private StringRedisTemplate redisTemplate;

    public boolean isDuplicate(String eventId) {
        // Set key with a 24-hour expiration window
        Boolean isNew = redisTemplate.opsForValue().setIfAbsent(
            "webhook:idempotency:" + eventId, 
            "processed", 
            Duration.ofHours(24)
        );
        return isNew == null || !isNew;
    }
}

2. Exponential Backoff with Jitter

When downstream systems fail, retrying requests immediately can cause a thundering herd DDoS. Instead, use exponential backoff with random jitter:

$$\text{Delay} = \text{Base Delay} \times (1.5^{\text{attempt}}) \pm \text{Jitter}$$

public class BackoffCalculator {
    public static long calculateDelay(int attempt) {
        long baseDelay = 1000; // 1 second
        double factor = 1.5;
        double jitter = Math.random() * 200; // 0-200ms random offset
        return (long) (baseDelay * Math.pow(factor, attempt) + jitter);
    }
}

3. The Dead-Letter Queue (DLQ)

If all retry attempts are exhausted, move the payload to a Dead-Letter Queue (DLQ). A DLQ preserves failing messages for engineer audit, database analysis, and manual replay after bug fixes.

Building a Secure API Starter Without Hiding the Hard Parts

Borino88 — Tue, 28 Jul 2026 17:26:09 +0000

Building a Secure API Starter Without Hiding the Hard Parts

Exposing REST APIs to the public internet requires defense-in-depth security. Too many tutorial implementations cut corners by skipping rate limiting, using simple database lookups for every request, or storing JSON Web Tokens (JWTs) in browser LocalStorage.

Here is how to design a secure FastAPI starter that addresses authentication, rate limiting, and secure storage from day one.

1. Defense-in-Depth Architecture

To build a secure API, you need multiple layers of protection:

[ Client ] 
   │
   ▼
[ Rate Limiting Layer (Token Bucket / Redis) ]
   │
   ▼
[ CORS Policy Guards ]
   │
   ▼
[ Stateless JWT / OAuth2 Security Middleware ]
   │
   ▼
[ Controller / Role-Based Access Control (RBAC) ]

2. Stateless Auth & JWT Revocation

While stateless JWT authentication is clean, token revocation (for logouts or password resets) is often neglected. We solve this by adding a Redis-backed token blacklist. We store the token's unique identifier (jti) in Redis with a Time-To-Live (TTL) matching the token's remaining lifespan.

Here is the FastAPI implementation:

from fastapi import FastAPI, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from jose import jwt, JWTError
from datetime import datetime, timedelta

SECRET_KEY = "your-high-entropy-source-key"
ALGORITHM = "HS256"
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")

# Mock Redis blacklist for demonstration
token_blacklist = set()

def revoke_token(jti: str):
    token_blacklist.add(jti)

def is_token_revoked(jti: str) -> bool:
    return jti in token_blacklist

async def get_current_user(token: str = Depends(oauth2_scheme)):
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
        username: str = payload.get("sub")
        jti: str = payload.get("jti")
        if username is None or jti is None:
            raise credentials_exception
        if is_token_revoked(jti):
            raise HTTPException(status_code=401, detail="Token has been revoked")
    except JWTError:
        raise credentials_exception

    return {"username": username, "role": payload.get("role", "user")}

3. Threat Modeling & Secure Storage

Token Storage Recommendations

Never store tokens in localStorage: They are vulnerable to Cross-Site Scripting (XSS) attacks.
Use HttpOnly, Secure, SameSite=Strict Cookies: This ensures the browser only sends tokens during first-party requests and hides them from client-side scripts.
Mobile Client Storage: Exclusively utilize hardware-backed security modules (Keychain for iOS, Keystore for Android).

Double-Entry Ledger Design for Software Engineers

Borino88 — Tue, 28 Jul 2026 17:25:33 +0000

Double-Entry Ledger Design for Software Engineers

When building financial systems, billing engines, or wallet platforms, the standard "single column balance" approach (e.g., updating a balance column in a users table) is a recipe for disaster. It fails audits, makes reconciliation impossible, and cannot survive concurrency failures.

To build a system that is secure, auditable, and resilient to failures, you must implement a Double-Entry Ledger.

The Core Principles of Double-Entry

In double-entry bookkeeping, money never appears or disappears; it is only transferred from one account to another. Every transaction consists of a set of entries where:

$$\text{Total Debits} = \text{Total Credits}$$

An account type determines whether a debit increases or decreases its value:

Assets / Expenses: Increased by debits, decreased by credits.
Liabilities / Equity / Revenue: Increased by credits, decreased by debits.

Relational Database Schema Design

A robust ledger database requires three primary tables:

Accounts: The containers holding balances of specific types.
Transactions: The events representing a business operation.
Entries: The individual credit or debit allocations mapped to accounts.

Here is the PostgreSQL schema:

CREATE TABLE accounts (
    id UUID PRIMARY KEY,
    name VARCHAR(255) NOT NULL,
    type VARCHAR(50) NOT NULL CHECK (type IN ('asset', 'liability', 'equity', 'revenue', 'expense')),
    currency VARCHAR(3) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE transactions (
    id UUID PRIMARY KEY,
    description TEXT NOT NULL,
    metadata JSONB,
    posted_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE entries (
    id UUID PRIMARY KEY,
    transaction_id UUID REFERENCES transactions(id) ON DELETE CASCADE,
    account_id UUID REFERENCES accounts(id),
    amount NUMERIC(18, 4) NOT NULL, -- Positive for debits/credits depending on context
    direction VARCHAR(2) NOT NULL CHECK (direction IN ('DB', 'CR')),
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_entries_account ON entries(account_id);
CREATE INDEX idx_entries_transaction ON entries(transaction_id);

Implementing Verification Logic in Python

To guarantee the integrity of your ledger, your application must verify that all transactions balance before persisting them. Here is a Python implementation utilizing SQL transactions and sanity checks:

import uuid
from decimal import Decimal
from typing import List, Dict

class LedgerError(Exception):
    pass

class LedgerEntry:
    def __init__(self, account_id: str, amount: Decimal, direction: str):
        self.account_id = account_id
        self.amount = amount
        self.direction = direction

def verify_and_save_transaction(db_connection, description: str, entries: List[LedgerEntry]):
    # 1. Calculate and verify zero-sum balance
    total_debits = sum(e.amount for e in entries if e.direction == 'DB')
    total_credits = sum(e.amount for e in entries if e.direction == 'CR')

    if total_debits != total_credits:
        raise LedgerError(f"Transaction does not balance. Debits: {total_debits}, Credits: {total_credits}")

    tx_id = str(uuid.uuid4())

    with db_connection.cursor() as cursor:
        # 2. Insert transaction
        cursor.execute(
            "INSERT INTO transactions (id, description) VALUES (%s, %s)",
            (tx_id, description)
        )

        # 3. Insert matching entries
        for entry in entries:
            cursor.execute(
                """
                INSERT INTO entries (id, transaction_id, account_id, amount, direction)
                VALUES (%s, %s, %s, %s, %s)
                """,
                (str(uuid.uuid4()), tx_id, entry.account_id, entry.amount, entry.direction)
            )

    print(f"Successfully posted transaction {tx_id} to database.")

Key Takeaways

Immutability: Never use SQL UPDATE on ledger entries. If a mistake is made, post a reversing transaction.
Strong Typing: Use arbitrary-precision numeric types (NUMERIC or DECIMAL) for monetary values—never float.
Zero-Sum Guarantees: Balance checks should be enforced both at the application tier and via database triggers or constraints.

Designing an Idempotent Transaction API: Preventing Duplicate Financial Operations

Borino88 — Sun, 26 Jul 2026 10:54:56 +0000

In financial backend systems, network retries represent a classic race-condition vector. If a client attempts to post a transaction, suffers a TCP timeout, and blindly retries, the backend risks posting the transaction twice.

To prevent duplicate financial operations, we must design an Idempotent API using Idempotency Keys.

The Architecture of Idempotency

sequenceDiagram
    participant Client
    participant API Gateway
    participant Redis Cache
    participant PostgreSQL

    Client->>API Gateway: POST /journal (Idempotency-Key: X)
    API Gateway->>Redis Cache: SetNX(Key: X, Status: IN_PROGRESS)
    alt Key exists
        Redis Cache-->>Client: 409 Conflict (Request In Progress) or Cached Response
    else Key is new
        API Gateway->>PostgreSQL: Execute Double-Entry Transaction
        PostgreSQL-->>API Gateway: Transaction Committed successfully
        API Gateway->>Redis Cache: Set(Key: X, Status: DONE, Response: Y)
        API Gateway-->>Client: 201 Created (Transaction Details)
    end

Core Database Schema & Unique Constraints

We enforce idempotency at the database level by binding the idempotency key directly to our journal entries table:

CREATE TABLE journal_entries (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    idempotency_key VARCHAR(255) UNIQUE NOT NULL,
    amount DECIMAL(18, 4) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

Python Implementation (FastAPI & Redis)

Here is a tested pattern for validating and caching requests in Redis:

from fastapi import FastAPI, Header, HTTPException, status
import redis

app = FastAPI()
r = redis.Redis(host='localhost', port=6379, decode_responses=True)

@app.post("/api/v1/journal", status_code=201)
def post_transaction(payload: dict, idempotency_key: str = Header(...)):
    # 1. Atomic Check-and-Set in Redis
    lock_acquired = r.set(f"idemp:{idempotency_key}", "IN_PROGRESS", nx=True, ex=3600)

    if not lock_acquired:
        status_val = r.get(f"idemp:{idempotency_key}")
        if status_val == "IN_PROGRESS":
            raise HTTPException(
                status_code=status.HTTP_409_CONFLICT,
                detail="Transaction in progress. Please wait."
            )
        # Return cached response if done
        return {"status": "success", "cached": True, "data": status_val}

    try:
        # 2. Process transaction inside a database transaction block
        # (DB commit logic goes here)
        result_data = {"transaction_id": "tx_abc123", "processed": True}

        # 3. Update status to completed and store response
        r.set(f"idemp:{idempotency_key}", str(result_data), ex=86400)
        return {"status": "success", "cached": False, "data": result_data}
    except Exception as e:
        r.delete(f"idemp:{idempotency_key}")
        raise HTTPException(
            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
            detail="Transaction failed."
        )

Testing for Concurrency

To ensure that concurrent requests with the exact same idempotency key are rejected, we can run multithreaded HTTP request tests using pytest and httpx.

You can view the full repository and test suites at secure-fintech-ledger.

Disclosure: This article was prepared with AI-assisted editing and research support. I reviewed the technical content, tested the code and take responsibility for the final article. #ABotWroteThis

A Practical Multi-Stage Docker Build for Python APIs

Borino88 — Sun, 26 Jul 2026 10:54:51 +0000

When containerizing Python web applications for production, developers often produce heavy, insecure images containing compilers, headers, and root execution privileges.

This post demonstrates how to build a hardened, multi-stage Docker container utilizing a non-root runtime environment, health check verification, and OCI image metadata.

The Production Dockerfile

This multi-stage Dockerfile uses python:3.11-slim-bookworm to separate building dependencies from runtime execution:

# ==============================================================================
# Stage 1: Build & Dependency Packaging
# ==============================================================================
FROM python:3.11-slim-bookworm AS builder

WORKDIR /build

RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

# ==============================================================================
# Stage 2: Production Runtime
# ==============================================================================
FROM python:3.11-slim-bookworm AS runtime

# OCI Standard Metadata Labels
LABEL org.opencontainers.image.title="Secure API Platform"
LABEL org.opencontainers.image.source="https://github.com/Borino88/secure-api-platform"
LABEL org.opencontainers.image.version="1.0.0"
LABEL org.opencontainers.image.licenses="MIT"
LABEL org.opencontainers.image.authors="Mahdi Fattahi <a.borino88@gmail.com>"

ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PATH="/usr/local/lib/python3.11/site-packages:/app:$PATH" \
    PYTHONPATH="/app"

RUN groupadd -g 1000 appgroup && \
    useradd -u 1000 -g appgroup -s /bin/bash -m appuser

WORKDIR /app

COPY --from=builder /install /usr/local
COPY requirements.txt ./
COPY src/ ./src/

RUN chown -R appuser:appgroup /app

USER appuser

HEALTHCHECK --interval=30s --timeout=5s --start-period=5s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health').getcode() == 200" || exit 1

EXPOSE 8000

CMD ["uvicorn", "src.api.main:app", "--host", "0.0.0.0", "--port", "8000"]

Security Best Practices Highlighted

Multi-Stage Separation: All compiler tools and cached wheels remain in Stage 1, leaving Stage 2 clean.
Non-Root Execution: We define appuser and appgroup to prevent root execution attacks.
Health Checks: A native, dependency-free Python health check endpoint verification command is integrated directly.
Vulnerability Mitigation: Hardened bases limit base image CVE counts, validated using Trivy scans.

The configuration templates are available in the public secure-api-platform repository.

Disclosure: This article was prepared with AI-assisted editing and research support. I reviewed the technical content, tested the code and take responsibility for the final article. #ABotWroteThis

Hello DEV — I’m Mahdi, a Full-Stack Engineer Building Reliable Systems and Open Tools

Borino88 — Sun, 26 Jul 2026 10:54:17 +0000

Hello DEV Community! I'm Mahdi, a senior full-stack engineer and international STEM educator. Over the past eight years, I've designed and shipped software systems spanning financial ledgers, microservice messaging hubs, real-time telemetry dashboards, and geospatial environmental pipelines.

I joined DEV to share hard-earned architectural patterns, dockerized deployment templates, and tested source code that solves real software engineering problems.

What I’m Building

Currently, I'm focusing on two major tracks:

System Reliability & Security: Developing API platforms with integrated rate-limiting, double-entry financial ledgers, and secure integration webhooks.
Open Educational Simulators: Packaging complex math, physics, biology, and computer science concepts into reproducible, interactive open-source laboratories.

What You Can Expect From My Posts

I believe technical articles are only as good as the code that backs them. Therefore, every tutorial or architecture breakdown I write will:

Include fully tested, runnable code.
Address trade-offs, performance limitations, and security considerations.
Link back to public, containerized GitHub repositories so you can spin up the stack instantly.

I look forward to engaging with the community, learning from your experiences, and building in public.

You can view my source code on GitHub.
Check out my interactive portfolio at fattahi.xyz.

Disclosure: This article was prepared with AI-assisted editing and research support. I reviewed the technical content, tested the code and take responsibility for the final article. #ABotWroteThis