DEV Community

Ruhid Ibadli
Ruhid Ibadli

Posted on

No React, no Celery, no Redis: shipping a real platform on a boring stack

I run a small brokerage in Azerbaijan that imports salvage and auction cars from the US — we bid on IAAI and Copart lots for customers, then truck the won cars through the port of Poti in Georgia into Azerbaijan. AutoMakler is the platform behind that: a real, deployed, production site that does live auction scraping, delivery-cost estimation, Carfax lookups, and payments.

Here is the part that gets a reaction from other developers: there is no React. No Vue. No build step. No node_modules. No Celery, no Redis, no WebSocket server. It is FastAPI, SQLAlchemy, Jinja2, Bootstrap, a single Postgres box, and Playwright for scraping. Server-rendered HTML with a sprinkle of vanilla JS.

I want to make the case that in 2026 this is not a compromise. For a product this size it was the correct call, and the most interesting engineering happened because of the constraints, not in spite of them.

The stack, stated plainly

  • Backend: FastAPI 0.111, Python 3.11, SQLAlchemy 2.0 (sync), Jinja2.
  • Frontend: Bootstrap 5.3, light/dark toggle, minimal vanilla JS. Zero frontend framework. Zero bundler.
  • DB: PostgreSQL 15. No Alembic — schema via create_all() plus a few idempotent startup ALTER TABLEs.
  • Scraping: Playwright headless Chromium with a stealth context.
  • Ship: Docker Compose, GitHub Actions CI/CD. 509 pytest tests (in-memory SQLite, so CI needs no Postgres or Chromium), including a dedicated OWASP Top 10 file. Deploy is gated on the suite passing.
  • It's also an installable PWA via a manifest and service worker.

The whole thing is one Python process serving HTML. When a page needs to change, the server renders the new HTML. That's it. No client-side router, no hydration, no state that lives in two places and drifts apart.

The catch: a couple of features genuinely need to feel live — a scrape takes several seconds, and a support chat has to update without a refresh. The lazy answer is "add Celery for the jobs and WebSockets for the chat." I didn't, and the rest of this post is why.

The calculator that polls itself

The delivery cost calculator is the busiest feature. You paste one thing into one text box — a lot number, a 17-character VIN, or a full Copart/IAAI URL — and it works out what it costs to land that car in Azerbaijan.

First job: figure out what you pasted. One input, three shapes, detected server-side (and mirrored in JS to show or hide the auction dropdown):

def parse_search_input(raw: str) -> SearchInput:
    s = raw.strip()
    if re.fullmatch(r"[A-HJ-NPR-Z0-9]{17}", s.upper()):     # VIN: 17 chars, no I/O/Q
        return SearchInput("vin", vin=s.upper())
    if "copart.com/lot/" in s or "iaai.com/VehicleDetail/" in s:
        return SearchInput("url", raw=s)
    if s.isdigit():                                         # lot number
        return SearchInput("lot", lot_number=s)
    return SearchInput("unknown", raw=s)
Enter fullscreen mode Exit fullscreen mode

Now the actual work — scraping an auction lot with a headless browser — takes seconds. Blocking the request thread for that is a non-starter. The textbook move is to push a job onto Celery with Redis as the broker. For a single-VPS app, that felt like installing a second database and a task daemon to run one kind of job.

FastAPI already runs on an asyncio event loop. So the POST handler writes a pending row, launches the scrape as a bare background task, and redirects immediately:

@router.post("/calculate")
def calculate(query: str = Form(...), auction: str | None = Form(None),
              db: Session = Depends(get_db)):
    parsed = parse_search_input(query)
    search = SearchLog(search_query=query, search_type=parsed.input_type,
                       lot_number=parsed.lot_number, status="pending")
    db.add(search); db.commit()
    asyncio.create_task(run_search_task(search.id))   # no broker, no worker
    return RedirectResponse("/calculator", status_code=303)
Enter fullscreen mode Exit fullscreen mode

The page comes back instantly with the search sitting in "pending." The browser then polls a tiny status endpoint every 3 seconds and reloads once the row flips to completed or failed:

const poll = setInterval(async () => {
  const r = await fetch(`/api/search/${id}/status`);
  const { status } = await r.json();
  if (status !== "pending") { clearInterval(poll); location.reload(); }
}, 3000);
Enter fullscreen mode Exit fullscreen mode

The database row is the job queue. Its status column is the job state. There's no separate place for a job to get stuck, and after a deploy any in-flight search is just a pending row a human can see.

Keeping memory bounded without a worker pool

The one thing Celery would have given me for free is concurrency control — you don't want fifty simultaneous requests spawning fifty Chromium processes and OOM-ing the box. An asyncio.Semaphore is the same idea in one line — the worker-pool bound Celery would sell you, without the pool:

_BROWSER_SLOTS = asyncio.Semaphore(3)   # at most 3 live Chromium contexts

async def run_search_task(search_id: int):
    async with _BROWSER_SLOTS:
        car = await scrape_lot(lot_number, auction)   # or the VIN / URL path
    # ...fall back, sum three delivery legs, stamp status
Enter fullscreen mode Exit fullscreen mode

Requests past the third wait their turn on the semaphore instead of piling browsers onto the heap. Bounded memory, no worker pool to run.

The scrape itself is its own saga. The primary source is bid.cars; the fallback is scraping Copart and IAAI directly — which matters most for upcoming lots that aren't indexed yet, exactly the lots someone about to bid cares about. Copart is a single-page app whose raw HTML is full of unrendered {location} / {brand} template placeholders, so the direct scraper reads the rendered text instead and pulls year/make/model from the title and JSON-LD:

text = await page.evaluate("() => document.body.innerText")
Enter fullscreen mode Exit fullscreen mode

A shared stealth context (drops navigator.webdriver, adds a window.chrome shim, sets a realistic viewport/locale/timezone) is what gets it past Incapsula and Cloudflare. That scraper is the engine behind the Copart and IAAI bidding service — the whole point is to price a car before you commit to the auction. Once a lot resolves, three delivery legs get summed: car location → US state, state → Poti, Poti → Azerbaijan by truck.

One Epoint account, many apps

Azerbaijan's dominant card gateway is Epoint.az, and it ships with a constraint that shaped an entire subsystem:

Epoint lets you register exactly one callback URL (result_url) per merchant account. One. Full stop.

The gateway POSTs the result of every payment on the account to that single URL. Fine — until a second app of mine on the same VPS (a separate project called skillfade) needed to take card payments too. A second merchant account meant another KYC cycle and another set of credentials to babysit. So instead I made AutoMakler's own /payment/callback smart enough to serve both: a multi-tenant payment hub behind one merchant account.

The routing key is the order_id

I needed the routing information to ride inside data Epoint already round-trips back to me, untouched. The natural candidate is the order_id — I choose it, and Epoint echoes it back verbatim. So every hub-initiated payment gets an id shaped <projectSlug>_<uuid>:

order_id = f"{project.slug}_{uuid4().hex}"   # "skillfade_9f2c..." — the slug IS the route
Enter fullscreen mode Exit fullscreen mode

AutoMakler's own native payments already used a different shape — carfax-42-a1b2c3d4, a hyphenated type instead of an underscore-prefixed slug. The two id spaces are structurally disjoint and can never collide, and the whole router leans on that.

My first instinct was cleaner: Epoint's API has an other_attr field that looked purpose-built for pass-through metadata, so I tried to send {"project": "skillfade"} in it. Epoint answered with an HTTP 500 and this gem:

Array to string conversion
Enter fullscreen mode Exit fullscreen mode

The field wants a scalar; its PHP backend blows up on an object. That one error is why the project identity lives in the order_id prefix and nowhere else. Sometimes the API decides your architecture for you.

One callback, two tables

Native and external payments live in separate tablespayment_transactions for AutoMakler's own (a subscription, a Carfax report, a reserve-price lookup) and gateway_transactions for payments started on behalf of an external project. The shared callback verifies the signature once (constant-time), then does a two-table lookup on order_id: found in the gateway table means forward it; not found means it's one of mine, fulfilled by the code that was already there.

@router.post("/payment/callback")
async def payment_callback(data: str = Form(...), signature: str = Form(...),
                           db: Session = Depends(get_db)):
    if not hmac.compare_digest(sign(data), signature):     # constant-time compare
        raise HTTPException(status_code=400, detail="bad signature")
    payload = decode_callback(data)
    order_id = payload.get("order_id", "")

    gw_tx = db.query(GatewayTransaction).filter_by(order_id=order_id).first()
    if gw_tx:                                   # slug-prefixed id → external project
        return handle_gateway_callback(db, gw_tx, payload)   # forward, don't fulfill

    tx = db.query(PaymentTransaction).filter_by(order_id=order_id).first()
    return fulfill_native_payment(db, tx, payload)          # the original path, untouched
Enter fullscreen mode Exit fullscreen mode

What I like is how little it disturbs the existing system. The gateway is one if gw_tx: early-return bolted onto the top of a route that already moves money — additive, not invasive. And keeping the two ledgers disjoint was deliberate: I could have added a nullable project_slug column to the native table, but distinct id shapes in distinct tables mean an external order_id can never be mistaken for a native one, even under a bad migration or a copy-paste bug. The router's correctness becomes a property of the schema, not of a careful WHERE.

Forwarding the result, on the event loop

The hub fulfills nothing for external apps — it doesn't know what a skillfade payment buys. Its only job is to record the result and tell the originating project, reliably, with a signed webhook the project can verify. Rather than invent a scheme, I reused the same signature construction the client already uses to verify Epoint itself, so tenants verify the hub with code they already wrote:

def sign_webhook(secret: str, data_b64: str) -> str:
    raw = secret + data_b64 + secret                        # same shape Epoint signs with
    return base64.b64encode(hashlib.sha1(raw.encode()).digest()).decode()
Enter fullscreen mode Exit fullscreen mode

Delivery is best-effort with backoff, and — on brand for this platform — there's no Celery, no Redis, no broker. It's the same asyncio.create_task primitive as the calculator, on its own DB session off the request:

RETRY_DELAYS = [5, 30, 120, 300]   # seconds, after an immediate first attempt

async def deliver_with_retry(tx_id: int):
    session = SessionLocal()
    try:
        tx = session.get(GatewayTransaction, tx_id)
        if deliver_once(session, tx):                       # signed POST to the project
            return
        for delay in RETRY_DELAYS:
            await asyncio.sleep(delay)
            session.refresh(tx)
            if tx.delivery_status == "delivered":           # a duplicate callback beat us
                return
            if deliver_once(session, tx):
                return
    finally:
        session.close()
Enter fullscreen mode Exit fullscreen mode

If every retry fails — say the VPS reboots mid-backoff and the in-memory task vanishes — the payment isn't lost: the project pulls the authoritative outcome from GET /gateway/status?order_id=.... A durable broker would make that window bulletproof; for four retries over a few minutes against a webhook that's almost always up, standing up Redis and a worker fleet is a trade I'm happy to decline. The pull backstop is the honest safety net, and it's a GET. The whole module — two tables, one service, its own routes — is self-contained enough to lift onto a standalone pay.automakler.az later with no rewrite.

The same polling trick, again: chat without WebSockets

Support used to mean "add us on WhatsApp." I replaced it with in-app chat between admins and users (guests included). WebSockets are the reflex choice, but I already had a polling primitive that worked — the one behind the notification bell — so the chat polls too. No socket server, no Redis pub/sub.

The nice part is it still models WhatsApp-style sent / delivered / read receipts without a live connection. A message is "delivered" when the recipient's client passively fetches it on its next poll, and "read" when they open the thread. The sender's own bubbles advance from one check to two to a colored double-check on their next poll, no reload — the receipt state is just derived from whose poll has run.

That "mark undelivered messages as delivered" sweep runs on every poll, so it must never become a full-table scan. A Postgres partial index does the trick by indexing only the rows the sweep ever touches:

CREATE INDEX ix_chat_messages_user_undelivered
ON chat_messages (conversation_id)
WHERE sender_role = 'user' AND delivered_at IS NULL;
Enter fullscreen mode Exit fullscreen mode

The index's WHERE is the sweep's WHERE, so it stays tiny — and the moment a row is stamped delivered it drops out of the index. It effectively deletes its own entries.

And because chat is user-generated content headed for an innerHTML sink on the client, every message is escaped twice — Jinja autoescape on the server and textContent on the client — so a body can never become script. That double-escape is exactly the kind of invariant the dedicated OWASP test file exists to pin down.

The boring parts that make it a real product

A few things I'll mention only in passing, because they're solved problems that server-rendering makes easier, not harder:

  • Schema migrations without Alembic. create_all() plus idempotent startup ALTER TABLEs. The one sharp edge with multiple workers is a DDL race on the first deploy of a new table, so init runs inside a transaction-scoped Postgres advisory lock — one worker does the DDL, the rest wait: SELECT pg_advisory_xact_lock(:key) then create_all(conn).
  • Multilingual SEO on server-rendered pages. Seven languages, and the content hub exists in six of them — where each language version has its own keyword-bearing slug (an English import-car-from-usa-guide vs a Russian kak-privezti-avto-iz-ssha) tied together by a stable translation_group key. One dict drives per-language hreflang alternates, the language switcher, and the xhtml:link entries in a dynamically generated sitemap.xml. The subtle bit: get_lang() reads a ?lang= query param first, so an external hreflang link lands on the right language without a redirect — otherwise your hreflang quietly lies. Server-rendered HTML with heavy JSON-LD is exactly what crawlers want; there's no hydration for Googlebot to wait on.
  • Real features, not demos. Carfax history reports with automatic PDF reuse per VIN, and a reserve-price lookup with a fairness rule — if we can't find the number, your paid lookup converts to a free credit.
  • Ship safety. Push to main runs the full suite; only if it's green does CI SSH to the VPS and deploy. The deploy builds the new image before stopping the old container, so the site stays up during a rebuild and a failed build never takes production down.

The case for boring — and where it breaks

Every dependency I didn't add is a thing that can't break at 2am. No Redis to run out of memory. No Celery workers to silently die and strand jobs. No WebSocket connections to reap. No frontend build to go stale against the backend. No node_modules to audit.

The costs are real, and worth naming honestly:

  • Polling is chattier than a socket. Its cost scales with the number of clients; a socket's scales with the number of events. Past a certain fan-out the poll storm wins and you should switch. At this product's size that crossover isn't close.
  • A background task dies with its process. asyncio.create_task is not a durable queue — after a restart, an in-flight job is a stranded pending row. That's fine for a scrape you can re-run, and it's exactly why the payment webhook has a pull backstop instead of pretending the task is durable. Know which world you're in.

A few things I'd hand to anyone boxed in by the same constraints:

  • The database row can be your job queue: one status column, polled, is the single source of truth for an async job.
  • A Semaphore(n) is the concurrency bound a worker pool would sell you, in one line.
  • One provider callback can go multi-tenant if you encode the tenant into an id you already control, keep the ledgers in disjoint tables, and fan results out with a signed webhook plus a pull backstop.
  • Reach for a broker or a socket when clients vastly outnumber events, or when a job must survive a crash mid-flight. Otherwise, don't.

"Boring" here doesn't mean unambitious. Stealth scraping past Cloudflare, a payment router built on a slug prefix, receipts on a polling chat, hreflang across seven languages — none of that is boring engineering. It's just boring infrastructure, and that's the point. Spend your novelty budget on the problem, not the plumbing.

It's all live in production. Ask me anything about the trade-offs in the comments.

Top comments (0)