I wanted to see if a learned policy — not scripted behavior — could produce realistic e-commerce shopper sessions: browsing, getting tempted, abandoning carts, and influencing other shoppers through a social graph. This is FunnelForge: 50 RL agents trained on real review data, simulating a shopping population end to end.
The problem
Most shopper simulations are scripted or rule-based. I wanted agents that learned to behave like real shoppers, including the realistic parts — like abandoning carts most of the time, not just converting on demand.
Architecture
Two planes, deliberately separated:
Data plane (produces behavior):
- A min-heap event scheduler instead of fixed timesteps — agents act on their own continuous-time schedule rather than lockstep ticks. More realistic, more complex than a tick loop.
- An Erdős–Rényi social graph with BFS influence propagation (1/2^depth decay) — one agent's purchase can nudge its neighbors, decaying with graph distance.
- A topological funnel gate — agents move through browse → product_detail → cart → checkout, gated so they can't skip steps.
- All three compiled to C++ via pybind11, used by default, with an automatic pure-Python fallback if the compiled modules aren't present.
Control plane (moves + observes):
- Every action publishes to Redis and Kafka, streamed over a FastAPI WebSocket to a live dashboard.
Training
Behavioral cloning on real e-commerce review data first, then PPO fine-tuning on top. Pure RL from scratch was impractical here — sparse reward over a large discrete action space (which product, out of thousands) doesn't converge well without a good prior. BC gives the policy a realistic starting point; PPO refines it.
Validation — and the honest part
The simulation is stochastic over ~50 agents, so I validate against a seeded, reproducible batch run (FF_SEED=42), not the live dashboard feed:
| Metric | Simulated | Reference | Verdict |
|---|---|---|---|
| Session length (items) | 6.8 ± 6.1 | real 8.16 ± 7.08 | ✅ PASS |
| Conversion rate | 10% | 2–20% (industry) | ✅ PASS |
| Cart abandonment | 90% | 60–95% (industry) | ✅ PASS |
| Social influence (top-5) | 0.039 | 0.20–0.40 | ❌ FAIL |
Three of four metrics land inside real industry ranges. The fourth — social-influence concentration — doesn't, and I want to talk about that one specifically, because I think how you handle a failing metric matters more than getting everything to pass.
I traced the cause: a fallback path in the product catalog was thinning the distribution the metric depends on, weakening the measured contagion effect. Instead of tuning parameters until the number looked right, I documented the diagnosis and left the actual fix as an open problem — with one constraint: fixing it can't regress the other three passing metrics. That turned out to be a much more interesting problem than the number itself; the easy "fix" (crank up the social-boost parameter) breaks conversion and abandonment in the process.
What I'd check earlier next time
Two things surfaced late that I'd catch sooner now:
- A compiled component silently unused. The C++ pybind11 modules existed and compiled fine — but the live simulation was importing the Python implementation instead, the whole time. Nothing crashed, nothing warned. Only caught it while packaging, by actually testing the runtime path end to end instead of trusting that "it builds" meant "it runs correctly."
- Two code paths, two different numbers. I had a live-dashboard path and a separate batch-validation path, and they'd drifted apart — different numbers, and the validator was quietly broken against an API change elsewhere in the codebase. Took real effort to find one canonical, reproducible number and make sure every doc cited the same one.
Both are the same underlying lesson: verify the actual runtime behavior, not just that the code compiles or the demo looks right.
Stack
Python, PyTorch (BC + PPO), C++17/pybind11, Redis, Kafka, FastAPI, React/D3.
Try it
Code + full README: github.com/ojas4414/FunnelForge
Free to run and study — source-available under PolyForm Perimeter. docker compose up gets you the whole stack, live dashboard included. There's also a paid documentation bundle (architecture deep-dive, guided walkthrough, graded exercises, interview prep) for anyone who wants a more structured path through it — linked in the README.
Curious what people think of the min-heap-over-fixed-timesteps call, or whether Erdős–Rényi was the wrong social graph model to start with. Happy to be told either was a bad call.
Top comments (2)
On the Erdos-Renyi question: it is almost certainly the wrong generator for a shopping graph, because it gives you a degree distribution with a sharp mean and no hubs. Real purchase-influence networks are heavy-tailed - a few accounts touch a disproportionate share of sessions - so a scale-free or configuration-model graph changes the dynamics you are trying to observe, not just the constants. Shipping the negative result is the more valuable part though; most simulation write-ups quietly drop the run that did not confirm the thesis.
Good point actually — scale-free targets the degree distribution directly, more surgical than the small-world idea someone else mentioned. Different fix, same root diagnosis I think: Erdős–Rényi's uniform degree probably was never going to give me real concentration.
And "changes the dynamics, not just the constants" is a better way to put it than how I'd been thinking about this — I was treating it as "will this move the number," but really it's "does the number mean the same thing under a different graph." Good to get that straight before I just swap it and call it fixed.
And appreciate that last bit. Wasn't really a hard call — hiding a bad metric just means someone else finds it eventually, and then it's worse.