When I started building Rynko.it—a video game price tracker built to take on the giants in the niche—I thought the hardest part would be building the scraping engine. I was wrong.
The real nightmare was data normalization.
In the gaming world, a single title isn't just one product. You have the Standard Edition, Deluxe Edition, GOTY Edition, Pre-order bonuses, and regional variants. Multiply that by dozens of different keyshops, and if you don't structure your database correctly from day one, you end up with duplicate listings, broken relations, and a terrible user experience.
The Stack & Setup
My stack revolves around Next.js for the frontend (Server-Side Rendering is non-negotiable for SEO in this niche), Node.js, and PostgreSQL.
Side note on my workflow: I actually do most of my development on a Surface Pro 12 thin client, connected via Tailscale and SSH directly to my Linux VPS. It forces me to keep my environment clean and lightweight.
The Architecture Pivot
Recently, I had to execute a massive database refactoring. Initially, the data was too flat. I had to introduce a strict parent-child hierarchy.
Now, every game variant (e.g., Cyberpunk 2077: Ultimate Edition) maps back to a master "parent" game entity. Keyshop classifications are handled relationally so we can update prices in real-time without locking up the main game tables. It was a massive headache to migrate the schema without breaking the live scraper pipelines, but totally worth it for the query speed and cleaner Next.js frontend delivery.
A Question for the Community
I'm still optimizing. For those of you dealing with heavy e-commerce or catalog data:
How do you handle complex product variants in Postgres to keep query times low?
Any pro-tips for keeping Next.js SSR blazing fast when dealing with heavy relational DB queries?
If you want to see how the frontend handles the data right now, check out Rynko.it. It's a work in progress, but the engine is running! Would love your feedback.
Top comments (1)