The latest articles on DEV Community by Krasimir Petkov (@krasimir_petkov_c14f3b461). https://dev.to/krasimir_petkov_c14f3b461 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3726717%2Ff8a42df4-583d-4628-b9af-b668c90b6f41.png https://dev.to/krasimir_petkov_c14f3b461 en Krasimir Petkov Thu, 22 Jan 2026 17:32:33 +0000 https://dev.to/krasimir_petkov_c14f3b461/building-a-real-time-stock-scanner-with-redis-websockets-and-postgres-2ig2 https://dev.to/krasimir_petkov_c14f3b461/building-a-real-time-stock-scanner-with-redis-websockets-and-postgres-2ig2 <h2> Background </h2> <p>This project started as an internal tool to understand unusual volume and intraday activity across thousands of US stocks.</p> <p>I recently built a real-time stock scanner that ingests thousands of symbols, processes live updates, and serves them to users with minimal delay. This post is a short breakdown of the architecture decisions that worked (and the ones that didn’t).</p> <h2> The Core Problem </h2> <p>Market data is:</p> <ul> <li>high-frequency</li> <li>bursty</li> <li>time-sensitive</li> </ul> <p>Writing every update directly to a relational database quickly becomes expensive and unnecessary — especially when most of the data is only relevant for seconds or minutes.</p> <h2> Architecture Overview </h2> <p>The system is split into three layers:</p> <ul> <li> <strong>Ingestion</strong> — pulling snapshot and streaming data from external APIs </li> <li> <strong>Processing</strong> — normalizing, filtering, and calculating derived metrics </li> <li> <strong>Distribution</strong> — pushing live updates to the frontend</li> </ul> <p>At a high level:</p> <ul> <li>Redis handles hot, ephemeral data</li> <li>Postgres stores validated, historical data</li> <li>WebSockets deliver updates to clients in real time</li> </ul> <h2> Why Redis for Live Data </h2> <p>Redis turned out to be a much better fit than Postgres for intraday updates:</p> <ul> <li>In-memory speed</li> <li>Natural TTL support for expiring symbols</li> <li>Pub/Sub for fan-out</li> <li>Zero WAL / IO pressure</li> </ul> <p>Postgres is still excellent — just not for data that changes every few seconds.</p> <h2> WebSockets for Distribution </h2> <p>Instead of polling:</p> <ul> <li>active symbols are tracked dynamically</li> <li>only “in-play” tickers receive updates</li> <li>clients subscribe to exactly what they need</li> </ul> <p>This keeps both bandwidth and server load under control.</p> <h2> What I’d Do Differently Next Time </h2> <ul> <li>Separate “hot” and “cold” data earlier</li> <li>Treat intraday data as disposable by default</li> <li>Avoid over-modeling early — schemas can come later</li> </ul> <h2> Live Demo </h2> <p>The project is live here:<br><br> <a href="https://zipston.com" rel="noopener noreferrer">https://zipston.com</a></p> <p>It’s still evolving, but the core real-time pipeline is already doing the heavy lifting.</p> <h2> Closing Thoughts </h2> <p>Real-time systems are less about frameworks and more about respecting the nature of your data.</p> <p>If your data expires quickly, your architecture should too.<br> Happy to answer questions or discuss real-time data tradeoffs.</p> webdev python devops opensource