The latest articles on DEV Community by Sendil (@senthilsivam41). https://dev.to/senthilsivam41 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F879865%2Fe707cf4d-544a-4030-919c-2a097b8c32aa.jpeg https://dev.to/senthilsivam41 en Sendil Sun, 05 Jul 2026 00:52:38 +0000 https://dev.to/senthilsivam41/olap-and-oltp-lines-are-blurring-31pe https://dev.to/senthilsivam41/olap-and-oltp-lines-are-blurring-31pe <p>The Data Engineering Holy Grail: Running intensive analytical queries without starving your operational transactional database connections on the same host machine.</p> <p>Historically, this was an anti-pattern. A single rogue reporting query could blow out system RAM and crash your core application database.</p> <p>Extensions like <code>pg_lake</code> bypass this limitation by decoupling the storage plane entirely into cloud data lakes using Apache Iceberg, while offloading the OLAP execution path to an isolated background process running a vectorized DuckDB engine.</p> <p>I’ve mapped out the end-to-end execution flow, resource safety boundaries, and scheduling differences between macro-distributed engines and micro-morsel processing engines.</p> <p>Dive into the full architecture layout here:<br> <a href="https://github.com/Senthilsivam41/solution-catalog/blob/main/DeepDiveDuckDB.md" rel="noopener noreferrer">DeepDiveDuckDB</a></p> duckdb pglake Sendil Sun, 05 Jul 2026 00:50:46 +0000 https://dev.to/senthilsivam41/olap-and-oltp-lines-are-getting-blurred-find-out--599a https://dev.to/senthilsivam41/olap-and-oltp-lines-are-getting-blurred-find-out--599a <div class="crayons-card c-embed text-styles text-styles--secondary"> <div class="c-embed__content"> <div class="c-embed__cover"> <a href="https://github.com/Senthilsivam41/solution-catalog/blob/main/DeepDiveDuckDB.md" class="c-link align-middle" rel="noopener noreferrer"> <img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fopengraph.githubassets.com%2Ff56968e6f1d932250ea81d9fbd2057aeb3d0478a860377395b9b08ab36a8276b%2FSenthilsivam41%2Fsolution-catalog" height="600" class="m-0" width="1200"> </a> </div> <div class="c-embed__body"> <h2 class="fs-xl lh-tight"> <a href="https://github.com/Senthilsivam41/solution-catalog/blob/main/DeepDiveDuckDB.md" rel="noopener noreferrer" class="c-link"> solution-catalog/DeepDiveDuckDB.md at main · Senthilsivam41/solution-catalog · GitHub </a> </h2> <p class="truncate-at-3"> Solution Catalog to update my ideas and approaches to modern problem statements - Senthilsivam41/solution-catalog </p> <div class="color-secondary fs-s flex items-center"> <img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fgithub.githubassets.com%2Ffavicons%2Ffavicon.svg" width="32" height="32"> github.com </div> </div> </div> </div>