The latest articles on DEV Community by Gaurav Sen (@gkcs). https://dev.to/gkcs https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1215866%2F350ac7e8-e013-4339-a24d-b00a9175ca66.jpeg https://dev.to/gkcs en Gaurav Sen Sun, 26 Nov 2023 11:18:01 +0000 https://dev.to/gkcs/whitepapers-worth-reading-595k https://dev.to/gkcs/whitepapers-worth-reading-595k <p>Here are some awesome white papers from Google, Amazon, Meta, and Apache. These white papers are worth your time if you are a software enthusiast or a tech geek.</p> <h2> Google </h2> <ol> <li><a href="https://static.googleusercontent.com/media/research.google.com/en//archive/gfs-sosp2003.pdf">Google File System</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/16cb30b4b92fd4989b8619a61752a2387c6dd474.pdf">Map Reduce Big Data Algorithm</a></li> <li><a href="https://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf">BigTable NoSQL Document Store</a></li> <li><a href="https://cloud.google.com/blog/products/storage-data-transfer/a-peek-behind-colossus-googles-file-system">Colossus Next Gen File Store</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/36971.pdf">Megastore Large Object Store</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/d84ab6c93881af998de877d0070a706de7bec6d8.pdf">Monarch Time Series DB</a></li> <li><a href="https://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf">Chubby Distributed Lock Management</a></li> <li><a href="https://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf">Spanner Distributed Database</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/45855.pdf">Spanner - CAP theorem considerations</a></li> <li><a href="https://static.googleusercontent.com/media/research.google.com/en//archive/papers/dapper-2010-1.pdf">Dapper Tracing System</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43438.pdf">Borg Cluster Management</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/10683a8987dbf0c6d4edcafb9b4f05cc9de5974a.pdf">Zanzibar Authentication System</a></li> <li><a href="https://15799.courses.cs.cmu.edu/fall2013/static/papers/p135-malewicz.pdf">Pregel Graph Processing</a></li> <li><a href="https://www.vldb.org/pvldb/vol14/p2986-sankaranarayanan.pdf">Napa - Data Warehousing</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/af91a9db5437586192a1ca87685e5b7d937a4e1a.pdf">Napa - Partitioning Algorithm</a></li> <li><a href="https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf">TensorFlow - Machine Learning at Scale</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/19fab8e81c9583f3ce67a8a137d1655970eb3efe.pdf">F1 - Fast Analytics</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/c9ebaf640152028a6cdedb684577a7c9e52b6f10.pdf">HALP - YouTube Content Delivery Network</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/42851.pdf">Mesa - Data Warehousing</a></li> <li><a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/d647cb73166040a82b7e5569574451be517f5c59.pdf">Firestore</a></li> </ol> <h2> Amazon </h2> <ol> <li><a href="https://assets.amazon.science/dc/2b/4ef2b89649f9a393d37d3e042f4e/amazon-aurora-design-considerations-for-high-throughput-cloud-native-relational-databases.pdf">Amazon Aurora DB Architecture</a></li> <li><a href="https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf">Dynamo DB NoSQL Database</a></li> </ol> <h2> Misc </h2> <ol> <li><a href="https://www.foundationdb.org/files/fdb-paper.pdf">Foundation DB NewSQL database</a></li> <li><a href="https://arxiv.org/pdf/2209.07663.pdf">Monolith Embedding in real time</a></li> <li><a href="https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf">Scalability at what COST</a></li> </ol> <h2> Meta </h2> <ol> <li><a href="https://www.vldb.org/pvldb/vol8/p1816-teller.pdf">Gorilla Time series DB</a></li> <li><a href="https://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf">Cassandra NoSQL DB</a></li> <li><a href="https://www.cidrdb.org/cidr2023/papers/p83-yadav.pdf">FlexiRaft Distributed Concensus Tradeoffs</a></li> <li><a href="https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final170_update.pdf">Memcache In-memory Cache</a></li> <li><a href="https://dl.acm.org/doi/pdf/10.1145/3517745.3561430">Millisampler Network Sampling</a></li> <li><a href="https://www.usenix.org/system/files/conference/atc13/atc13-bronson.pdf">TAO Graph Database</a></li> <li><a href="https://arxiv.org/pdf/2010.09974.pdf">MineSweeper - Root Cause Analysis</a></li> <li><a href="https://peerj.com/preprints/3190.pdf">Prophet - Forecasting at Scale</a></li> <li><a href="https://research.facebook.com/file/245575980870853/Shard-Manager-A-Generic-Shard-Management-Framework-for-Geo-distributed-Applications.pdf">ShardManager</a></li> <li><a href="https://research.facebook.com/file/2435031296630175/hive-a-warehousing-solution-over-a-map-reduce-framework.pdf">Hive - Map Reduce Jobs</a></li> <li><a href="https://thrift.apache.org/static/files/thrift-20070401.pdf">Thrift - Definition Language</a></li> <li><a href="https://research.facebook.com/file/214174457337055/Twine-A-Unified-Cluster-Management-System-for-Shared-Infrastructure-v2.pdf">Twine - Unified Cluster Management System</a></li> <li><a href="https://www.usenix.org/system/files/osdi23-saokar.pdf">ServiceRouter - Hyperscale and Servicemesh from Meta</a></li> </ol> <h2> Apache </h2> <ol> <li><a href="https://storageconference.us/2010/Papers/MSST/Shvachko.pdf">Hadoop File System</a></li> <li><a href="https://notes.stephenholiday.com/Kafka.pdf">Kafka Event Bus</a></li> <li><a href="https://asterios.katsifodimos.com/assets/publications/flink-deb.pdf">Flink</a></li> </ol> <p>Whitepapers have the benefit of discussing practical implementation details and the related tradeoffs to engineering products. </p> <p>The best engineers worldwide have written them after years of work, so don't miss any of them!</p> softwareengineering whitepapers google cloudcomputing