DEV Community

CodingBlocks

Episode 45 – Caching Overview and Hardware

In this episode we give a general overview of caching, where it’s used, why it’s used, and what the differences in hardware implementations mean in terms we can understand.  This will be foundational to understanding caching at a software level in an upcoming episode.  There’s also something about the number 37 that may be the most important number to remember…ever…

The original version of the shownotes can be found at:
http://www.codingblocks.net/episode45

This episode’s survey:
#yop-poll-container-20_yp57bd12390ca57 { width: 400px; background:#fff; padding:10px; color:#555; overflow:hidden; font-size:12px; } #yop-poll-container-20_yp57bd12390ca57 input[type='text'] { margin:0px 0px 5px 0px; padding:2%; width:96%; text-indent:2%; font-size:12px; } .yop-poll-name-20_yp57bd12390ca57 { font-weight:bold; background:#327BD6; color:#fff; padding:5px; text-align:center; font-size:12px; } #yop-poll-questions-container-20_yp57bd12390ca57 { font-size:14px; margin:5px 0px; } .yop-poll-question-container-20_yp57bd12390ca57 { padding: 2px; } .yop-poll-question-20_yp57bd12390ca57 { background:#327BD6; color:#fff; margin-bottom: 21px; margin-top: -10px; font-style: italic; text-align: center; width: 100%; padding:5px; } .yop-poll-answers-20_yp57bd12390ca57 { } .yop-poll-answers-20_yp57bd12390ca57 ul { list-style: none outside none; margin: 0; padding: 0; } .yop-poll-li-answer-20_yp57bd12390ca57 { font-style:normal; margin:0px 0px 10px 0px; padding:0px; font-size:12px; margin-bottom:20px; } .yop-poll-li-answer-20_yp57bd12390ca57 input { margin:0px; float:none; } .yop-poll-li-answer-20_yp57bd12390ca57 label { margin:0px; font-style:normal; font-weight:normal; font-size:12px; float:none; } .yop-poll-results-20_yp57bd12390ca57 { font-size: 12px; font-style: italic; font-weight: normal; margin-left: 15px; } .yop-poll-customs-20_yp57bd12390ca57 { } .yop-poll-customs-20_yp57bd12390ca57 ul { list-style: none outside none; margin: 0; padding: 0; } .yop-poll-li-custom-20_yp57bd12390ca57 { padding:0px; margin:0px; font-size:14px; } /* Start CAPTCHA div style*/ #yop-poll-captcha-input-div-20_yp57bd12390ca57 { margin-top:5px; } #yop-poll-captcha-helpers-div-20_yp57bd12390ca57 { width:30px; float:left; margin-left:5px; height:0px; } #yop-poll-captcha-helpers-div-20_yp57bd12390ca57 img { margin-bottom:2px; } #yop-poll-captcha-image-div-20_yp57bd12390ca57 { margin-bottom:5px; } #yop_poll_captcha_image_20_yp57bd12390ca57 { float:left; } /* End CAPTCHA div style*/ .yop-poll-clear-20_yp57bd12390ca57 { clear:both; } #yop-poll-vote-20_yp57bd12390ca57 { } /* Start Result bar*/ .yop-poll-results-bar-20_yp57bd12390ca57 { background:#f5f5f5; height:10px; } .yop-poll-results-bar-20_yp57bd12390ca57 div { background:#555; height:10px; } /* End Result bar*/ /* Start Vote Button*/ #yop-poll-vote-20_yp57bd12390ca57 div#yop-poll-vote-20_yp57bd12390ca57 button { float:left; } #yop-poll-vote-20_yp57bd12390ca57 div#yop-poll-results-20_yp57bd12390ca57 { float: right; margin-bottom: 20px; margin-top: -20px; width: auto; } #yop-poll-vote-20_yp57bd12390ca57 div#yop-poll-results-20_yp57bd12390ca57 a { color:#fff; text-decoration:underline; font-size:12px; } #yop-poll-vote-20_yp57bd12390ca57 div#yop-poll-back-20_yp57bd12390ca57 a { color:#555; text-decoration:underline; font-size:12px; } #yop-poll-vote-20_yp57bd12390ca57 div#yop-poll-archive-20_yp57bd12390ca57 a { color:#555; text-decoration:underline; font-size:12px; } #yop-poll-vote-20_yp57bd12390ca57 div { float:left; width:100%; } /* End Vote Button*/ /* Start Messages*/ #yop-poll-container-error-20_yp57bd12390ca57 { font-size:12px; font-style:italic; color:red; text-transform:lowercase; margin-bottom:20px; text-align:center; } #yop-poll-container-success-20_yp57bd12390ca57 { font-size:12px; font-style:italic; color:green; margin-bottom:20px; text-align:center; } /* End Messages*/#yop-poll-container-20_yp57bd12390ca57 img { max-width: 400px; } .yop-poll-forms-display{}

If you were stranded on an island and you could have only one programming book, which would it be?
  • Code Complete: A Practical Handbook of Software Construction
  • Clean Code: A Handbook of Agile Software Craftsmanship
  • Structure and Interpretation of Computer Programs
  • Design Patterns: Elements of Reusable Object-Oriented Software
  • Head First Design Patterns
  • Refactoring: Improving the Design of Existing Code
  • Working Effectively With Legacy Code
  • Introduction to Algorithms
  • The Pragmatic Programmer
  • Cracking the Coding Interview
  • The Art of Unit Testing
  • The Art of Computer Programming (all volumes)
  • The Practice of Programming
  • Clean Coder : A Code of Conduct for Professional Programmers
  • How to Survive on a Deserted Island: Tim O'Shei
  • Rapid development - sidsavara
  • Adaptive Code via C# - Hall
  • Other

Podcast News

Thanks for your patience, we had a couple of rough audio situations – and we appreciate you sticking with us!

iTunes Reviews
Hedgehog, Thiagoramos.ai, Btn1992, Jonajonlee, UndeadCodemonkey, zmckinnon, hillsidecruzr, Dibjibjub, ddurose

Stitcher Reviews
pchtsp, rafaelh, CK142, TheMiddleMan124, LocalJoost

Clean Code episodes coming soon + book giveaway – Stay Tuned!

Caching: Turtles all the way down

Turtles all the way down???
https://en.wikipedia.org/wiki/Turtles_all_the_way_down

  • Storing a subset of information for faster retrieval
  • The hit ratio dramatically increases as the cache size increases
  • Think about a simple web request…
  • Browser cache
  • DNS cache
  • ISP caching
  • CDN
  • Whatever your application is doing (redis, framework, database, etc)
  • PLUS whatever the various computers are doing

Why don’t we cache everything?

  • Fast is expensive!
  • Cache Invalidation is hard!

Caching at the hardware level

Interactive Cache Visualization

Interactive Cache Visualization

Latency Numbers Every Programmer Should Know
https://gist.github.com/jboner/2841832

Relative Memory Access Interactive Demo
http://www.overbyte.com.au/misc/Lesson3/CacheFun.html

Caching is a strategy that computers use going all the way down to the processor

L1

  • .5ns
  • As quick as a it gets, how long it takes light to travel 6″
  • Managed by the CPU itself, no assembly available!

L2

  • 7ns
  • 14 x slower than L1
  • L3 / L4 / Scratch etc

Main Memory

  • Have numbers for a “reference” and a 1mb sequential read
  • 100ns – 250,000ns
  • 14 – 35,714 x slower than L2
  • 200 – 500,000 x slower than L1

Network

  • Sending is quick, there are numbers for that
  • In general, a lot of variability here
    Same datacenter
  • 500,000 ns
  • 2 x slower than Main Memory
  • 1 million times slower than L1

SSD

  • Wait, network faster than the hd??? Yes, but no
  • 1mb sequential
  • 1 million ns
  • 2 x slower than Network
  • 2 million x slower than L1

Spinning Disk

  • Get your employer to get you an ssd!
  • 1mb sequential read
  • 20 million ns
  • 20 x slower than SSD
  • 40 million x slower than L1

Internet

  • Rough gauge of internet speeds
  • Highly variable (CDN + ISP caching, for example), but gives you a sense of scale
  • 150 million ns
  • 7.5 x slower than spinning disk
  • 300 million times slower than L1

In more relatable terms.

  • 1 second for L1 Cache
  • 5 days for memory
  • 11 days for data center
  • 23 days for SSD
  • 15 months for HD
  • Almost 10 years for internet!

Think about how those numbers cache

  • RAM / Application cache
  • Local Hard drive
  • Network storage
  • Cache Server
  • DB

Summary

Hope we gave you a good idea of the importance and scale of caching in computing at the hardware level

Things we didn’t talk about coming in a future episode:

  • Application / Software caching and caching algorithms

Resources we Like

Latency Numbers Every Programmer Should Know
https://gist.github.com/jboner/2841832

How L1 and L2 caching work
http://www.extremetech.com/extreme/188776-how-l1-and-l2-cpu-caches-work-and-why-theyre-an-essential-part-of-modern-chips

Relative Memory Access Interactive Demo
http://www.overbyte.com.au/misc/Lesson3/CacheFun.html

Miscellaneous

Michael’s Favorite Meetup Ever
The Atlanta JavaScript Meetup
http://www.meetup.com/AtlantaJavaScript/events/222696324/?a=cr1_grp&rv=cr1

Hacking Interviews with:
Nick Larsen – http://cultureofdevelopment.com/
Sam Lawrence – http://www.samelawrence.com/

Tip of the Week

Algorithms to Live By

Algorithms to Live By

Joe: Algorithms to Live By
There’s something about the number 37%…
http://amzn.to/2aX1iJk

Michael: Use Sublime to replace \n with an actual new line by turning on RegEx search and replace. Or in Michael’s case, replace
with actual \n\t characters.
http://stackoverflow.com/questions/20515670/replace-n-with-actual-new-line-in-sublime-text

Allen: Collaborative Markdown Editor – What?!
http://www.hackmd.io

Episode source