<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Rob Cox</title>
    <description>The latest articles on DEV Community by Rob Cox (@robbie_cox).</description>
    <link>https://dev.to/robbie_cox</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4211330%2F40e0274a-1d8a-4324-8f2f-75aa503362f5.png</url>
      <title>DEV Community: Rob Cox</title>
      <link>https://dev.to/robbie_cox</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/robbie_cox"/>
    <language>en</language>
    <item>
      <title>I built a search engine from scratch to show my kids how Google finds things so fast</title>
      <dc:creator>Rob Cox</dc:creator>
      <pubDate>Thu, 12 Nov 2026 11:57:50 +0000</pubDate>
      <link>https://dev.to/robbie_cox/i-built-a-search-engine-from-scratch-to-show-my-kids-how-google-finds-things-so-fast-oa6</link>
      <guid>https://dev.to/robbie_cox/i-built-a-search-engine-from-scratch-to-show-my-kids-how-google-finds-things-so-fast-oa6</guid>
      <description>&lt;p&gt;How does a search box find one word in a million pages in the blink of an eye? It certainly isn't reading them all while you wait. I've been coding for 30+ years and I'd never actually built one, so I did.&lt;/p&gt;

&lt;p&gt;PySearch is a small full-text search engine in Python, with every idea explained in plain English. It has no dependencies and it's nowhere near Elasticsearch, but you can read the whole thing.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip install fromscratch-pysearch
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The librarian who read every book
&lt;/h2&gt;

&lt;p&gt;The docs open with a picture I love. Imagine a librarian who has read every book in the library. Ask her "which books mention dinosaurs?" and she doesn't open a single one, because she kept notes while she was reading.&lt;/p&gt;

&lt;p&gt;A search engine works the same way: it does the hard work before you ask. Here's a small library:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pysearch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;engine&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;SearchEngine&lt;/span&gt;

&lt;span class="n"&gt;engine&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;SearchEngine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"trex"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"The T. rex was a huge dinosaur with tiny arms and huge teeth"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"volcano"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"A volcano erupts hot lava and ash"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"fossils"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Fossils show us how a dinosaur lived. Dinosaur bones turn to stone"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"birds"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Birds are living dinosaurs that fly and sing"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"lava"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Lava cools and hardens into new rock"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"dinosaur"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"lava"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"dinosaur AND bones"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"volcano OR fossils"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;doc_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;score&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"  {doc_id:8} {score:.3f}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;dinosaur
  fossils  1.022
  birds    0.511
  trex     0.511
lava
  lava     0.916
  volcano  0.916
dinosaur AND bones
  fossils  2.631
volcano OR fossils
  fossils  1.609
  volcano  1.609
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Behind that, the text was chopped into words, the words were filed in a notebook, and the matches were ranked. Each step is its own short Python file.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cutting sentences into word cards
&lt;/h2&gt;

&lt;p&gt;First, the tokenizer. Picture the sentence written on a long strip of paper. You cut it into word cards, lowercase each one and bin the punctuation. Then you bin the boring cards too: "the", "a", "is", "and" and friends are stop words, so common they tell you nothing.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pysearch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;tokenizer&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;tokenize&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;tokenize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"The cat is on the mat"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'cat'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'mat'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Finally, a little stemmer trims each card to its root, so "dinosaurs" and "dinosaur" count as the same word. That's why the &lt;code&gt;birds&lt;/code&gt; page turned up above even though it only says "dinosaurs".&lt;/p&gt;

&lt;p&gt;The stemmer is deliberately simple and just snips common endings. Here's the fossils page as word cards:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;tokenize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"Fossils show us how a dinosaur lived. Dinosaur bones turn to stone"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'fossil'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'show'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'us'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'how'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'dinosaur'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'liv'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'dinosaur'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'bon'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'turn'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'stone'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;"liv" and "bon" look odd, but only the computer reads these cards. As long as "bones" in a page and "bones" in a question get trimmed the same way, they still match.&lt;/p&gt;

&lt;h2&gt;
  
  
  The librarian's notebook
&lt;/h2&gt;

&lt;p&gt;Now the clever bit. A normal index goes from a book to its words, like a table of contents. An inverted index flips it round and goes from a word to the books that contain it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="s2"&gt;"dinosaur"&lt;/span&gt; &lt;span class="err"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;fossils&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;birds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;trex&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="s2"&gt;"lava"&lt;/span&gt;     &lt;span class="err"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;volcano&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;lava&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In PySearch that's literally a dictionary of dictionaries: word, then document, then how many times the word appears. Finding a word is one dictionary lookup instead of a trip through every page.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pysearch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;index&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;InvertedIndex&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;InvertedIndex&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_document&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"trex"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"The T. rex was a huge dinosaur with tiny arms and huge teeth"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_document&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"fossils"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Fossils show us how a dinosaur lived. Dinosaur bones turn to stone"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_document&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"birds"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Birds are living dinosaurs that fly and sing"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;sorted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"dinosaurs"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'birds'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'fossils'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'trex'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_term_frequency&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"dinosaur"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"fossils"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_document_frequency&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"dinosaur"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;How much does the notebook actually help? I made 100,000 fake pages of 50 random words each, hid the word "stegosaurus" on one of them, and timed it on a modest two-core machine running Python 3.14 (your numbers will differ):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;indexed 100,000 pages in 7.9 s
index lookup: [('page42', 11.512925464970229)] in 0.110 ms
reading every page: ['page42'] in 223.7 ms
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Filing everything took a few seconds, once. After that the lookup came back in about a tenth of a millisecond, against roughly two tenths of a second to read every page, and the read-every-page loop was even cheating by skipping the stemming.&lt;/p&gt;

&lt;h2&gt;
  
  
  Which book comes first? A report card
&lt;/h2&gt;

&lt;p&gt;Finding matches is half the job. If ten books mention dinosaurs, which should you read first? PySearch uses TF-IDF, which multiplies two questions together.&lt;/p&gt;

&lt;p&gt;TF (term frequency) asks how often the word appears in this book. The fossils page says "dinosaur" twice, so it beats the pages that say it once. IDF (inverse document frequency) asks how rare the word is across all the books. It's &lt;code&gt;log(N / df)&lt;/code&gt;: five books, three mention dinosaurs, so &lt;code&gt;log(5 / 3) = 0.511&lt;/code&gt;. "Lava" is in only two of the five, so it's worth more: &lt;code&gt;log(5 / 2) = 0.916&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;When you search for several words, the docs describe it like a school report card: each word is a subject, each gets a grade, and the grades are added up. That's why "dinosaur AND bones" gave the fossils page 2.631: 1.022 for dinosaur, plus 1.609 for bones, a word that appears in only one book.&lt;/p&gt;

&lt;p&gt;The docs are upfront about the sharp edge here. A word that's in every book scores an IDF of exactly zero, which zeroes the whole score. A library with one book shows it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;solo&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;SearchEngine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;solo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"only"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Dinosaurs roar"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;solo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"dinosaur"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'only'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It still finds the book, it just can't tell you it's relevant. Smoothing the formula to &lt;code&gt;log(N / df) + 1&lt;/code&gt; is left as an exercise.&lt;/p&gt;

&lt;h2&gt;
  
  
  Understanding your question
&lt;/h2&gt;

&lt;p&gt;The query parser turns what you typed into a little tree. Words side by side mean "all of them", and longer chains read left to right:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pysearch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;parse_query&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;parse_query&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"cat AND dog OR fish"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nc"&gt;OrQuery&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;AndQuery&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;TermQuery&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;term&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'cat'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;right&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;TermQuery&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;term&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'dog'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;right&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;TermQuery&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;term&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'fish'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Query words get exactly the same clean-up as the books did, so &lt;code&gt;fox,&lt;/code&gt; searches for &lt;code&gt;fox&lt;/code&gt;, and &lt;code&gt;state-of-the-art&lt;/code&gt; becomes &lt;code&gt;state AND art&lt;/code&gt;. If nothing searchable is left, or an operator has nothing on one side, you get a &lt;code&gt;QueryParseError&lt;/code&gt; instead of a guess.&lt;/p&gt;

&lt;p&gt;There's also &lt;code&gt;NOT&lt;/code&gt;, which works like sorting a pile of books: grab everything that matches the left side, then put back any book that mentions the word on the right. Using the same little library:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;doc_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;score&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"dinosaur NOT bones"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;     &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"  {doc_id:8} {score:.3f}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="n"&gt;birds&lt;/span&gt;    &lt;span class="mf"&gt;0.511&lt;/span&gt;
  &lt;span class="n"&gt;trex&lt;/span&gt;     &lt;span class="mf"&gt;0.511&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The fossils page, which topped the plain "dinosaur" search, is gone because it mentions bones. The scores didn't change, because the word after &lt;code&gt;NOT&lt;/code&gt; only filters and never adds to a score. (A book can't be a better match for a word it doesn't contain.)&lt;/p&gt;

&lt;h2&gt;
  
  
  Closing the library for the night
&lt;/h2&gt;

&lt;p&gt;Everything so far lives in memory, which vanishes when the program stops. So the librarian writes every book into a notebook, a JSON file, and files them all again in the morning:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pysearch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;engine&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;SearchEngine&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pysearch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;persistence&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;load_engine&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;save_engine&lt;/span&gt;

&lt;span class="n"&gt;engine&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;SearchEngine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"cats"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Cats purr and nap in the sun"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;save_engine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;engine&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"library.json"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;tomorrow&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;load_engine&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"library.json"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tomorrow&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"purr"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  &lt;span class="c1"&gt;# [('cats', 0.0)] -- one book, so that IDF of zero again&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It saves the original text rather than the index, plus a format number so a future version refuses an old file instead of misreading it. Saving is done like a careful student copying out homework: write on a fresh sheet, wait for the ink to dry (&lt;code&gt;fsync&lt;/code&gt;), then swap it in for the old one in a single step. A crash halfway through leaves yesterday's notebook untouched.&lt;/p&gt;

&lt;h2&gt;
  
  
  What real search engines do that this doesn't
&lt;/h2&gt;

&lt;p&gt;PySearch keeps the simple version of nearly everything, and you notice some of it quickly when you play with it.&lt;/p&gt;

&lt;p&gt;Real engines use a proper stemmer (Porter, Snowball) or a dictionary. PySearch just snips endings, so "running" becomes &lt;code&gt;runn&lt;/code&gt; and "runs" becomes &lt;code&gt;run&lt;/code&gt;, and they don't match each other. "ran" doesn't match either. The notebook also stores counts rather than positions, so there's no &lt;code&gt;"tiny arms"&lt;/code&gt; phrase search and no bonus for words that sit close together.&lt;/p&gt;

&lt;p&gt;Ranking is simpler too. Real engines use BM25, which stops a word mentioned 50 times from beating one mentioned 5 times by tenfold, and allows for document length. Google adds links, freshness and much more. And at real scale the posting lists get compressed and sharded across many machines instead of living in one Python dictionary, with a crawler feeding it pages. PySearch starts from text you hand it.&lt;/p&gt;

&lt;p&gt;Any of those would make a good weekend exercise. I was surprised how little the core needs: word cards, a notebook from words to books, and a report card for ranking.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Install: &lt;code&gt;pip install fromscratch-pysearch&lt;/code&gt;, then &lt;code&gt;from pysearch.engine import SearchEngine&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Docs, with every idea explained simply: &lt;a href="https://robcox2910.github.io/pysearch/" rel="noopener noreferrer"&gt;https://robcox2910.github.io/pysearch/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Code: &lt;a href="https://github.com/robcox2910/pysearch" rel="noopener noreferrer"&gt;https://github.com/robcox2910/pysearch&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;PySearch is part of From scratch in Python, a series of 18 projects that rebuild real systems: a database, an operating system, git, a tiny GPT and more. Get the whole series with &lt;code&gt;pip install fromscratch-pystack&lt;/code&gt;, or start at &lt;a href="https://robcox2910.github.io/" rel="noopener noreferrer"&gt;https://robcox2910.github.io/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;How this was made: I designed PySearch and directed every part of it, but most of the code was written with Claude Code, under strict test-driven development, and this article was drafted with Claude's help too. Every output shown above comes from actually running it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>beginners</category>
      <category>algorithms</category>
    </item>
    <item>
      <title>I built a tiny GPT from scratch to show my kids how chatbots really work</title>
      <dc:creator>Rob Cox</dc:creator>
      <pubDate>Wed, 04 Nov 2026 22:09:55 +0000</pubDate>
      <link>https://dev.to/robbie_cox/i-built-a-tiny-gpt-from-scratch-to-show-my-kids-how-chatbots-really-work-238c</link>
      <guid>https://dev.to/robbie_cox/i-built-a-tiny-gpt-from-scratch-to-show-my-kids-how-chatbots-really-work-238c</guid>
      <description>&lt;p&gt;What actually happens inside a GPT? Everyone talks about "attention" and "transformers" and "training", but if you had to build one with nothing but arrays of numbers, where would you even start? I've been coding for over 30 years and I couldn't have told you.&lt;/p&gt;

&lt;p&gt;So I built a small one in Python and called it PyLLM. There's no PyTorch and no TensorFlow, just numpy for the array maths. The autograd engine, the layers, attention, the tokenizer, the optimiser, the training loop and the sampling are all plain Python you can read.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;fromscratch-pyllm
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Dream up some Pokémon
&lt;/h2&gt;

&lt;p&gt;PyLLM ships with a tiny model that's already trained, so it starts writing straight away:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pyllm &lt;span class="nt"&gt;--seed&lt;/span&gt; 0 &lt;span class="nt"&gt;--max-new-tokens&lt;/span&gt; 40
&lt;span class="go"&gt;machop
machoke
machamp
bellsprout
weepin
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It wrote that one letter at a time: it guessed &lt;code&gt;m&lt;/code&gt;, then &lt;code&gt;a&lt;/code&gt;, then &lt;code&gt;c&lt;/code&gt;, and fed each guess back in as the start of the next one. (&lt;code&gt;weepin&lt;/code&gt; gets cut off because I only gave it 40 characters.) A GPT does exactly this, just with far more practice.&lt;/p&gt;

&lt;h2&gt;
  
  
  Chopping text into Lego bricks
&lt;/h2&gt;

&lt;p&gt;A model can't read letters. First the text gets chopped into tokens, a bit like snapping a sentence into Lego bricks, and every brick gets a number. The model only ever sees the numbers:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pyllm tokenize &lt;span class="nt"&gt;--text&lt;/span&gt; &lt;span class="s2"&gt;"Pika"&lt;/span&gt;
&lt;span class="gp"&gt;'P' -&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;I have not seen that character yet&lt;span class="o"&gt;)&lt;/span&gt;
&lt;span class="gp"&gt;'i' -&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;9
&lt;span class="gp"&gt;'k' -&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;11
&lt;span class="gp"&gt;'a' -&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Here every character is its own brick. The alphabet was learned from lower-case Pokémon names, so a capital &lt;code&gt;P&lt;/code&gt; simply isn't in it. (There's also a BPE tokenizer that learns bigger bricks like &lt;code&gt;"ing"&lt;/code&gt;, which is closer to what the big models use.)&lt;/p&gt;

&lt;h2&gt;
  
  
  The breadcrumb trail
&lt;/h2&gt;

&lt;p&gt;Learning means asking every number in the model: if I nudge you up a little, does the answer get better or worse, and by how much? That's a gradient, and working it out automatically is called autograd.&lt;/p&gt;

&lt;p&gt;The docs explain it with breadcrumbs. Every time PyLLM's &lt;code&gt;Tensor&lt;/code&gt; does some maths, it drops a breadcrumb saying who its parents were and how to pass the blame back to them. Then &lt;code&gt;backward()&lt;/code&gt; follows the trail in reverse:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pyllm.autograd&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Tensor&lt;/span&gt;

&lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Tensor&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="mf"&gt;2.0&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;span class="n"&gt;b&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Tensor&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="mf"&gt;3.0&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;      &lt;span class="c1"&gt;# c = 2*3 + 2 = 8
&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;backward&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;grad&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;grad&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;# [8.] [4.] [2.]
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;a&lt;/code&gt; was used twice, once multiplied by 3 and once on its own, so its blame adds up to 4. Each op is a few lines: work out the answer, then leave a small &lt;code&gt;_backward&lt;/code&gt; recipe behind.&lt;/p&gt;

&lt;p&gt;How do you know the breadcrumbs are right? You check them the slow, obvious way: nudge each number a tiny bit and see how far the output moves. PyLLM's &lt;code&gt;numerical_grad&lt;/code&gt; does exactly that, and the tests use it on every op and layer. Everything else in PyLLM sits on top of this one idea.&lt;/p&gt;

&lt;h2&gt;
  
  
  Re-reading a sentence
&lt;/h2&gt;

&lt;p&gt;When you read "The trophy didn't fit in the case because &lt;em&gt;it&lt;/em&gt; was too big", you glance back to work out what "it" means. Attention is the model doing the same thing. Each token asks a question (a query), every earlier token holds up a label (a key), and the good matches hand over their information (a value). A causal mask stops anyone peeking at letters that haven't been written yet.&lt;/p&gt;

&lt;p&gt;A few attention "heads" side by side, plus a bit of private thinking (the feed-forward part), make one Transformer block. Stack a few blocks and you've got a GPT. The bundled Pokémon model has two blocks and 59,739 numbers in total. A real chatbot has billions.&lt;/p&gt;

&lt;h2&gt;
  
  
  The guessing game, three ways
&lt;/h2&gt;

&lt;p&gt;Training is a guessing game. Guess the next letter, find out how surprised you should have been (the loss), follow the breadcrumbs to share out the blame, nudge every number a little, and repeat.&lt;/p&gt;

&lt;p&gt;PyLLM gives you a ladder of three models, each one fixing a weakness of the one below. I trained each for 2,000 steps on the bundled list of 149 Pokémon names:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pyllm train &lt;span class="nt"&gt;--model&lt;/span&gt; bigram &lt;span class="nt"&gt;--steps&lt;/span&gt; 2000 &lt;span class="nt"&gt;--seed&lt;/span&gt; 0 &lt;span class="nt"&gt;--out&lt;/span&gt; bigram.npz
&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pyllm train &lt;span class="nt"&gt;--model&lt;/span&gt; mlp    &lt;span class="nt"&gt;--steps&lt;/span&gt; 2000 &lt;span class="nt"&gt;--seed&lt;/span&gt; 0 &lt;span class="nt"&gt;--out&lt;/span&gt; mlp.npz
&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pyllm train &lt;span class="nt"&gt;--model&lt;/span&gt; gpt    &lt;span class="nt"&gt;--steps&lt;/span&gt; 2000 &lt;span class="nt"&gt;--seed&lt;/span&gt; 0 &lt;span class="nt"&gt;--out&lt;/span&gt; gpt.npz
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;What it looks at&lt;/th&gt;
&lt;th&gt;Final loss&lt;/th&gt;
&lt;th&gt;Time on my run&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Bigram&lt;/td&gt;
&lt;td&gt;just the last letter&lt;/td&gt;
&lt;td&gt;2.3501&lt;/td&gt;
&lt;td&gt;about 1 second&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MLP&lt;/td&gt;
&lt;td&gt;the last 3 letters&lt;/td&gt;
&lt;td&gt;0.7192&lt;/td&gt;
&lt;td&gt;about 2 seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;GPT&lt;/td&gt;
&lt;td&gt;the last 16 letters, with attention&lt;/td&gt;
&lt;td&gt;0.2996&lt;/td&gt;
&lt;td&gt;about 4 minutes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Then I asked each one for names with the same seed (&lt;code&gt;pyllm --checkpoint &amp;lt;model&amp;gt;.npz --seed 3 --max-new-tokens 60&lt;/code&gt;):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Bigram&lt;/th&gt;
&lt;th&gt;MLP&lt;/th&gt;
&lt;th&gt;GPT&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;chon&lt;/td&gt;
&lt;td&gt;dodrio&lt;/td&gt;
&lt;td&gt;choke&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;le&lt;/td&gt;
&lt;td&gt;meapodeon&lt;/td&gt;
&lt;td&gt;machamp&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;r&lt;/td&gt;
&lt;td&gt;flearowth&lt;/td&gt;
&lt;td&gt;bellsprout&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;kangowarie&lt;/td&gt;
&lt;td&gt;geom&lt;/td&gt;
&lt;td&gt;weepinbell&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;wakingo&lt;/td&gt;
&lt;td&gt;vilectrete&lt;/td&gt;
&lt;td&gt;victreebel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;picltararlasioraro&lt;/td&gt;
&lt;td&gt;ponyte&lt;/td&gt;
&lt;td&gt;tentacool&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;sefaratarar&lt;/td&gt;
&lt;td&gt;mandshrew&lt;/td&gt;
&lt;td&gt;ten&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;This was the most interesting thing I learned. The bigram only sees one letter, so it babbles. The MLP makes up names that sound right, like "flearowth" and "mandshrew". The GPT has the lowest loss and mostly just recites real Pokémon in order. (It starts with "choke" rather than "machoke" because it begins with nothing to read.)&lt;/p&gt;

&lt;p&gt;With only 149 names to learn from, the strongest model basically memorised the list, and the bundled checkpoint does too. Over 20 seeds at the default setting, only 7 of the 223 complete names it wrote weren't real Pokémon.&lt;/p&gt;

&lt;p&gt;You can push it to be bolder with a dial called temperature. Think of the model's scores as a weighted dice where the letters it likes get bigger sides. At 0 it always picks the favourite. Turn it up and the dice flattens out, so long shots get a chance:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pyllm &lt;span class="nt"&gt;--seed&lt;/span&gt; 0 &lt;span class="nt"&gt;--temperature&lt;/span&gt; 1.5 &lt;span class="nt"&gt;--max-new-tokens&lt;/span&gt; 60
&lt;span class="go"&gt;ndbell
victafble
vulpix
nizweepinbell
victreebel
tentacool
t
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Counted the same way at 1.5, 66 of 217 names were new. "Nizweepinbell" is my favourite.&lt;/p&gt;

&lt;h2&gt;
  
  
  Teaching it a language with no data
&lt;/h2&gt;

&lt;p&gt;The main demo is harder. Pebble is a small programming language from the same series, and there's no pile of Pebble code anywhere in the world to learn from. So PyLLM grows its own. The docs put it as planting your own garden instead of foraging for wild berries.&lt;/p&gt;

&lt;p&gt;A generator builds random programs by rolling dice at every choice ("loop or print? add or multiply?"), but it only makes moves the grammar allows, so every program it writes is valid. Add the real snippets from Pebble's docs and you get the bundled corpus: 1,001 programs, about 354,000 characters.&lt;/p&gt;

&lt;p&gt;Because the series has Pebble's real parser too, we can mark the model's homework honestly. Does what it wrote actually parse?&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pyllm pebble &lt;span class="nt"&gt;--seed&lt;/span&gt; 7 &lt;span class="nt"&gt;--temperature&lt;/span&gt; 0.5 &lt;span class="nt"&gt;--score&lt;/span&gt;
&lt;span class="go"&gt;let total = n * (5 + n)
        let a = 6
    }
    let a = result * (6 - result)
    }
}
&lt;/span&gt;&lt;span class="c"&gt;...
&lt;/span&gt;&lt;span class="go"&gt;parse rate: 0.0% of 1 programs
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;(Output trimmed.) It looks like Pebble, but the braces are all over the place. Over 20 seeds, only 4% of 27 programs parsed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Call in the referee
&lt;/h2&gt;

&lt;p&gt;So PyLLM brings in a referee. A football referee doesn't play and doesn't tell anyone where to kick. They know the rules, and when someone tries an illegal move they blow the whistle, the move doesn't count, and the player has to choose again.&lt;/p&gt;

&lt;p&gt;PyLLM's referee is a small hand-written parser that answers two questions about half-written code:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pyllm.pebble&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;referee&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;let total = 5 +&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;let total = 5 +&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;let total = 5 + 2&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;let total = = 5&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="si"&gt;!r:&lt;/span&gt;&lt;span class="mi"&gt;22&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; viable=&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;referee&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;viable&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;!s:&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; complete=&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;referee&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;complete&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;'let total = 5 +'      viable=True  complete=False
'let total = 5 +\n'    viable=False complete=False
'let total = 5 + 2'    viable=True  complete=True
'let total = = 5'      viable=False complete=False
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;While the model writes, every character it picks gets checked. If the code could no longer turn out valid, the whistle goes: that character is crossed off the dice and the model rolls again. Same tiny model, but now:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pyllm pebble &lt;span class="nt"&gt;--seed&lt;/span&gt; 3 &lt;span class="nt"&gt;--constrained&lt;/span&gt; &lt;span class="nt"&gt;--score&lt;/span&gt; &lt;span class="nt"&gt;--max-new-tokens&lt;/span&gt; 300
&lt;span class="go"&gt;let a = 10

let x = calc() == 7 + 15 - (n + 97)
print(step(1 - 89 - z, y - 8))

parse rate: 100.0% of 2 programs
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Over the same 20 seeds, 100% of 37 programs parsed.&lt;/p&gt;

&lt;p&gt;There's a catch, though: parsing and working are different things. Save that program to a file and run it with Pebble itself (the &lt;code&gt;pebble&lt;/code&gt; command comes with the &lt;code&gt;[pebble]&lt;/code&gt; extra):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pebble s3.pbl
&lt;span class="go"&gt;3 | let x = calc() == 7 + 15 - (n + 97)
            ^
Undeclared function 'calc'
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The referee checks the grammar, not the meaning. To be fair to the model, the generated training programs call made-up functions like &lt;code&gt;compute()&lt;/code&gt; and &lt;code&gt;combine()&lt;/code&gt; too, so it learned to write exactly what it was shown.&lt;/p&gt;

&lt;h2&gt;
  
  
  What real LLMs do that this doesn't
&lt;/h2&gt;

&lt;p&gt;PyLLM is built to be read, so it keeps the simple version of a lot of things.&lt;/p&gt;

&lt;p&gt;The obvious one is scale: about 60,000 numbers here, billions in a real model, trained on a large chunk of the internet on racks of GPUs. It's the same loop at a very different size. The bundled models also read one character at a time, where real models use learned sub-word tokens (the BPE tokenizer shows the idea), and the Pokémon model can only look back 16 characters, where real ones look back hundreds of thousands of tokens.&lt;/p&gt;

&lt;p&gt;There's no GPU and no caching of earlier attention work either, so every step is plain numpy. And real assistants get extra training to follow requests and be helpful, while PyLLM only ever learns to continue text.&lt;/p&gt;

&lt;p&gt;The referee is the one part that matches production systems. They use the same "veto illegal tokens" trick to force valid JSON or code, and PyLLM's version is small enough to read through.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Install: &lt;code&gt;pip install fromscratch-pyllm&lt;/code&gt; (add &lt;code&gt;'fromscratch-pyllm[pebble]'&lt;/code&gt; for parse scores and the &lt;code&gt;pebble&lt;/code&gt; command), then run &lt;code&gt;pyllm&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Docs, with every idea explained simply: &lt;a href="https://robcox2910.github.io/pyllm/" rel="noopener noreferrer"&gt;https://robcox2910.github.io/pyllm/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Code: &lt;a href="https://github.com/robcox2910/pyllm" rel="noopener noreferrer"&gt;https://github.com/robcox2910/pyllm&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;PyLLM is part of From scratch in Python, a series of 18 projects that rebuild real systems: a database, an operating system, git, a search engine, a programming language and more. Get the whole series with &lt;code&gt;pip install fromscratch-pystack&lt;/code&gt;, or start at &lt;a href="https://robcox2910.github.io/" rel="noopener noreferrer"&gt;https://robcox2910.github.io/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;How this was made: I designed PyLLM and directed every part of it, but most of the code was written with Claude Code, under strict test-driven development, and this article was drafted with Claude's help too. Every output shown above comes from actually running it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>ai</category>
      <category>beginners</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>I built Git from scratch in Python to find out what’s inside the .git folder</title>
      <dc:creator>Rob Cox</dc:creator>
      <pubDate>Wed, 28 Oct 2026 16:25:26 +0000</pubDate>
      <link>https://dev.to/robbie_cox/i-built-git-from-scratch-in-python-to-find-out-whats-inside-the-git-folder-49ck</link>
      <guid>https://dev.to/robbie_cox/i-built-git-from-scratch-in-python-to-find-out-whats-inside-the-git-folder-49ck</guid>
      <description>&lt;p&gt;Every git project has a hidden &lt;code&gt;.git&lt;/code&gt; folder, and somehow that one folder holds &lt;em&gt;every version of every file you've ever committed&lt;/em&gt;. Branches, history, the lot. I've used git for years without really knowing what was in there.&lt;/p&gt;

&lt;p&gt;So I built a small version of it from scratch in Python, called &lt;strong&gt;PyGit&lt;/strong&gt;, to find out. It turns out the core idea is simple enough to explain with a thumbprint, a scrapbook and a diary.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;fromscratch-pygit
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Make a repository
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pygit init
&lt;span class="go"&gt;Initialized empty pygit repository in /home/you/story
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That creates a hidden &lt;code&gt;.pygit&lt;/code&gt; folder, which is PyGit's version of &lt;code&gt;.git&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;.pygit/
  HEAD          which branch you're on
  index         the staging area
  objects/      every version of every file, ever
  refs/heads/   one tiny file per branch
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Almost everything interesting lives in &lt;code&gt;objects/&lt;/code&gt;. Let's put something in it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Fingerprints for files
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"Once upon a time"&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; story.txt
&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pygit add story.txt
&lt;span class="go"&gt;Staged story.txt (12abed85)
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That &lt;code&gt;12abed85&lt;/code&gt; is a &lt;strong&gt;hash&lt;/strong&gt;: a fingerprint of the file's contents. Press your thumb on an ink pad and you get a print that's unique to you, and the same every time. A hash works the same way for data. The same content always gives the same fingerprint, and change one letter and you get a completely different one.&lt;/p&gt;

&lt;p&gt;Here's the fun part. Ask the real git for the fingerprint of the same text:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"Once upon a time"&lt;/span&gt; | git hash-object &lt;span class="nt"&gt;--stdin&lt;/span&gt;
&lt;span class="go"&gt;12abed85bde98c2c401eb221069e59386ae3fd2e
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Identical. PyGit uses the same recipe as git: put the label &lt;code&gt;blob &amp;lt;size&amp;gt;&lt;/code&gt; in front of the content and run it through SHA-1.&lt;/p&gt;

&lt;h2&gt;
  
  
  A scrapbook of photos
&lt;/h2&gt;

&lt;p&gt;The file's content is now saved under its fingerprint:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;cat&lt;/span&gt; .pygit/objects/12/abed85bde98c2c401eb221069e59386ae3fd2e
&lt;span class="go"&gt;blob 17 Once upon a time
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is a &lt;strong&gt;blob&lt;/strong&gt;. Think of it as a photo of a page in a scrapbook. It stores the content, labelled with its fingerprint, but it doesn't know the file's name or where it lives. (The first two characters of the hash become a folder name so that no single folder ends up with millions of files in it.)&lt;/p&gt;

&lt;p&gt;Storing things by fingerprint has a lovely side effect. Copy the file and add the copy:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;cp &lt;/span&gt;story.txt copy.txt
&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pygit add copy.txt
&lt;span class="go"&gt;Staged copy.txt (12abed85)
&lt;/span&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;find .pygit/objects &lt;span class="nt"&gt;-type&lt;/span&gt; f | &lt;span class="nb"&gt;wc&lt;/span&gt; &lt;span class="nt"&gt;-l&lt;/span&gt;
&lt;span class="go"&gt;1
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Same content, same fingerprint, so it's stored &lt;strong&gt;once&lt;/strong&gt;. Two names, one photo. That's a big part of why a git repository stays smaller than you'd expect.&lt;/p&gt;

&lt;h2&gt;
  
  
  A table of contents and a diary
&lt;/h2&gt;

&lt;p&gt;If blobs don't know their names, something else must. That's a &lt;strong&gt;tree&lt;/strong&gt;: a table of contents that lists each filename and the fingerprint of its blob, with sub-trees for folders.&lt;/p&gt;

&lt;p&gt;Then a &lt;strong&gt;commit&lt;/strong&gt; is a diary entry pointing at a tree:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pygit commit &lt;span class="nt"&gt;-m&lt;/span&gt; &lt;span class="s2"&gt;"Start the story"&lt;/span&gt;
&lt;span class="go"&gt;[main 2c1e492d] Start the story
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Make a change and commit again, then look inside the second commit (hashes shortened here):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;cat&lt;/span&gt; .pygit/objects/de/811a25ee2a3bd0b859c6b983d5bc39e5208c37
&lt;span class="go"&gt;commit 205 {"tree": "accd9d19...", "parent": "2c1e492d...", "author": "Anonymous",
            "timestamp": "2026-10-27T16:02:24+00:00", "message": "Add a dragon"}
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Everything a diary entry needs: &lt;em&gt;what&lt;/em&gt; the project looked like (the tree), &lt;em&gt;who&lt;/em&gt;, &lt;em&gt;when&lt;/em&gt;, &lt;em&gt;why&lt;/em&gt;, and crucially &lt;strong&gt;what came before&lt;/strong&gt; (the parent). Each commit points back to its parent, so the history is just a chain you can walk backwards:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;pygit log          &lt;span class="c"&gt;# author and date lines trimmed&lt;/span&gt;
&lt;span class="go"&gt;commit de811a25ee2a3bd0b859c6b983d5bc39e5208c37
    Add a dragon

commit 2c1e492d2642f1769b58c26dbfd7a2d2e8acfd66
    Start the story
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And because a commit's fingerprint covers its parent's fingerprint, you can't quietly change history: alter an old commit and every fingerprint after it changes too.&lt;/p&gt;

&lt;h2&gt;
  
  
  A branch is just a sticky note
&lt;/h2&gt;

&lt;p&gt;This was the bit that surprised me most. A branch is a tiny text file containing one commit's fingerprint. That's it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;$&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;cat&lt;/span&gt; .pygit/refs/heads/main
&lt;span class="go"&gt;de811a25ee2a3bd0b859c6b983d5bc39e5208c37
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Making a branch writes one more of those files. Committing on a branch just moves its sticky note forward:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pygit.repository&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Repository&lt;/span&gt;

&lt;span class="n"&gt;repo&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Repository&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;story&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;repo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create_branch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;happy-ending&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;repo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;switch_branch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;happy-ending&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;# ...edit story.txt...
&lt;/span&gt;&lt;span class="n"&gt;repo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;story.txt&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;repo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;commit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Give it a happy ending&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;repo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;()]&lt;/span&gt;
&lt;span class="c1"&gt;# ['Give it a happy ending', 'Add a dragon', 'Start the story']
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Switch back to &lt;code&gt;main&lt;/code&gt; and its log still stops at "Add a dragon". The happy ending only exists on its own branch, like a page in a Choose Your Own Adventure book.&lt;/p&gt;

&lt;h2&gt;
  
  
  Spot the difference
&lt;/h2&gt;

&lt;p&gt;Finally, &lt;strong&gt;diff&lt;/strong&gt; compares two versions line by line, like a spot-the-difference puzzle:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pygit.diff&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;diff_lines&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;format_diff&lt;/span&gt;

&lt;span class="n"&gt;before&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Once upon a time&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="s"&gt;there was a dragon&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="n"&gt;after&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;before&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;who became a friend&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;format_diff&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;diff_lines&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;before&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;after&lt;/span&gt;&lt;span class="p"&gt;)))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;  Once upon a time
  there was a dragon
&lt;span class="gi"&gt;+ who became a friend
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Lines with &lt;code&gt;+&lt;/code&gt; were added, &lt;code&gt;-&lt;/code&gt; removed, and everything else is unchanged.&lt;/p&gt;

&lt;h2&gt;
  
  
  What real git does that PyGit doesn't
&lt;/h2&gt;

&lt;p&gt;PyGit is built to be read, so I kept the simple version of a few things. Each one makes a good exercise:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Compression.&lt;/strong&gt; Real git squeezes every object with zlib. PyGit stores them as plain text so you can &lt;code&gt;cat&lt;/code&gt; them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tree and commit format.&lt;/strong&gt; Blob hashes match git exactly, but PyGit writes trees and commits as simple JSON, so those hashes differ from git's.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Checkout.&lt;/strong&gt; Switching branches in PyGit moves &lt;code&gt;HEAD&lt;/code&gt; but doesn't rewrite your files to match. Real git does both.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Merging and packfiles.&lt;/strong&gt; Not there yet. Merging two branches is the next big idea to build.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Even so, the core of git turned out to be four ideas stacked on top of each other: fingerprints, blobs, trees and commits. Once you've built them, that &lt;code&gt;.git&lt;/code&gt; folder stops being a mystery.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Install: &lt;code&gt;pip install fromscratch-pygit&lt;/code&gt;, then &lt;code&gt;pygit init&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Docs, with every idea explained simply: &lt;a href="https://robcox2910.github.io/pygit/" rel="noopener noreferrer"&gt;https://robcox2910.github.io/pygit/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Code: &lt;a href="https://github.com/robcox2910/pygit" rel="noopener noreferrer"&gt;https://github.com/robcox2910/pygit&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;PyGit is part of &lt;strong&gt;From scratch in Python&lt;/strong&gt;, a series of 18 projects that rebuild real systems: a database, an operating system, a search engine, a tiny GPT and more. Get the whole series with &lt;code&gt;pip install fromscratch-pystack&lt;/code&gt;, or start at &lt;a href="https://robcox2910.github.io/" rel="noopener noreferrer"&gt;https://robcox2910.github.io/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;How this was made: I designed PyGit and directed every part of it, but most of the code was written with Claude Code, under strict test-driven development, and this article was drafted with Claude’s help too. Every output shown above comes from actually running it.&lt;/p&gt;

</description>
      <category>python</category>
      <category>git</category>
      <category>beginners</category>
      <category>learning</category>
    </item>
    <item>
      <title>I built a database from scratch to show my kids how databases work</title>
      <dc:creator>Rob Cox</dc:creator>
      <pubDate>Tue, 27 Oct 2026 15:57:11 +0000</pubDate>
      <link>https://dev.to/robbie_cox/i-built-a-database-from-scratch-to-show-my-kids-how-databases-work-19a7</link>
      <guid>https://dev.to/robbie_cox/i-built-a-database-from-scratch-to-show-my-kids-how-databases-work-19a7</guid>
      <description>&lt;p&gt;Databases feel like magic. You type &lt;code&gt;SELECT * FROM cards WHERE name = 'Pikachu'&lt;/code&gt; and an answer comes back, instantly, from millions of rows. I've been coding for 30+ years and I still couldn't have explained to my kids what actually happens in between.&lt;/p&gt;

&lt;p&gt;So I built one. &lt;strong&gt;PyDB&lt;/strong&gt; is a small relational database written from scratch in Python, with every idea explained in plain English. It's not meant to replace Postgres. It's meant to be &lt;em&gt;read&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;You can try it in one line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;fromscratch-pydb
pydb
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Start with a table
&lt;/h2&gt;

&lt;p&gt;A table is just a grid: columns across the top, one row per thing. Let's make some trading cards.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;pydb&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="nb"&gt;TEXT&lt;/span&gt; &lt;span class="k"&gt;NOT&lt;/span&gt; &lt;span class="k"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="nb"&gt;TEXT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;power&lt;/span&gt; &lt;span class="nb"&gt;INTEGER&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;pydb&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;INSERT&lt;/span&gt; &lt;span class="k"&gt;INTO&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt; &lt;span class="k"&gt;VALUES&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Pikachu'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Electric'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;55&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;pydb&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;INSERT&lt;/span&gt; &lt;span class="k"&gt;INTO&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt; &lt;span class="k"&gt;VALUES&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Raichu'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Electric'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;90&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;pydb&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;INSERT&lt;/span&gt; &lt;span class="k"&gt;INTO&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt; &lt;span class="k"&gt;VALUES&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Charmander'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Fire'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;52&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Behind that one line of SQL there's a whole pipeline. A &lt;strong&gt;tokenizer&lt;/strong&gt; chops the text into words (&lt;code&gt;SELECT&lt;/code&gt;, &lt;code&gt;cards&lt;/code&gt;, &lt;code&gt;'Pikachu'&lt;/code&gt;), a &lt;strong&gt;parser&lt;/strong&gt; turns those words into a structure the program understands, and an &lt;strong&gt;executor&lt;/strong&gt; goes and fetches the rows. Each of those is its own small, readable Python file.&lt;/p&gt;

&lt;p&gt;Grouping works too:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;pydb&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="k"&gt;type&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;COUNT&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="k"&gt;AVG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;power&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt; &lt;span class="k"&gt;GROUP&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="k"&gt;type&lt;/span&gt;
&lt;span class="err"&gt;┌──────────┬──────────┬────────────┐&lt;/span&gt;
&lt;span class="err"&gt;│&lt;/span&gt; &lt;span class="k"&gt;type&lt;/span&gt;     &lt;span class="err"&gt;│&lt;/span&gt; &lt;span class="k"&gt;COUNT&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="err"&gt;│&lt;/span&gt; &lt;span class="k"&gt;AVG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;power&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="err"&gt;│&lt;/span&gt;
&lt;span class="err"&gt;├──────────┼──────────┼────────────┤&lt;/span&gt;
&lt;span class="err"&gt;│&lt;/span&gt; &lt;span class="n"&gt;Electric&lt;/span&gt; &lt;span class="err"&gt;│&lt;/span&gt;        &lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="err"&gt;│&lt;/span&gt;       &lt;span class="mi"&gt;72&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="err"&gt;│&lt;/span&gt;
&lt;span class="err"&gt;│&lt;/span&gt; &lt;span class="n"&gt;Fire&lt;/span&gt;     &lt;span class="err"&gt;│&lt;/span&gt;        &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="err"&gt;│&lt;/span&gt;       &lt;span class="mi"&gt;52&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="err"&gt;│&lt;/span&gt;
&lt;span class="err"&gt;└──────────┴──────────┴────────────┘&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The back of the textbook
&lt;/h2&gt;

&lt;p&gt;Here's the question that got my kids interested: if a table has a million rows, how does the database find Pikachu without reading all of them?&lt;/p&gt;

&lt;p&gt;Ask the database how it &lt;em&gt;plans&lt;/em&gt; to find him:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;pydb&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;EXPLAIN&lt;/span&gt; &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt; &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'Pikachu'&lt;/span&gt;
&lt;span class="k"&gt;Full&lt;/span&gt; &lt;span class="k"&gt;table&lt;/span&gt; &lt;span class="n"&gt;scan&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A &lt;strong&gt;full table scan&lt;/strong&gt; means checking every row, one by one. Fine for three cards, painful for a million.&lt;/p&gt;

&lt;p&gt;Now think about a school textbook. You don't read the whole book to find "Fossils". You flip to the &lt;strong&gt;index&lt;/strong&gt; at the back, see "page 55", and jump straight there. A database index is exactly that:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;pydb&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;INDEX&lt;/span&gt; &lt;span class="n"&gt;idx_name&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;pydb&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="k"&gt;EXPLAIN&lt;/span&gt; &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;cards&lt;/span&gt; &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'Pikachu'&lt;/span&gt;
&lt;span class="k"&gt;Index&lt;/span&gt; &lt;span class="n"&gt;lookup&lt;/span&gt; &lt;span class="k"&gt;on&lt;/span&gt; &lt;span class="n"&gt;idx_name&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;...)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Same question, completely different plan. The index in PyDB is a &lt;strong&gt;B-tree&lt;/strong&gt;, which I explain as a magical filing cabinet: each drawer holds a few cards in alphabetical order, and when a drawer gets too full it splits in two and puts a label on the cabinet saying which drawer to open. Because it always stays balanced, you reach any card in a handful of steps, even with millions of them.&lt;/p&gt;

&lt;h2&gt;
  
  
  No take-backs (with a safety net)
&lt;/h2&gt;

&lt;p&gt;The idea my kids liked most was &lt;strong&gt;transactions&lt;/strong&gt;. Imagine trading cards with a friend: "I'll give you my Pikachu if you give me your Charmander." Both halves have to happen, or neither does. You don't want to hand over Pikachu and have your friend run off.&lt;/p&gt;

&lt;p&gt;A transaction groups several changes and promises all of them happen, or none do. If something goes wrong halfway, the database &lt;strong&gt;rolls back&lt;/strong&gt; as if nothing happened. In PyDB, every change in a transaction writes a little note saying how to undo itself, so rolling back means reading the notes backwards.&lt;/p&gt;

&lt;p&gt;Turn on the journal (&lt;code&gt;Database(path, journal=True)&lt;/code&gt;) and it goes one step further: every commit is written to disk &lt;em&gt;before&lt;/em&gt; it counts. Pull the plug mid-write, reopen the database, and it replays the journal. Your committed work is still there, and half-finished work leaves no trace.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's in there
&lt;/h2&gt;

&lt;p&gt;Everything below is real, working code you can step through:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;SQL: &lt;code&gt;SELECT&lt;/code&gt;, &lt;code&gt;INSERT&lt;/code&gt;, &lt;code&gt;UPDATE&lt;/code&gt;, &lt;code&gt;DELETE&lt;/code&gt;, &lt;code&gt;CREATE&lt;/code&gt;/&lt;code&gt;DROP TABLE&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;JOIN&lt;/code&gt;s, &lt;code&gt;GROUP BY&lt;/code&gt; / &lt;code&gt;HAVING&lt;/code&gt;, subqueries and views&lt;/li&gt;
&lt;li&gt;B-tree indexes and a query planner (&lt;code&gt;EXPLAIN&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;PRIMARY KEY&lt;/code&gt;, &lt;code&gt;NOT NULL&lt;/code&gt; and &lt;code&gt;UNIQUE&lt;/code&gt; constraints&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;NULL&lt;/code&gt; with proper three-valued logic (TRUE / FALSE / UNKNOWN)&lt;/li&gt;
&lt;li&gt;Transactions with commit and rollback&lt;/li&gt;
&lt;li&gt;A crash-safe journal with checkpoints&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It's also tested the way I'd test production code: test-first, strictly typed, and with &lt;strong&gt;property-based tests&lt;/strong&gt; (using Hypothesis) that throw hundreds of random inputs at it looking for a rule it breaks. The docs include a page walking through one of those properties and the bug it catches.&lt;/p&gt;

&lt;h2&gt;
  
  
  What building it taught me
&lt;/h2&gt;

&lt;p&gt;The biggest surprise was how much of a "real" database is ordinary code. There's no single clever trick. It's a stack of small, understandable ideas, each solving one problem, and once you've built them yourself the magic turns into engineering. That's exactly what I wanted my kids to see.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it, read it, break it
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Install: &lt;code&gt;pip install fromscratch-pydb&lt;/code&gt;, then run &lt;code&gt;pydb&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Docs, with every concept explained simply: &lt;a href="https://robcox2910.github.io/pydb/" rel="noopener noreferrer"&gt;https://robcox2910.github.io/pydb/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Code: &lt;a href="https://github.com/robcox2910/pydb" rel="noopener noreferrer"&gt;https://github.com/robcox2910/pydb&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;PyDB is one of 18 projects in a series called &lt;strong&gt;From scratch in Python&lt;/strong&gt;. There's also an operating system, a programming language, git, a search engine, a tiny GPT and more, all tied together by PyStack. If you want the lot: &lt;code&gt;pip install fromscratch-pystack&lt;/code&gt;. The series home is &lt;a href="https://robcox2910.github.io/" rel="noopener noreferrer"&gt;https://robcox2910.github.io/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I'd love to hear what's confusing, what's missing, or which system you'd want built next.&lt;/p&gt;

&lt;p&gt;How this was made: I designed PyDB and directed every part of it, but most of the code was written with Claude Code, under strict test-driven development, and this article was drafted with Claude’s help too. Every output shown above comes from actually running it.&lt;/p&gt;

</description>
      <category>python</category>
      <category>learning</category>
      <category>beginners</category>
      <category>database</category>
    </item>
  </channel>
</rss>
