<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Osaf Ali Sayed</title>
    <description>The latest articles on DEV Community by Osaf Ali Sayed (@osafalisayed).</description>
    <link>https://dev.to/osafalisayed</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3725930%2F6d91d90e-58e8-4494-bbc4-48d75e649e7e.jpeg</url>
      <title>DEV Community: Osaf Ali Sayed</title>
      <link>https://dev.to/osafalisayed</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/osafalisayed"/>
    <language>en</language>
    <item>
      <title>From Full-Stack Developer to Tech Lead: Lessons I Learned the Hard Way</title>
      <dc:creator>Osaf Ali Sayed</dc:creator>
      <pubDate>Tue, 28 Jul 2026 04:27:43 +0000</pubDate>
      <link>https://dev.to/osafalisayed/from-full-stack-developer-to-tech-lead-lessons-i-learned-the-hard-way-3ded</link>
      <guid>https://dev.to/osafalisayed/from-full-stack-developer-to-tech-lead-lessons-i-learned-the-hard-way-3ded</guid>
      <description>&lt;p&gt;Stepping into a leadership role can feel surreal. When you’re early in your career, the transition from an individual contributor to a team lead is often unexpected and far more challenging than it looks from the outside.&lt;/p&gt;

&lt;p&gt;My journey into engineering leadership began with confusion, a few painful missteps, and ultimately a powerful realization about what it truly means to own responsibility beyond just writing code.&lt;/p&gt;

&lt;h2&gt;
  
  
  Leadership Is Not Just Coding
&lt;/h2&gt;

&lt;p&gt;I joined Ecomlytics as a full-stack developer alongside another engineer. Almost immediately, my teammate left the company. Overnight, I found myself stepping into a leadership role I hadn’t planned for.&lt;/p&gt;

&lt;p&gt;At first, I did what felt natural: I focused harder on coding. I pushed myself to deliver features, interviewed potential hires, and casually assigned tasks to understand how others worked. In my mind, staying productive as a developer was leadership.&lt;/p&gt;

&lt;p&gt;I was wrong.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Wake-Up Call That Changed Everything
&lt;/h2&gt;

&lt;p&gt;Things started to fall apart when my manager began asking questions like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;
“When do you think this task will be completed?”
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These weren’t questions about implementation details or architecture - they were about timelines, ownership, and accountability. I often didn’t have clear answers, and that resulted in missed deadlines and uncomfortable feedback.&lt;/p&gt;

&lt;p&gt;That moment hit hard. I realized I was no longer evaluated solely on my code. I was now responsible for delivery, clarity, and team progress.&lt;/p&gt;

&lt;h2&gt;
  
  
  Learning to Lead Instead of Just Building
&lt;/h2&gt;

&lt;p&gt;Once I accepted this shift, I consciously changed how I worked. I began embracing responsibilities I had previously avoided:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Regular and timely pull request reviews&lt;/li&gt;
&lt;li&gt;Proactively tracking deadlines and progress&lt;/li&gt;
&lt;li&gt;Running weekly stand-up meetings&lt;/li&gt;
&lt;li&gt;Identifying blockers before they become problems&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;More importantly, I reframed how I saw my role. As a tech lead, my impact wasn’t just measured by lines of code—it was reflected in how effectively my team moved forward.&lt;/p&gt;

&lt;p&gt;I wanted to create an environment where my teammates could rely on me, learn from me, and stay motivated. At the same time, I knew I still had a lot to learn myself.&lt;/p&gt;

&lt;p&gt;To strike the right balance, I intentionally split my time:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;50% on team and task management&lt;/li&gt;
&lt;li&gt;50% on technical responsibilities like improving infrastructure, maintaining production systems, ensuring code quality, and reviewing pull requests&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Plot Twist: Leadership Can Make You Lose Your Coding Edge&lt;/p&gt;

&lt;p&gt;Here’s something no one warned me about - when you step into engineering leadership early, your hands-on coding skills can start to fade.&lt;/p&gt;

&lt;p&gt;I noticed it happening. I was thinking more, typing less, and relying heavily on AI tools. While that improved productivity, it also triggered a quiet fear: Am I losing touch with real coding?&lt;/p&gt;

&lt;p&gt;So I built a few habits to stay sharp.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Staying Technically Curious
&lt;/h3&gt;

&lt;p&gt;I installed &lt;a href="https://daily.dev" rel="noopener noreferrer"&gt;daily.dev&lt;/a&gt; and made it a habit to read about backend engineering, system design, scalability, big data, and generative AI. These are areas I actively use - or plan to use - in real production systems.&lt;/p&gt;

&lt;p&gt;P.S originally it was only &lt;a href="https://daily.dev" rel="noopener noreferrer"&gt;daily.dev&lt;/a&gt;. now it is (dev.to)[&lt;a href="https://dev.to"&gt;https://dev.to&lt;/a&gt;] as well!&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Coding Outside of Work
&lt;/h3&gt;

&lt;p&gt;To fight developer FOMO, I spend at least 30 minutes a day coding on personal projects—manually. No shortcuts. This keeps my muscle memory alive and reminds me why I fell in love with building software in the first place.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Getting My Hands Dirty at Work
&lt;/h3&gt;

&lt;p&gt;Whenever possible, I pick up small coding tasks within my team. It helps me stay connected to the codebase and better understand the challenges my teammates face daily.&lt;/p&gt;

&lt;p&gt;Looking back, this transition taught me that tech leadership isn’t about authority - it’s about ownership. It’s about being accountable for people, progress, and outcomes.&lt;/p&gt;

&lt;p&gt;I feel genuinely grateful to work with a team of incredibly talented developers and to be building something that creates real business impact. The journey from full-stack developer to tech lead hasn’t been smooth - but it’s been deeply rewarding.&lt;/p&gt;

&lt;p&gt;And this is just the beginning.&lt;/p&gt;

</description>
      <category>management</category>
      <category>teamlead</category>
      <category>development</category>
    </item>
    <item>
      <title>How I Made My First Contribution at Frappe</title>
      <dc:creator>Osaf Ali Sayed</dc:creator>
      <pubDate>Sat, 25 Jul 2026 10:27:12 +0000</pubDate>
      <link>https://dev.to/osafalisayed/how-i-made-my-first-contribution-at-frappe-3pip</link>
      <guid>https://dev.to/osafalisayed/how-i-made-my-first-contribution-at-frappe-3pip</guid>
      <description>&lt;p&gt;Contributing to open source has always been a goal of mine, but finding the right starting point was challenging. During my college years, despite learning development, I lacked the courage to make an open source contribution.ived valuable feedback on my pull request. My foundational knowledge of the Frappe framework and Vue.JS enabled me to implement the necessary changes smoothly.&lt;/p&gt;

&lt;p&gt;Recently, I joined&amp;nbsp;&lt;a href="https://zenduty.com/" rel="noopener noreferrer"&gt;Zenduty&lt;/a&gt;, a startup specializing in IT Service Management. At Zenduty, we heavily utilize Python for backend development, and JavaScript, React.JS, and Vite.JS for frontend work. This experience exposed me to various technologies like Docker, Docker Compose, and GitHub Actions, significantly boosting my confidence.&lt;/p&gt;

&lt;p&gt;Motivated by this newfound confidence, I decided to make weekly open source contributions to various platforms.&lt;/p&gt;

&lt;h2&gt;
  
  
  &lt;a href="https://www.osafalisayed.com/how-i-made-my-first-contribution-frappe#heading-discovering-frappe" rel="noopener noreferrer"&gt;&lt;/a&gt;Discovering Frappe
&lt;/h2&gt;

&lt;p&gt;With a strong background in Django and DRF for backend, and React for frontend, I was ready for a new challenge. While searching for open source projects, a friend introduced me to Frappe. Their organization hosts a wealth of content, including&amp;nbsp;&lt;a href="https://github.com/frappe/erpnext/" rel="noopener noreferrer"&gt;ERPNext&lt;/a&gt;, a&amp;nbsp;&lt;a href="http://github.com/frappe/lms" rel="noopener noreferrer"&gt;Learning Management System&lt;/a&gt;, and&amp;nbsp;&lt;a href="https://github.com/frappe/bench" rel="noopener noreferrer"&gt;Bench&lt;/a&gt;, a terminal tool for interconnecting everything. The interconnected nature of these projects fascinated me.&lt;/p&gt;

&lt;p&gt;Frappe primarily uses Python for backend development and their own Frappe framework, with Vue.JS for frontend. This meant I needed to familiarize myself with their framework before contributing. I began by exploring the "good first issues" in their repository and found a task to create a related courses component. A previous attempt at this task had been made but was incomplete.&lt;/p&gt;

&lt;h3&gt;
  
  
  &lt;a href="https://www.osafalisayed.com/how-i-made-my-first-contribution-frappe#heading-diving-into-frappe-and-vuejs" rel="noopener noreferrer"&gt;&lt;/a&gt;Diving into Frappe and Vue.JS
&lt;/h3&gt;

&lt;p&gt;I embarked on understanding how the Frappe backend operates and how Vue.JS differs from React.JS, particularly in component loading. Frappe's approach to database models and API requests is distinct from Django, with migrations managed through Bench.&lt;/p&gt;

&lt;p&gt;With guidance from&amp;nbsp;&lt;a href="https://www.linkedin.com/in/jannat-patel/" rel="noopener noreferrer"&gt;Jannat Patel&lt;/a&gt;, a product engineer at Frappe, I received valuable feedback on my pull request. My foundational knowledge of the Frappe framework and Vue.JS enabled me to implement the necessary changes smoothly.&lt;/p&gt;

&lt;h3&gt;
  
  
  &lt;a href="https://www.osafalisayed.com/how-i-made-my-first-contribution-frappe#heading-looking-ahead" rel="noopener noreferrer"&gt;&lt;/a&gt;Looking Ahead
&lt;/h3&gt;

&lt;p&gt;I am now focused on submitting more pull requests to Frappe. My ultimate goal is to propose a feature for their Learning Management System. Achieving this would not only be a personal milestone but also contribute to a project that benefits others.&lt;/p&gt;

&lt;p&gt;For those of you who are interested in the PR and the related issue you can find it&amp;nbsp;&lt;a href="https://github.com/frappe/lms/pull/1565" rel="noopener noreferrer"&gt;here&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>vue</category>
      <category>python</category>
      <category>frappe</category>
      <category>lms</category>
    </item>
    <item>
      <title>I Built My Own Load Balancer In Python Without AI</title>
      <dc:creator>Osaf Ali Sayed</dc:creator>
      <pubDate>Thu, 23 Jul 2026 06:35:56 +0000</pubDate>
      <link>https://dev.to/osafalisayed/i-built-my-own-load-balancer-in-python-without-ai-2f15</link>
      <guid>https://dev.to/osafalisayed/i-built-my-own-load-balancer-in-python-without-ai-2f15</guid>
      <description>&lt;p&gt;In the age of AI, nothing is black and white anymore.&lt;/p&gt;

&lt;p&gt;Everyone on the internet has an opinion, and everyone shares it. AI is going to take your job. No, AI won't take your job. Junior roles are done for, but senior engineers are exactly who companies will fight over. Pick any thread on any given day and you'll find all of these being argued at once, usually with equal confidence.&lt;/p&gt;

&lt;p&gt;This is my website, so here's mine.&lt;/p&gt;

&lt;p&gt;I think the age of AI is great. We ship fast. We build faster. But the part I keep coming back to is this: it's the one &lt;em&gt;skilled&lt;/em&gt; developer who's quietly making absurd money right now. The developer who persisted, who has years of real experience, who can craft a production-ready system before writing a line — that person just got a 4x multiplier on their output. AI didn't replace them. It handed them a bigger lever.&lt;/p&gt;

&lt;p&gt;Which is slightly uncomfortable, because it means the moat isn't the typing. It's the judgment. And judgment is the one thing you can't prompt your way into overnight.&lt;/p&gt;

&lt;p&gt;So I decided to do something about it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem with letting AI drive
&lt;/h2&gt;

&lt;p&gt;When you let AI write everything, something quiet happens. You stop making the small decisions. You stop feeling the friction of "wait, how &lt;em&gt;should&lt;/em&gt; these two things talk to each other?" You get output, skim it, accept it, move on. It works. It ships. And your own architectural muscle goes soft without you noticing.&lt;/p&gt;

&lt;p&gt;I didn't want to wake up in five years and find out I'm a vibe coder.&lt;/p&gt;

&lt;p&gt;So I picked something I've read about a hundred times in system design material but had never built with my own hands: a &lt;strong&gt;load balancer&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Do we need to write our own? No. Should you run a hand-rolled one in production? Absolutely not — use nginx, HAProxy, or whatever your cloud hands you. But should you build one &lt;em&gt;for learning&lt;/em&gt;? A hundred percent. It's small enough to finish and deep enough to teach you something real about how traffic actually gets spread across servers.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Python, of all things
&lt;/h2&gt;

&lt;p&gt;I wrote this in Python mostly because I knew nobody writes a load balancer in Python. 😂 I found a repo and an article walking through one in Go, and Go is arguably the "correct" choice here. But I didn't want correct. I wanted to spin something up in a language I'm comfortable in so the language never got in the way of the thinking. The whole point was to wrestle with the &lt;em&gt;ideas&lt;/em&gt;, not fight a borrowed syntax I'd have to look up every ten seconds.&lt;/p&gt;

&lt;h2&gt;
  
  
  The rules I set for myself
&lt;/h2&gt;

&lt;p&gt;To make this worth anything, I set two ground rules and stuck to them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;No AI anywhere in the actual code.&lt;/strong&gt; No matter how ugly it got, no matter how badly I wanted to just ask, I wrote every line by hand. I gave it five hours a week. Honestly, I think every freelancer and every developer in their growth phase should carve out five hours a week for their &lt;em&gt;core skill&lt;/em&gt; — no client work, no shortcuts, just you and the fundamentals.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Google and AI are for lookups only.&lt;/strong&gt; If I forget the exact syntax for, say, a class-level variable versus an instance one in Python, I'm allowed to look it up. That's just remembering vocabulary. What I'm &lt;em&gt;not&lt;/em&gt; allowed to do is ask for architecture, logic, or "how should I structure this." The thinking has to be mine. The syntax can come from anywhere.&lt;/p&gt;

&lt;p&gt;That line between &lt;em&gt;looking up a word&lt;/em&gt; and &lt;em&gt;outsourcing the thinking&lt;/em&gt; turned out to be the whole point of the exercise.&lt;/p&gt;

&lt;h2&gt;
  
  
  Scope of the project
&lt;/h2&gt;

&lt;p&gt;For this project I did not want to implement something crazy and I wanted to wrap things up in 5 hours of coding session. So I limited the goals for this project:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;A simple load balancer.&lt;/strong&gt; It should be able to balance the load between the server using an algorithm in this case it will be &lt;strong&gt;round robin&lt;/strong&gt; algorithm. we can replace such things into separate files later on to add more algorithms but that is out of scope for today's session.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Health Checks.&lt;/strong&gt; One day you are sleeping and you realize that a server is down oh no! oh my god! what to do now! and you are using Osaf Ali Sayed's very own production-ready load balancer. nobody would want their load balancer to crash just because a server crashed do they?🙂 We need to make sure if a server goes down, our load balancer can detect it and handle it properly. for this we will write a thread which will run along side the load balancer, it will keep checking the /health endpoint on all the servers. if a server is down we do not send request to that server.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Server Death Checks.&lt;/strong&gt; If by chance we hit the server before the health check was able to mark it dead. we need to detect this during the request to make the system more robust. Ideally here we would redirect to another server if request failed but for now this is out of scope.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  What actually happened
&lt;/h2&gt;

&lt;p&gt;It took me five hours. I wrote &lt;strong&gt;200 lines of code&lt;/strong&gt;. That's it.&lt;/p&gt;

&lt;p&gt;And it was an eye-opener.&lt;/p&gt;

&lt;p&gt;There's a specific feeling you get when you build something entirely yourself, where you understand every line because you had to fight for it. I hadn't felt that in a while. Somewhere in the AI-assisted daily grind, I'd traded it away for speed without realizing I'd made the trade.&lt;/p&gt;

&lt;p&gt;For the balancing, I went with &lt;strong&gt;round robin&lt;/strong&gt;  the simplest, most classic load-balancing algorithm there is. You keep a list of your backend servers, and each incoming request goes to the next server in line, cycling back to the start when you reach the end. Request one to server A, request two to B, request three to C, request four back to A. No weights, no cleverness. Just fair, even rotation.&lt;/p&gt;

&lt;p&gt;Here's the round-robin selection:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;BaseHTTPRequestHandler&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;

    &lt;span class="n"&gt;counter&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
    &lt;span class="n"&gt;lock&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;threading&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Lock&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;server_state&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;servers&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

    &lt;span class="c1"&gt;# ... inside do_GET, under the lock ...
&lt;/span&gt;    &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;lock&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;checked&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
        &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;checked&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;servers&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
                &lt;span class="c1"&gt;# every server is dead
&lt;/span&gt;                &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send_response&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;502&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;end_headers&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
                &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;wfile&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;b&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;error&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;all servers are dead&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt;

            &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;counter&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;counter&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;%&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;servers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;server_state&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;counter&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
                &lt;span class="n"&gt;choosen_index&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;counter&lt;/span&gt;
                &lt;span class="k"&gt;break&lt;/span&gt;

            &lt;span class="n"&gt;checked&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;

        &lt;span class="n"&gt;server_url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;servers&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;choosen_index&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Then a server died, and things got interesting
&lt;/h2&gt;

&lt;p&gt;Plain round robin has an obvious flaw: it doesn't know or care whether a server is actually alive. If server B falls over, round robin will happily keep sending it every third request, and every one of those requests just dies.&lt;/p&gt;

&lt;p&gt;So I added &lt;strong&gt;health checks&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The idea is simple. A background thread runs on a loop, and every few seconds it pings a &lt;code&gt;/health&lt;/code&gt; endpoint on each backend. If a server answers with a 200, it's marked alive. If it refuses the connection or answers with anything else, it gets marked dead. The router then skips any server currently marked dead when it's picking where to send the next request.&lt;/p&gt;

&lt;p&gt;The nice part is that this cuts both ways. A server going down isn't a permanent death sentence. The health-check loop keeps pinging it anyway, so the moment it comes back up and answers a 200 again, it gets flipped back to alive and starts receiving traffic like nothing happened. No restart, no manual intervention. The balancer just notices and adjusts.&lt;/p&gt;

&lt;p&gt;Here's the health-check loop:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;health_check_loop&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;backend&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;servers&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
            &lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;backend&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;outgoing_headers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Host&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;backend&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

            &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;conn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;HTTPConnection&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
                &lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GET&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;/health&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;outgoing_headers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="n"&gt;resp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getresponse&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

                &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                    &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;server_state&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;
                &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                    &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;server_state&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;

            &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;ConnectionRefusedError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;server_state&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;

        &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="c1"&gt;# started as a background daemon thread so it runs alongside the server
&lt;/span&gt;&lt;span class="n"&gt;t&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;threading&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Thread&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;target&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;health_check_loop&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;daemon&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;t&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;start&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And the request handler that picks a live server and forwards the request:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;do_GET&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;path&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;
    &lt;span class="n"&gt;headers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;
    &lt;span class="n"&gt;outgoing_headers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# only the tiny critical section is locked: read + bump the shared
&lt;/span&gt;    &lt;span class="c1"&gt;# counter and pick a live server. everything slow stays outside.
&lt;/span&gt;    &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;lock&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;checked&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
        &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;checked&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;servers&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
                &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send_response&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;502&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;end_headers&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
                &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;wfile&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;b&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;error&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;all servers are dead&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt;

            &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;counter&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;counter&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;%&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;servers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;server_state&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;counter&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
                &lt;span class="n"&gt;choosen_index&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;counter&lt;/span&gt;
                &lt;span class="k"&gt;break&lt;/span&gt;

            &lt;span class="n"&gt;checked&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;

        &lt;span class="n"&gt;server_url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;servers&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;choosen_index&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

    &lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;server_url&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;outgoing_headers&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Host&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;server_url&lt;/span&gt;
    &lt;span class="n"&gt;outgoing_headers&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;X-Forwarded-For&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;client_address&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

    &lt;span class="c1"&gt;# forwarding happens outside the lock, so requests run in parallel
&lt;/span&gt;    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;conn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;HTTPConnection&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;GET&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;outgoing_headers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;resp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getresponse&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

        &lt;span class="n"&gt;status&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt;
        &lt;span class="n"&gt;body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send_response&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;status&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;end_headers&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;wfile&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;ConnectionRefusedError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="c1"&gt;# server died mid-request: mark it dead and report failure
&lt;/span&gt;        &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;server_state&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;choosen_index&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send_response&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;503&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;end_headers&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;wfile&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;b&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;error&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;:&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;this server is dead!&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="n"&gt;server&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ThreadingHTTPServer&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;127.0.0.1&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;8080&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;ProxyHandler&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;server&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;serve_forever&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;There's also a nice fallback baked in: if the handler walks the whole list and every server is marked dead, it doesn't just hang or crash. It returns a 502 saying all servers are down. And if a request gets routed to a server that dies &lt;em&gt;right at that moment&lt;/em&gt;, the handler catches the refused connection, marks that server dead on the spot, and returns a 503 instead of silently failing.&lt;/p&gt;

&lt;h2&gt;
  
  
  The part that genuinely humbled me: threading
&lt;/h2&gt;

&lt;p&gt;Here's what I did not expect to be the hard part. Threading is &lt;em&gt;brutal&lt;/em&gt; to reason about.&lt;/p&gt;

&lt;p&gt;A load balancer has to handle multiple requests at once, that's the entire job. So it's inherently concurrent, which means multiple threads are touching the same shared counter that tracks whose turn it is. And the second you have multiple threads reading and writing the same variable, you're one bad interleaving away from two requests grabbing the same "next" server, or the counter skipping, or worse.&lt;/p&gt;

&lt;p&gt;This is the thing we all studied in college and mostly nodded along to: the &lt;strong&gt;race condition&lt;/strong&gt;. It's very different to actually hit one in code you wrote yourself.&lt;/p&gt;

&lt;p&gt;The fix is a lock. But the real lesson wasn't &lt;em&gt;that&lt;/em&gt; I needed a lock, it was &lt;em&gt;where&lt;/em&gt;. My first instinct was to wrap everything in the lock and move on. That "works," but it's stupid: it makes the whole thing synchronous. One request at a time. You've just built a load balancer that can't handle load, which defeats the entire purpose.&lt;/p&gt;

&lt;p&gt;So I had to actually think about it. I ended up repositioning my code so the lock only wraps the tiny critical section, the part that reads and bumps the shared counter and picks the index. Everything else, the slow part, the actual forwarding of the request to the backend, happens &lt;em&gt;outside&lt;/em&gt; the lock, so requests can genuinely run in parallel. Getting that boundary right meant understanding the &lt;em&gt;context&lt;/em&gt; of my own code: which few lines truly can't be touched by two threads at once, and which lines are fine to run concurrently.&lt;/p&gt;

&lt;p&gt;That was the moment the whole exercise paid for itself. Not the load balancer. The forced understanding of exactly where concurrency bites.&lt;/p&gt;

&lt;p&gt;AI is a lever. A huge one. But a lever does nothing without someone who knows where to put it. The developers who'll thrive aren't the ones who prompt the fastest they're the ones who still understand the machine well enough to catch it when it's wrong.&lt;/p&gt;

&lt;p&gt;So build the small thing by hand. Kill the autocomplete for a few hours. Feel the friction on purpose. Five hours a week is nothing against a career, and it's the cheapest insurance you can buy against becoming a vibe coder.&lt;/p&gt;

&lt;p&gt;Here is the github repo if you want to see one file pulled together: &lt;a href="https://github.com/OsafAliSayed/load-balancer" rel="noopener noreferrer"&gt;github.com/OsafAliSayed/load-balancer&lt;/a&gt;&lt;/p&gt;

</description>
      <category>loadbalancer</category>
      <category>roundrobin</category>
      <category>buildyourownx</category>
      <category>healthchecks</category>
    </item>
  </channel>
</rss>
