<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Azam Akram</title>
    <description>The latest articles on DEV Community by Azam Akram (@azam-akram).</description>
    <link>https://dev.to/azam-akram</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3974919%2Fe4b398b8-3571-4282-b0f4-2d73a6e38962.jpg</url>
      <title>DEV Community: Azam Akram</title>
      <link>https://dev.to/azam-akram</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/azam-akram"/>
    <language>en</language>
    <item>
      <title>Gzip Base64 encoder / decoder</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Sun, 28 Jun 2026 15:44:55 +0000</pubDate>
      <link>https://dev.to/azam-akram/gzip-base64-encoder-decoder-12hc</link>
      <guid>https://dev.to/azam-akram/gzip-base64-encoder-decoder-12hc</guid>
      <description>&lt;p&gt;Compress text with Gzip and encode the result as Base64 in a single step — or go the other way and decode then decompress a Base64 payload back to the original string. Everything runs in your browser.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.solutiontoolkit.com/tools/gzip-base64-encoder-decoder" rel="noopener noreferrer"&gt;https://www.solutiontoolkit.com/tools/gzip-base64-encoder-decoder&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  When is this useful?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;API payloads: some services accept compressed, Base64-encoded bodies to reduce bandwidth. Decode them here to inspect the original JSON or XML.&lt;/li&gt;
&lt;li&gt;Environment variables: large config blobs are often Gzip+Base64 encoded before being stored as env vars or Kubernetes secrets.&lt;/li&gt;
&lt;li&gt;CloudWatch / logging: AWS Lambda logs shipped via Kinesis are Gzip+Base64 encoded. Paste the payload here to read the raw log lines.&lt;/li&gt;
&lt;li&gt;Cookie compression: session cookies containing JSON are sometimes compressed to stay under browser size limits.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  How it works
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Compress &amp;amp; Encode: runs Gzip compression on your text using the pako library, then encodes the binary output as a Base64 string safe for transport in JSON, URLs, and headers.&lt;/li&gt;
&lt;li&gt;Decompress &amp;amp; Decode: validates the Base64 format, decodes the bytes, then runs Gzip inflate to recover the original text.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Short strings often produce a larger Base64 output than the original because Gzip's header and Base64 expansion (x1.33) outweigh the compression gain. Gzip shines on repetitive text over ~1 KB.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to Use This Gzip Base64 Decoder and Encoder
&lt;/h2&gt;

&lt;p&gt;gzip base64 decode is designed for quick browser-based work when you need to gzip compress text. Paste or select your input in the tool area above, run the conversion or formatting step, then review the result before copying it into code, documentation, tickets, test data, or an API client.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Example input:&lt;/strong&gt;&lt;br&gt;
&lt;code&gt;Gzip-compressed Base64 API payload&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Example output:&lt;/strong&gt;&lt;br&gt;
&lt;code&gt;H4sIAAAAAAAAA3OvyizQTc7PLShKLS5OTVFwSixONTNRcAzwVChIrMzJT0wBAOnU+sEiAAAA&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Common Issues to Check&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A normal Base64 string will not decompress unless the decoded bytes are gzip data&lt;/li&gt;
&lt;li&gt;Copied payloads can fail when quotes, escaping, or whitespace are included&lt;/li&gt;
&lt;li&gt;Very large payloads can take longer because compression happens in the browser&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>gzip</category>
      <category>webdev</category>
      <category>base64</category>
      <category>encode</category>
    </item>
    <item>
      <title>Stop rewriting configs by hand. Paste JSON, get YAML (or the other way around) in one click. 

https://www.solutiontoolkit.com/tools/json-yaml-converter

#json #yaml</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Sun, 28 Jun 2026 15:36:03 +0000</pubDate>
      <link>https://dev.to/azam-akram/stop-rewriting-configs-by-hand-paste-json-get-yaml-or-the-other-way-around-in-one-click-407a</link>
      <guid>https://dev.to/azam-akram/stop-rewriting-configs-by-hand-paste-json-get-yaml-or-the-other-way-around-in-one-click-407a</guid>
      <description>&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
      &lt;div class="c-embed__body flex items-center justify-between"&gt;
        &lt;a href="https://www.solutiontoolkit.com/tools/json-yaml-converter" rel="noopener noreferrer" class="c-link fw-bold flex items-center"&gt;
          &lt;span class="mr-2"&gt;solutiontoolkit.com&lt;/span&gt;
          

        &lt;/a&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


</description>
    </item>
    <item>
      <title>Your API wants JSON. Your config is YAML. Convert it here, no signup needed. https://www.solutiontoolkit.com/tools/yaml-to-json-converter
#json #yaml</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Sat, 27 Jun 2026 23:42:12 +0000</pubDate>
      <link>https://dev.to/azam-akram/your-api-wants-json-your-config-is-yaml-convert-it-here-no-signup-needed-3n9m</link>
      <guid>https://dev.to/azam-akram/your-api-wants-json-your-config-is-yaml-convert-it-here-no-signup-needed-3n9m</guid>
      <description>&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
      &lt;div class="c-embed__body flex items-center justify-between"&gt;
        &lt;a href="https://www.solutiontoolkit.com/tools/yaml-to-json-converter" rel="noopener noreferrer" class="c-link fw-bold flex items-center"&gt;
          &lt;span class="mr-2"&gt;solutiontoolkit.com&lt;/span&gt;
          

        &lt;/a&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


</description>
    </item>
    <item>
      <title>Stop rewriting configs by hand. Paste JSON, get YAML (or the other way around) in one click. https://www.solutiontoolkit.com/tools/json-yaml-converter
#json #yaml #converter</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Sat, 27 Jun 2026 23:40:03 +0000</pubDate>
      <link>https://dev.to/azam-akram/stop-rewriting-configs-by-hand-paste-json-get-yaml-or-the-other-way-around-in-one-click-31d</link>
      <guid>https://dev.to/azam-akram/stop-rewriting-configs-by-hand-paste-json-get-yaml-or-the-other-way-around-in-one-click-31d</guid>
      <description>&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
      &lt;div class="c-embed__body flex items-center justify-between"&gt;
        &lt;a href="https://www.solutiontoolkit.com/tools/json-yaml-converter" rel="noopener noreferrer" class="c-link fw-bold flex items-center"&gt;
          &lt;span class="mr-2"&gt;solutiontoolkit.com&lt;/span&gt;
          

        &lt;/a&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


</description>
    </item>
    <item>
      <title>Tired of pasting JSON into random sites? Here's a free in-browser validator - nothing leaves your machine. 
https://www.solutiontoolkit.com/tools/validate-and-prettify-json
#json #pretty #validate</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Sat, 27 Jun 2026 23:37:58 +0000</pubDate>
      <link>https://dev.to/azam-akram/tired-of-pasting-json-into-random-sites-heres-a-free-in-browser-validator-nothing-leaves-your-1fc6</link>
      <guid>https://dev.to/azam-akram/tired-of-pasting-json-into-random-sites-heres-a-free-in-browser-validator-nothing-leaves-your-1fc6</guid>
      <description>&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
      &lt;div class="c-embed__body flex items-center justify-between"&gt;
        &lt;a href="https://www.solutiontoolkit.com/tools/validate-and-prettify-json" rel="noopener noreferrer" class="c-link fw-bold flex items-center"&gt;
          &lt;span class="mr-2"&gt;solutiontoolkit.com&lt;/span&gt;
          

        &lt;/a&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


</description>
    </item>
    <item>
      <title>Check live scores, full group stage schedule, and knockout fixtures all in one place: 
👉 https://www.solutiontoolkit.com/world-cup-2026/schedule
#WorldCup2026 #FIFA2026 #NextJS #SolutionToolkit #WebDev #Football</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Thu, 25 Jun 2026 23:18:46 +0000</pubDate>
      <link>https://dev.to/azam-akram/check-live-scores-full-group-stage-schedule-and-knockout-fixtures-all-in-one-place-1jb8</link>
      <guid>https://dev.to/azam-akram/check-live-scores-full-group-stage-schedule-and-knockout-fixtures-all-in-one-place-1jb8</guid>
      <description>&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
      &lt;div class="c-embed__body flex items-center justify-between"&gt;
        &lt;a href="https://www.solutiontoolkit.com/world-cup-2026/schedule" rel="noopener noreferrer" class="c-link fw-bold flex items-center"&gt;
          &lt;span class="mr-2"&gt;solutiontoolkit.com&lt;/span&gt;
          

        &lt;/a&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


</description>
    </item>
    <item>
      <title>Debugging GZip Base64 compressed payloads from AWS API Gateway / Lambda</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Mon, 22 Jun 2026 17:40:12 +0000</pubDate>
      <link>https://dev.to/azam-akram/debugging-gzip-base64-compressed-payloads-from-aws-api-gateway-lambda-3ehm</link>
      <guid>https://dev.to/azam-akram/debugging-gzip-base64-compressed-payloads-from-aws-api-gateway-lambda-3ehm</guid>
      <description>&lt;p&gt;You call an API. The response comes back and one of the fields looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;H4sIAAAAAAAAE6tWKkktLlGyUlIqS04sLknMSwUAAAD//wMAAAD//w==
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you have ever looked at something like that wondering what is hiding inside, you are looking at a GZip-compressed payload encoded in Base64.&lt;/p&gt;

&lt;p&gt;This pattern appears more often than you might expect — in AWS, Azure, Kafka, webhook systems, and internal microservices. Understanding why it exists and how to quickly decode it will save you time every time you hit it.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Just need to decode one now?&lt;/strong&gt; Paste the string into the &lt;a href="https://www.solutiontoolkit.com/tools/gzip-base64-encoder-decoder" rel="noopener noreferrer"&gt;GZip Base64 Encoder / Decoder&lt;/a&gt; and get the result instantly — no terminal required.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  The Problem: Binary Data in Text Channels
&lt;/h2&gt;

&lt;p&gt;GZip compression produces binary output — raw bytes that can include any value from 0 to 255. Most text-based protocols (HTTP headers, JSON fields, environment variables, message queues) are designed for printable characters. If you try to embed raw binary bytes into a JSON string, you will get parsing errors or silent data corruption.&lt;/p&gt;

&lt;p&gt;Base64 solves this by converting binary data into a safe alphabet of 64 printable ASCII characters. The trade-off is a roughly 33% increase in size — but after GZip compression, the total payload is still significantly smaller than the original.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why APIs Combine GZip and Base64
&lt;/h2&gt;

&lt;p&gt;The two techniques solve different problems and work well together:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;GZip&lt;/strong&gt; reduces size. For repetitive, structured data like JSON or XML, GZip typically achieves 60–80% compression. A 100 KB JSON payload can compress to 15–20 KB.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Base64&lt;/strong&gt; makes binary data safe to transport. It guarantees the compressed bytes can be embedded in a JSON field, stored in a database column, passed through an environment variable, or included in a URL query string without corruption.&lt;/p&gt;

&lt;p&gt;Together they allow a system to transmit large, structured payloads efficiently through channels that only support text.&lt;/p&gt;

&lt;h3&gt;
  
  
  Real-World Examples
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;AWS Lambda and API Gateway&lt;/strong&gt; — response bodies larger than a certain threshold can be returned as Base64-encoded compressed content, with a flag telling the client to decompress&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Kafka messages&lt;/strong&gt; — event payloads are often GZip + Base64 encoded before publishing so they fit within message size limits and serialize cleanly&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Webhook events&lt;/strong&gt; — third-party services sometimes send compressed event data to reduce egress costs&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Distributed tracing / logging&lt;/strong&gt; — trace context or structured log data embedded in HTTP headers&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  How to Debug a Compressed Payload
&lt;/h2&gt;

&lt;p&gt;When you encounter one of these strings during development or debugging, the workflow is straightforward:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Copy the encoded string from the API response, log line, or message payload&lt;/li&gt;
&lt;li&gt;Decode the Base64 layer to recover the GZip binary&lt;/li&gt;
&lt;li&gt;Decompress the GZip to recover the original data&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In a terminal you can do this with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"H4sIAAAAAAAAE6tWKkktLlGyUlIqS04sLknMSwUAAAD//wMAAAD//w=="&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | &lt;span class="nb"&gt;base64&lt;/span&gt; &lt;span class="nt"&gt;--decode&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | &lt;span class="nb"&gt;gunzip&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That works fine in a Linux or macOS shell. On Windows, or when you just want a quick result without opening a terminal, an online tool is faster.&lt;/p&gt;

&lt;h2&gt;
  
  
  Decode It Instantly
&lt;/h2&gt;

&lt;p&gt;The &lt;a href="https://www.solutiontoolkit.com/tools/gzip-base64-encoder-decoder" rel="noopener noreferrer"&gt;GZip Base64 Encoder / Decoder&lt;/a&gt; tool on this site handles both directions — paste your encoded string to decode it, or paste plain text and JSON to compress and encode it. No installation, no terminal, works in the browser with no data leaving your machine.&lt;/p&gt;

&lt;p&gt;It is useful when you need to inspect a payload quickly during an incident, verify what a service is actually sending before writing decoding logic in your application, or compress test data to embed in a config or environment variable.&lt;/p&gt;

&lt;p&gt;The next time you hit an unreadable string in an API response, you will know exactly what it is and how to open it up.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>devops</category>
      <category>api</category>
      <category>aws</category>
    </item>
    <item>
      <title>Generate MD5, SHA-1, SHA-256, and SHA-512 Hashes Instantly in Your Browser</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Thu, 18 Jun 2026 17:32:40 +0000</pubDate>
      <link>https://dev.to/azam-akram/generate-md5-sha-1-sha-256-and-sha-512-hashes-instantly-in-your-browser-2o10</link>
      <guid>https://dev.to/azam-akram/generate-md5-sha-1-sha-256-and-sha-512-hashes-instantly-in-your-browser-2o10</guid>
      <description>&lt;p&gt;A free, client-side tool for hashing text — no installation, no account, no server.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Is a Hash and When Do You Need One?
&lt;/h2&gt;

&lt;p&gt;A hash function takes any input — a word, a sentence, a JSON blob — and produces a fixed-length string of characters. The same input always produces the same output. Change even a single character and the output changes completely.&lt;/p&gt;

&lt;p&gt;Developers reach for hashing in a surprisingly wide range of situations:&lt;/p&gt;

&lt;p&gt;Verifying data integrity — confirm a file or string has not been tampered with&lt;br&gt;
Checksums — compare two pieces of data without storing or transmitting the originals&lt;br&gt;
Password storage — store a hash instead of a plaintext password (with a proper algorithm)&lt;br&gt;
Caching and deduplication — use a hash as a unique key for a piece of content&lt;br&gt;
Debugging — quickly fingerprint a string to check if two values are identical across environments&lt;br&gt;
The &lt;a href="https://www.solutiontoolkit.com/tools/text-hash-generator" rel="noopener noreferrer"&gt;Text Hash Generator&lt;/a&gt; lets you generate hashes across all four commonly used algorithms in one step, directly in your browser.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Four Algorithms
&lt;/h2&gt;

&lt;p&gt;MD5–128-bit MD5 is fast and widely recognised. It is no longer considered cryptographically secure for security-sensitive use cases, but remains practical for checksums, non-security fingerprinting, and working with legacy systems that expect MD5 output.&lt;/p&gt;

&lt;p&gt;SHA-1–160-bit SHA-1 is similarly deprecated for security use but still appears in older protocols, version control systems, and certificate chains. Useful when you need to match or verify a SHA-1 value produced by another system.&lt;/p&gt;

&lt;p&gt;SHA-256–256-bit SHA-256 is the workhorse of modern cryptography. It is part of the SHA-2 family and is widely used in TLS certificates, code signing, blockchain, and general-purpose integrity verification. When in doubt, use SHA-256.&lt;/p&gt;

&lt;p&gt;SHA-512–512-bit SHA-512 produces a longer digest and offers a higher security margin. It is preferred in high-security contexts and, on 64-bit processors, can actually be faster than SHA-256 due to how modern CPUs handle 64-bit operations.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the Tool Does
&lt;/h2&gt;

&lt;p&gt;Open the &lt;a href="https://www.solutiontoolkit.com/tools/text-hash-generator" rel="noopener noreferrer"&gt;Text Hash Generator&lt;/a&gt;, type or paste any text, and click Generate Hashes (or press Ctrl+Enter). All four hashes are computed and displayed simultaneously.&lt;/p&gt;

&lt;p&gt;Uppercase toggle Switch between lowercase and uppercase hex output with a single checkbox. Some systems expect uppercase hash strings — this saves you a manual conversion step.&lt;/p&gt;

&lt;p&gt;Copy individual or all Each hash has its own Copy button. There is also a Copy all hashes option that puts all four results on the clipboard in a labelled format, ready to paste into documentation or a ticket.&lt;/p&gt;

&lt;p&gt;Character count The input field shows a live character count as you type, useful when you need to know the exact length of the string you are hashing.&lt;/p&gt;

&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;p&gt;SHA-1, SHA-256, and SHA-512 are computed using the browser’s built-in Web Crypto API via crypto.subtle.digest. MD5 is handled by the js-md5 library, since MD5 is not included in the Web Crypto API (intentionally — browsers do not endorse it for cryptographic use). All four run client-side. Open DevTools, go to the Network tab, type some text, and generate — you will see zero outbound requests.&lt;/p&gt;

&lt;p&gt;Try It&lt;br&gt;
Open the &lt;a href="https://www.solutiontoolkit.com/tools/text-hash-generator" rel="noopener noreferrer"&gt;Text Hash Generator&lt;/a&gt; — paste any text and get all four hashes in one click.&lt;/p&gt;

&lt;p&gt;The Text Hash Generator is one of several developer tools at &lt;a href="https://www.solutiontoolkit.com/" rel="noopener noreferrer"&gt;SolutionToolkit&lt;/a&gt;. All tools run client-side with no server-side processing.&lt;/p&gt;

</description>
      <category>javascript</category>
      <category>security</category>
      <category>web</category>
      <category>browser</category>
    </item>
    <item>
      <title>A Client-Side JWT Debugger That Runs Entirely in Your Browser</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Thu, 18 Jun 2026 13:08:27 +0000</pubDate>
      <link>https://dev.to/azam-akram/a-client-side-jwt-debugger-that-runs-entirely-in-your-browser-4ha4</link>
      <guid>https://dev.to/azam-akram/a-client-side-jwt-debugger-that-runs-entirely-in-your-browser-4ha4</guid>
      <description>&lt;p&gt;Why Another JWT Tool?&lt;br&gt;
Most JWT debugging workflows involve copying a token into an online tool, inspecting the payload, and moving on. That works fine for simple use cases, but falls short when you need to do more — sign a token with a specific algorithm, verify a signature against a public key, or quickly check whether a token has expired.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://www.solutiontoolkit.com/tools/jwt-debugger" rel="noopener noreferrer"&gt;JWT Debugger&lt;/a&gt; is a free, browser-based tool that covers the full JWT workflow: decode, inspect, sign, and verify — all in one place, with no installation and no account required.&lt;br&gt;
What It Does&lt;/p&gt;

&lt;p&gt;Decode and inspect Paste any JWT and the tool splits it into header, payload, and signature — colour-coded so each part is visually distinct. Timestamp claims (iat, exp, nbf) are automatically converted to human-readable dates with a relative time display ("expires in 2h", "3 days ago").&lt;/p&gt;

&lt;p&gt;Status at a glance A badge shows whether the token is Active, Expired, or has No Expiry set. If a token is expired, you see exactly how long ago — no manual Unix timestamp conversion needed.&lt;/p&gt;

&lt;p&gt;Sign tokens Build a header and payload from scratch and sign them. All twelve standard algorithms are supported:&lt;/p&gt;

&lt;p&gt;FamilyAlgorithmsHMACHS256, HS384, HS512RSA PKCS#1RS256, RS384, RS512RSA-PSSPS256, PS384, PS512ECDSAES256, ES384, ES512&lt;/p&gt;

&lt;p&gt;For HMAC algorithms, enter your shared secret. For asymmetric algorithms, paste your PKCS#8 private key. The tool signs the token locally using crypto.subtle.sign and gives you the complete JWT.&lt;/p&gt;

&lt;p&gt;Verify signatures Paste a JWT and your key, click Verify, and the tool confirms whether the signature is valid. For RSA and ECDSA, paste the public key in SPKI PEM format. Verification runs entirely in crypto.subtle.verify.&lt;/p&gt;

&lt;p&gt;Edit and re-encode Modify the header or payload JSON and the encoded token updates in real time. Useful for quickly testing how a claim change affects the token structure, or for building a test token before signing it.&lt;/p&gt;

&lt;p&gt;How It Works Under the Hood&lt;br&gt;
The tool is built on the Web Crypto API — a browser-native cryptography interface available in all modern browsers. There are no third-party cryptography libraries. Every signing and verification operation calls crypto.subtle directly.&lt;/p&gt;

&lt;p&gt;Base64url encoding and decoding are handled with TextEncoder and atob/btoa. PEM keys are stripped of their headers and decoded from base64 before being passed to crypto.subtle.importKey. The signing input follows the JWT spec — base64url(header) + "." + base64url(payload) — and the resulting signature bytes are base64url-encoded and appended as the third segment.&lt;/p&gt;

&lt;p&gt;If you want to verify the behaviour yourself: open DevTools, go to the Network tab, and paste a token. You will see zero outbound requests.&lt;/p&gt;

&lt;p&gt;Try It&lt;br&gt;
Open the &lt;a href="https://www.solutiontoolkit.com/tools/jwt-debugger" rel="noopener noreferrer"&gt;JWT Debugger&lt;/a&gt; and paste any JWT into the top field. The decoded header and payload appear immediately. No account, no signup.&lt;/p&gt;

&lt;p&gt;If you want to understand the theory behind what you are looking at — how the header, payload, and signature fit together, what RS256 means, and how the private/public key split works — the companion article JWT Shared Secret: How JWTs are Signed and Shared Across Services covers the fundamentals.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://www.solutiontoolkit.com/tools/jwt-debugger" rel="noopener noreferrer"&gt;JWT Debugger&lt;/a&gt; is one of several developer tools at &lt;a href="https://www.solutiontoolkit.com" rel="noopener noreferrer"&gt;SolutionToolkit&lt;/a&gt;. All tools run client-side with no server-side processing.&lt;/p&gt;

</description>
      <category>javascript</category>
      <category>security</category>
      <category>web</category>
      <category>jwt</category>
    </item>
    <item>
      <title>I Built a Collection of 50+ Free Developer Tools That Run Entirely in the Browser</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Tue, 09 Jun 2026 00:01:16 +0000</pubDate>
      <link>https://dev.to/azam-akram/i-built-a-collection-of-100-free-developer-tools-that-run-entirely-in-the-browser-4421</link>
      <guid>https://dev.to/azam-akram/i-built-a-collection-of-100-free-developer-tools-that-run-entirely-in-the-browser-4421</guid>
      <description>&lt;h1&gt;
  
  
  I Built a Collection of 50+ Free Developer Tools That Run Entirely in the Browser
&lt;/h1&gt;

&lt;p&gt;As developers, we constantly run into small tasks that interrupt our workflow:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Formatting JSON&lt;/li&gt;
&lt;li&gt;Decoding JWTs&lt;/li&gt;
&lt;li&gt;Converting timestamps&lt;/li&gt;
&lt;li&gt;Comparing configuration files&lt;/li&gt;
&lt;li&gt;Generating UUIDs&lt;/li&gt;
&lt;li&gt;Building cron expressions&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Most of these tasks don't require a full application or a paid SaaS product. They just need a simple utility that works immediately.&lt;/p&gt;

&lt;p&gt;That's why I started building &lt;strong&gt;Solution Toolkit&lt;/strong&gt; — a collection of browser-based developer tools focused on everyday development tasks.&lt;/p&gt;

&lt;h2&gt;
  
  
  My Goals
&lt;/h2&gt;

&lt;p&gt;When building the toolkit, I wanted it to be:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Free to use&lt;/li&gt;
&lt;li&gt;No sign-up required&lt;/li&gt;
&lt;li&gt;Privacy-first&lt;/li&gt;
&lt;li&gt;Fast and lightweight&lt;/li&gt;
&lt;li&gt;Available directly in the browser&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Most tools run entirely client-side, so data never leaves your device.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's Included?
&lt;/h2&gt;

&lt;p&gt;Some of the categories currently available:&lt;/p&gt;

&lt;h3&gt;
  
  
  JSON &amp;amp; API Tools
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;JSON Formatter&lt;/li&gt;
&lt;li&gt;JSON Validator&lt;/li&gt;
&lt;li&gt;JSON Escape / Unescape&lt;/li&gt;
&lt;li&gt;JSON Converters&lt;/li&gt;
&lt;li&gt;JWT Debugger&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Encoding &amp;amp; Decoding
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Base64 Encoder / Decoder&lt;/li&gt;
&lt;li&gt;URL Encoder / Decoder&lt;/li&gt;
&lt;li&gt;HTML Encoder / Decoder&lt;/li&gt;
&lt;li&gt;ASCII Converter&lt;/li&gt;
&lt;li&gt;GZip Base64 Decoder&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Formatting &amp;amp; Validation
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;SQL Formatter&lt;/li&gt;
&lt;li&gt;YAML Formatter&lt;/li&gt;
&lt;li&gt;CSS Formatter&lt;/li&gt;
&lt;li&gt;JavaScript Formatter&lt;/li&gt;
&lt;li&gt;HTML Formatter&lt;/li&gt;
&lt;li&gt;Markdown Formatter&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Developer Utilities
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;UUID Generator&lt;/li&gt;
&lt;li&gt;Cron Expression Builder&lt;/li&gt;
&lt;li&gt;Epoch Timestamp Converter&lt;/li&gt;
&lt;li&gt;QR Code Generator&lt;/li&gt;
&lt;li&gt;Hash Generators&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Image Tools
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Image Compression&lt;/li&gt;
&lt;li&gt;Image Resizing&lt;/li&gt;
&lt;li&gt;PNG/JPG/WebP Conversion&lt;/li&gt;
&lt;li&gt;SVG Utilities&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What I Learned Building It
&lt;/h2&gt;

&lt;p&gt;A few things surprised me:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Developers care a lot about privacy for utility tools.&lt;/li&gt;
&lt;li&gt;Browser APIs are powerful enough to handle many tasks entirely client-side.&lt;/li&gt;
&lt;li&gt;Small utilities often provide more daily value than large applications.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Looking for Feedback
&lt;/h2&gt;

&lt;p&gt;I'm actively adding new tools and improving existing ones.&lt;/p&gt;

&lt;p&gt;What browser-based developer tool do you wish existed?&lt;/p&gt;

&lt;p&gt;Or what's a utility you find yourself searching for repeatedly?&lt;/p&gt;

&lt;p&gt;I'd love to hear suggestions.&lt;/p&gt;

&lt;p&gt;🔗 Project: &lt;a href="https://www.solutiontoolkit.com" rel="noopener noreferrer"&gt;https://www.solutiontoolkit.com&lt;/a&gt;&lt;/p&gt;

</description>
      <category>productivity</category>
      <category>showdev</category>
      <category>tooling</category>
      <category>webdev</category>
    </item>
    <item>
      <title>AWS ALB Scaling: Set Up Application Load Balancer with Auto Scaling Group (ASG)</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Mon, 08 Jun 2026 23:38:40 +0000</pubDate>
      <link>https://dev.to/azam-akram/aws-alb-scaling-set-up-application-load-balancer-with-auto-scaling-group-asg-2hdc</link>
      <guid>https://dev.to/azam-akram/aws-alb-scaling-set-up-application-load-balancer-with-auto-scaling-group-asg-2hdc</guid>
      <description>&lt;p&gt;In any well-architected cloud setup, managing traffic efficiently and scaling resources on demand are key to keeping your applications fast, reliable, and cost-efficient.&lt;br&gt;&lt;br&gt;
AWS makes this easy with two core services: the &lt;a href="https://aws.amazon.com/elasticloadbalancing/" rel="noopener noreferrer"&gt;&lt;strong&gt;Elastic Load Balancer (ELB)&lt;/strong&gt;&lt;/a&gt; for routing traffic, and &lt;a href="https://docs.aws.amazon.com/autoscaling/ec2/userguide/auto-scaling-groups.html" rel="noopener noreferrer"&gt;&lt;strong&gt;Auto Scaling Groups (ASG)&lt;/strong&gt;&lt;/a&gt; for automatically adjusting compute capacity as traffic changes.&lt;/p&gt;

&lt;p&gt;The &lt;a href="https://aws.amazon.com/elasticloadbalancing/" rel="noopener noreferrer"&gt;&lt;strong&gt;AWS Elastic Load Balancer&lt;/strong&gt;&lt;/a&gt; automatically distributes incoming requests across multiple targets, such as, EC2 instances, containers, or IPs, across different &lt;strong&gt;Availability Zones (AZs)&lt;/strong&gt;. AWS offers several types of load balancers, including &lt;strong&gt;Application&lt;/strong&gt;, &lt;strong&gt;Network&lt;/strong&gt;, and &lt;strong&gt;Gateway Load Balancers&lt;/strong&gt;, each suited for different scenarios.&lt;/p&gt;

&lt;p&gt;In this guide, we’ll set up an &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt; connected to an &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt; using the &lt;strong&gt;AWS CLI&lt;/strong&gt;.&lt;br&gt;&lt;br&gt;
Our setup will dynamically launch and manage &lt;a href="https://docs.aws.amazon.com/ec2/" rel="noopener noreferrer"&gt;&lt;strong&gt;EC2 instances&lt;/strong&gt;&lt;/a&gt;, organized into a &lt;a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html" rel="noopener noreferrer"&gt;&lt;strong&gt;Target Group&lt;/strong&gt;&lt;/a&gt;, ensuring smooth load distribution and high availability.&lt;/p&gt;
&lt;h2&gt;
  
  
  What We'll Do
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Create a &lt;strong&gt;security group&lt;/strong&gt; to allow necessary inbound traffic.
&lt;/li&gt;
&lt;li&gt;Define a &lt;strong&gt;launch template&lt;/strong&gt; with EC2 instance configuration and a &lt;strong&gt;user-data script&lt;/strong&gt; to deploy a simple web server.
&lt;/li&gt;
&lt;li&gt;Set up an &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt; to handle scaling based on demand.
&lt;/li&gt;
&lt;li&gt;Create a &lt;strong&gt;Target Group&lt;/strong&gt; to manage and distribute traffic.
&lt;/li&gt;
&lt;li&gt;Configure an &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt; and &lt;strong&gt;listener&lt;/strong&gt;, linking it to the Target Group for seamless load balancing.
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;An &lt;strong&gt;AWS account&lt;/strong&gt;. If you don’t have one, &lt;a href="https://dev.to/blog/how-to-get-started-with-aws-in-10-minutes/"&gt;follow this quick guide to create a free-tier AWS account&lt;/a&gt;.
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;AWS access keys&lt;/strong&gt; configured in your local CLI environment. These are needed for deploying resources.
Check out &lt;a href="https://dev.to/blog/how-to-get-started-with-aws-in-10-minutes/#9-get-aws-keys"&gt;Section 9&lt;/a&gt; of our AWS starter guide for details.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;
  
  
  Why Combine ALB with ASG?
&lt;/h2&gt;

&lt;p&gt;Pairing an &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt; with an &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt; gives you a scalable, cost-effective, and fault-tolerant setup. Here’s why it’s a great combo:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Automatic Scaling&lt;/strong&gt; – ASG adjusts the number of EC2 instances in real time as traffic changes.
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost Optimization&lt;/strong&gt; – You only pay for what you use, saving costs during low-traffic hours.
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;High Availability&lt;/strong&gt; – ALB spreads requests across healthy instances in multiple AZs, avoiding single points of failure.
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Better Fault Tolerance&lt;/strong&gt; – If one instance fails, ALB automatically routes traffic to healthy ones, ensuring smooth performance.
&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Let’s dive in and start building!&lt;/strong&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 1: Create a Security Group
&lt;/h2&gt;

&lt;p&gt;Before launching EC2 instances or attaching them to a Load Balancer, we need to define a &lt;strong&gt;security group&lt;/strong&gt;, which is a virtual firewall that controls inbound and outbound traffic. In this step, we’ll create a security group that allows &lt;strong&gt;SSH&lt;/strong&gt; (for remote access) and &lt;strong&gt;HTTP&lt;/strong&gt; (for web traffic) connections.&lt;/p&gt;

&lt;p&gt;Run the following command to create a new security group:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws ec2 create-security-group &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--group-name&lt;/span&gt; my-test-security-group &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--description&lt;/span&gt; &lt;span class="s2"&gt;"Security Group for My ALB"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--region&lt;/span&gt; &amp;lt;your-region-id&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This command creates a security group in your specified AWS region. Next, we’ll add inbound rules that define what types of traffic are allowed to reach our EC2 instances.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws ec2 authorize-security-group-ingress &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--group-name&lt;/span&gt; my-test-security-group &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--protocol&lt;/span&gt; tcp &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--port&lt;/span&gt; 22 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--cidr&lt;/span&gt; 0.0.0.0/0 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--region&lt;/span&gt; &amp;lt;your-region-id&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This rule enables SSH access on port 22, allowing you to connect to the instance from your terminal or SSH client.&lt;br&gt;
We use tcp as the protocol since SSH operates over the TCP layer.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: Allowing SSH from all IP addresses (0.0.0.0/0) is convenient for testing, but not secure for production. In real deployments, you should restrict this to your own IP or a trusted network range, for example: &lt;code&gt;--cidr 203.0.113.25/32&lt;/code&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Now we create another inbound rule to allow HTTP traffic on port 80 over TCP.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws ec2 authorize-security-group-ingress &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--group-name&lt;/span&gt; my-test-security-group &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--protocol&lt;/span&gt; tcp &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--port&lt;/span&gt; 80 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--cidr&lt;/span&gt; 0.0.0.0/0 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--region&lt;/span&gt; &amp;lt;your-region-id&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/ec2-security-group-console.webp"&lt;br&gt;
  alt="ec2-security-group-console"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;

&lt;p&gt;Let's verify the security group in the AWS Management Console:&lt;/p&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/ec2-security-group-gui.webp"&lt;br&gt;
  alt="ec2-security-group-gui"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 2: Create a Launch Template
&lt;/h2&gt;

&lt;p&gt;An &lt;strong&gt;AWS Launch Template&lt;/strong&gt; is a reusable blueprint that defines the configuration settings for your EC2 instances, such as the AMI, instance type, key pair, security groups, and optional user-data scripts.&lt;br&gt;&lt;br&gt;
Using a launch template helps &lt;strong&gt;standardize instance launches&lt;/strong&gt; and simplifies scaling operations within an &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;In this step, we’ll manually create a launch template using the &lt;strong&gt;AWS Management Console&lt;/strong&gt;.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Open the &lt;strong&gt;EC2 Dashboard&lt;/strong&gt; in the AWS Management Console.
&lt;/li&gt;
&lt;li&gt;In the left sidebar, choose &lt;strong&gt;Launch Templates&lt;/strong&gt;, then click &lt;strong&gt;Create launch template&lt;/strong&gt;.
&lt;/li&gt;
&lt;li&gt;Enter a name, such as &lt;code&gt;MyLaunchTemplate&lt;/code&gt;.
&lt;/li&gt;
&lt;li&gt;Under &lt;strong&gt;Application and OS Images (Amazon Machine Image)&lt;/strong&gt;, select &lt;strong&gt;Amazon Linux 2023 AMI&lt;/strong&gt; (eligible for the free tier).
&lt;/li&gt;
&lt;li&gt;Choose an &lt;strong&gt;Instance type&lt;/strong&gt;, e.g., &lt;code&gt;t2.micro&lt;/code&gt;.
&lt;/li&gt;
&lt;li&gt;Skip the &lt;strong&gt;Key pair&lt;/strong&gt; step for now (this is fine for testing, but in production, you should always create one for SSH access).
&lt;/li&gt;
&lt;li&gt;Under &lt;strong&gt;Network settings&lt;/strong&gt;, select the previously created &lt;strong&gt;security group (&lt;code&gt;my-test-security-group&lt;/code&gt;)&lt;/strong&gt;.
&lt;/li&gt;
&lt;li&gt;Expand &lt;strong&gt;Advanced details&lt;/strong&gt; and paste the following &lt;strong&gt;user data&lt;/strong&gt; script to automatically install and configure a simple web server.
Finally, click &lt;strong&gt;Create launch template&lt;/strong&gt;.
&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;#!/bin/bash&lt;/span&gt;
yum update &lt;span class="nt"&gt;-y&lt;/span&gt;
yum &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-y&lt;/span&gt; httpd
systemctl start httpd
systemctl &lt;span class="nb"&gt;enable &lt;/span&gt;httpd

&lt;span class="c"&gt;# Get the EC2 instance's availability zone&lt;/span&gt;
&lt;span class="nv"&gt;EC2AZ&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nv"&gt;TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sb"&gt;`&lt;/span&gt;curl &lt;span class="nt"&gt;-X&lt;/span&gt; PUT &lt;span class="s2"&gt;"http://169.254.169.254/latest/api/token"&lt;/span&gt; &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"X-aws-ec2-metadata-token-ttl-seconds: 21600"&lt;/span&gt;&lt;span class="sb"&gt;`&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; curl &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"X-aws-ec2-metadata-token: &lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;-v&lt;/span&gt; http://169.254.169.254/latest/meta-data/placement/availability-zone&lt;span class="si"&gt;)&lt;/span&gt;

&lt;span class="c"&gt;# Create a simple HTML file with the availability zone info&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s1"&gt;'&amp;lt;center&amp;gt;&amp;lt;h1&amp;gt;Hello from Web Server in Availability Zone: AZID &amp;lt;/h1&amp;gt;&amp;lt;/center&amp;gt;'&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; /var/www/html/index.txt
&lt;span class="nb"&gt;sed&lt;/span&gt; &lt;span class="s2"&gt;"s/AZID/&lt;/span&gt;&lt;span class="nv"&gt;$EC2AZ&lt;/span&gt;&lt;span class="s2"&gt;/"&lt;/span&gt; /var/www/html/index.txt &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; /var/www/html/index.html
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;This user-data script runs automatically when the instance starts. It:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Installs and enables the &lt;strong&gt;Apache (httpd)&lt;/strong&gt; web server.&lt;/li&gt;
&lt;li&gt;Fetches the &lt;strong&gt;Availability Zone&lt;/strong&gt; from the instance’s metadata.&lt;/li&gt;
&lt;li&gt;Displays a custom message in the browser showing the zone where the instance is running — useful when testing load balancing across multiple zones.&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Every EC2 instance has access to &lt;strong&gt;instance metadata&lt;/strong&gt;, which contains information like instance ID, region, IP addresses, and Availability Zone.&lt;br&gt;&lt;br&gt;
AWS exposes this data through a special internal endpoint — &lt;strong&gt;&lt;code&gt;169.254.169.254&lt;/code&gt;&lt;/strong&gt; — accessible only from within the instance.&lt;br&gt;&lt;br&gt;
&lt;a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html" rel="noopener noreferrer"&gt;Learn more about instance metadata in the official AWS documentation.&lt;/a&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/launch-template.webp"&lt;br&gt;
  alt="launch-template"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 3: Create an Auto Scaling Group (ASG)
&lt;/h2&gt;

&lt;p&gt;Now we will create an Auto Scaling group named "my-auto-scaling-group" using a launch template we previously created "MyLaunchTemplate." It sets the desired capacity to 2 instances, with a minimum of 1 and a maximum of 5 instances, distributed across two availability zones. The instances are launched in two different subnets within a Virtual Private Cloud (VPC) to ensure scalability and high availability.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws autoscaling create-auto-scaling-group &lt;span class="nt"&gt;--auto-scaling-group-name&lt;/span&gt; my-auto-scaling-group &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--launch-template&lt;/span&gt; &lt;span class="s2"&gt;"LaunchTemplateName=MyLaunchTemplate"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--min-size&lt;/span&gt; 1 &lt;span class="nt"&gt;--max-size&lt;/span&gt; 5 &lt;span class="nt"&gt;--desired-capacity&lt;/span&gt; 2 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--availability-zones&lt;/span&gt; &lt;span class="s2"&gt;"&amp;lt;availability-zone-1&amp;gt;"&lt;/span&gt; &lt;span class="s2"&gt;"&amp;lt;availability-zone-2&amp;gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--vpc-zone-identifier&lt;/span&gt; &lt;span class="s2"&gt;"&amp;lt;subnet-id-1&amp;gt;,&amp;lt;subnet-id-2&amp;gt;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Note: Replace &lt;code&gt;&amp;lt;subnet-id-x&amp;gt;&lt;/code&gt; and &lt;code&gt;&amp;lt;availablity-zone-x&amp;gt;&lt;/code&gt; according to your setup.&lt;/p&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/security-group.webp"&lt;br&gt;
  alt="security-group"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 4: Create Target Group and Load Balancer
&lt;/h2&gt;
&lt;h3&gt;
  
  
  Create a &lt;strong&gt;Target Group&lt;/strong&gt;
&lt;/h3&gt;

&lt;p&gt;A &lt;strong&gt;Target Group&lt;/strong&gt; defines the set of resources (such as EC2 instances) that an &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt; routes incoming requests to. It continuously performs &lt;strong&gt;health checks&lt;/strong&gt; to ensure traffic is sent only to healthy instances, improving reliability and availability.&lt;/p&gt;

&lt;p&gt;In this step, we’ll create an HTTP-based target group named &lt;code&gt;my-target-group&lt;/code&gt; that listens on &lt;strong&gt;port 80&lt;/strong&gt; within your VPC. This target group will later be linked to your &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt; so that any new instances launched by the ASG are automatically registered as targets.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws elbv2 create-target-group &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--name&lt;/span&gt; my-target-group &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--protocol&lt;/span&gt; HTTP &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--port&lt;/span&gt; 80 &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--vpc-id&lt;/span&gt; &amp;lt;vpc-id&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/target-group.webp"&lt;br&gt;
  alt="target-group"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Target group is created but no EC2 instances are registered yet. Instances will be registered automatically once the Auto Scaling Group is attached in a later step.&lt;/p&gt;
&lt;h3&gt;
  
  
  Create an &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt;
&lt;/h3&gt;

&lt;p&gt;An &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt; intelligently distributes incoming HTTP and HTTPS traffic across multiple targets, such as EC2 instances, in one or more &lt;strong&gt;Availability Zones (AZs)&lt;/strong&gt;. &lt;/p&gt;

&lt;p&gt;The following command creates an ALB named &lt;code&gt;my-alb&lt;/code&gt;, specifying the subnets and security groups that determine where and how the load balancer operates:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws elbv2 create-load-balancer &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--name&lt;/span&gt; my-alb &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--subnets&lt;/span&gt; &amp;lt;subnet-id-1&amp;gt; &amp;lt;subnet-id-2&amp;gt; &lt;span class="se"&gt;\&lt;/span&gt;
&lt;span class="nt"&gt;--security-groups&lt;/span&gt; &amp;lt;security-group-id&amp;gt; &lt;span class="se"&gt;\&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This command:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Creates an &lt;strong&gt;Application Load Balancer&lt;/strong&gt; called &lt;code&gt;my-alb&lt;/code&gt;.
&lt;/li&gt;
&lt;li&gt;Associates it with &lt;strong&gt;two subnets&lt;/strong&gt; (usually in different Availability Zones) for high availability.
&lt;/li&gt;
&lt;li&gt;Applies the specified &lt;strong&gt;security group&lt;/strong&gt;, which defines what inbound and outbound traffic is allowed for the load balancer.&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Replace &lt;code&gt;&amp;lt;subnet-id-1&amp;gt;&lt;/code&gt;, &lt;code&gt;&amp;lt;subnet-id-2&amp;gt;&lt;/code&gt;, and &lt;code&gt;&amp;lt;security-group-id&amp;gt;&lt;/code&gt; with actual values from your AWS setup. &lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/create-load-balancer.webp"&lt;br&gt;
  alt="create-load-balancer"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: You need to note down LoadBalancerArn, which we will use in coming sections.&lt;/p&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/load-balancer-setting.webp"&lt;br&gt;
  alt="load-balancer-setting"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;
&lt;h3&gt;
  
  
  Create a &lt;strong&gt;Listener&lt;/strong&gt; and Link it to the Target Group
&lt;/h3&gt;

&lt;p&gt;A &lt;strong&gt;Listener&lt;/strong&gt; is a key component of an &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt;. It checks for incoming connection requests on a specific port and protocol, then forwards the traffic to a designated &lt;strong&gt;Target Group&lt;/strong&gt; based on defined rules.&lt;br&gt;
In this step, we’ll create an &lt;strong&gt;HTTP listener&lt;/strong&gt; on &lt;strong&gt;port 80&lt;/strong&gt; for our ALB, which will route all incoming web requests to the previously created target group.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws elbv2 create-listener &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--load-balancer-arn&lt;/span&gt; &amp;lt;alb-arn&amp;gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--protocol&lt;/span&gt; HTTP &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--port&lt;/span&gt; 80 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--default-actions&lt;/span&gt; &lt;span class="nv"&gt;Type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;forward,TargetGroupArn&lt;span class="o"&gt;=&lt;/span&gt;&amp;lt;target-group-arn&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/create-listener.webp"&lt;br&gt;
  alt="create-listener"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;
&lt;h3&gt;
  
  
  Attach the &lt;strong&gt;Target Group&lt;/strong&gt; to the &lt;strong&gt;Auto Scaling Group&lt;/strong&gt;
&lt;/h3&gt;

&lt;p&gt;The &lt;strong&gt;Target Group&lt;/strong&gt; must be linked to your &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt; so that any instances launched by the ASG are automatically registered with the &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt;.&lt;br&gt;&lt;br&gt;
This ensures that traffic is distributed only to healthy, actively running instances managed by the ASG.&lt;/p&gt;

&lt;p&gt;Use the following command to attach the target group to your Auto Scaling group named &lt;code&gt;ASG1&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws autoscaling attach-load-balancer-target-groups &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--auto-scaling-group-name&lt;/span&gt; ASG1 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--target-group-arns&lt;/span&gt; &amp;lt;target-group-arn&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This command:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Connects the specified &lt;strong&gt;Target Group&lt;/strong&gt; to your &lt;strong&gt;ASG&lt;/strong&gt;.
&lt;/li&gt;
&lt;li&gt;Ensures that new instances launched by the ASG are automatically added to the target group for load balancing.
&lt;/li&gt;
&lt;li&gt;Enables health checks so unhealthy instances are replaced automatically.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;After this step, the Auto Scaling Group will launch two &lt;strong&gt;EC2&lt;/strong&gt; instances (based on the desired capacity you configured earlier).&lt;br&gt;&lt;br&gt;
Once these instances pass their &lt;strong&gt;health checks&lt;/strong&gt;, they’ll be marked as &lt;strong&gt;healthy&lt;/strong&gt;, and the &lt;strong&gt;ALB&lt;/strong&gt; will begin routing traffic to them automatically.&lt;/p&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/create-auto-scaling.webp"&lt;br&gt;
  alt="create-auto-scaling"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;

&lt;p&gt;We can also check the Target Group to see that both instances are healthy.&lt;/p&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/target-group.webp"&lt;br&gt;
  alt="target-group"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 5: Testing the Load Balancer
&lt;/h2&gt;

&lt;p&gt;To verify that the &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt; is correctly distributing traffic, open the &lt;strong&gt;ALB Dashboard&lt;/strong&gt; in the AWS Management Console and note down the &lt;strong&gt;DNS name&lt;/strong&gt; of your ALB.&lt;/p&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/test-elb.webp"&lt;br&gt;
  alt="test-elb"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;

&lt;p&gt;Enter this ALB DNS name in your web browser.&lt;/p&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/browser-elb-1.webp"&lt;br&gt;
  alt="browser-elb-1"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;

&lt;p&gt;Now, refresh the browser several times.&lt;/p&gt;

&lt;p&gt;
  src="/static/images/blog/setup-aws-alb-with-auto-scaling-group-asg-using-aws-cli/browser-elb-2.webp"&lt;br&gt;
  alt="browser-elb-2"&lt;br&gt;
  className="mx-auto my-6 w-full rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;

&lt;p&gt;As you refresh, you’ll notice that the web page alternates between different &lt;strong&gt;Availability Zones (AZs)&lt;/strong&gt;. This confirms that the &lt;strong&gt;Application Load Balancer&lt;/strong&gt; is intelligently distributing incoming traffic across multiple &lt;strong&gt;EC2 instances&lt;/strong&gt; (targets) in different zones.&lt;br&gt;&lt;br&gt;
Each request may reach a different instance within the &lt;strong&gt;Target Group&lt;/strong&gt;, depending on factors like health status, capacity, and routing algorithm.  &lt;/p&gt;

&lt;p&gt;&lt;strong&gt;In summary&lt;/strong&gt;, this test proves that your &lt;strong&gt;ALB&lt;/strong&gt; and &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt; are working together as intended, automatically balancing the load between multiple healthy EC2 instances.&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 6: Deleting the Setup
&lt;/h2&gt;

&lt;p&gt;Once you’ve verified that your &lt;strong&gt;Application Load Balancer (ALB)&lt;/strong&gt; and &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt; are working correctly, it’s good practice to clean up the resources to avoid ongoing AWS charges.&lt;/p&gt;

&lt;p&gt;Run the following commands to delete the setup:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;aws elbv2 delete-load-balancer &lt;span class="nt"&gt;--load-balancer-arn&lt;/span&gt; &amp;lt;alb-arn&amp;gt;
aws autoscaling delete-auto-scaling-group &lt;span class="nt"&gt;--auto-scaling-group-name&lt;/span&gt; ASG1 &lt;span class="nt"&gt;--force-delete&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These commands:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Remove the &lt;strong&gt;ALB&lt;/strong&gt; and its associated listeners and target group.
&lt;/li&gt;
&lt;li&gt;Delete the &lt;strong&gt;Auto Scaling Group&lt;/strong&gt; along with any EC2 instances it launched (using the &lt;code&gt;--force-delete&lt;/code&gt; flag ensures immediate deletion).
&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; You may also want to delete other related resources such as the &lt;strong&gt;launch template&lt;/strong&gt;, &lt;strong&gt;target group&lt;/strong&gt;, and &lt;strong&gt;security group&lt;/strong&gt; if they are no longer needed.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;You’ve successfully set up an &lt;strong&gt;AWS Application Load Balancer (ALB)&lt;/strong&gt; integrated with an &lt;strong&gt;Auto Scaling Group (ASG)&lt;/strong&gt; and &lt;strong&gt;Target Group&lt;/strong&gt; — all using the &lt;strong&gt;AWS CLI&lt;/strong&gt;.  &lt;/p&gt;

&lt;p&gt;This architecture automatically adjusts capacity based on demand while distributing traffic evenly across multiple instances, ensuring:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;High availability&lt;/strong&gt; across Availability Zones,
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fault tolerance&lt;/strong&gt; through automatic health checks and scaling, and
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost efficiency&lt;/strong&gt; by running only the resources you need.
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;With this foundation, you can now expand the setup, for example, adding HTTPS with an SSL certificate, customizing health checks, or deploying containerized workloads behind the load balancer.&lt;/p&gt;

&lt;p&gt;Thanks for reading!&lt;/p&gt;

&lt;p&gt;If you found this article useful, you may also enjoy the other technical articles, tutorials, and free developer tools I build at:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.solutiontoolkit.com" rel="noopener noreferrer"&gt;Solution Toolkit&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;You'll find browser-based tools for JSON, JWTs, YAML, timestamps, UUIDs, image processing, encoding/decoding, and many other everyday development tasks.&lt;/p&gt;

</description>
      <category>aws</category>
      <category>devops</category>
      <category>infrastructure</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Building Kafka Producer-Consumer Using Go and Docker</title>
      <dc:creator>Azam Akram</dc:creator>
      <pubDate>Mon, 08 Jun 2026 23:34:22 +0000</pubDate>
      <link>https://dev.to/azam-akram/building-kafka-producer-consumer-using-go-and-docker-493h</link>
      <guid>https://dev.to/azam-akram/building-kafka-producer-consumer-using-go-and-docker-493h</guid>
      <description>&lt;p&gt;In this blog, we'll walk through the process of building Kafka Producer and Consumer microservices using Go, integrated with Docker.&lt;/p&gt;

&lt;p&gt;We will,&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Build a Go Kafka producer with an HTTP endpoint.&lt;/li&gt;
&lt;li&gt;Build a Go Kafka consumer with retry and dead-letter support.&lt;/li&gt;
&lt;li&gt;Set up Kafka using Docker (KRaft mode — no Zookeeper required).&lt;/li&gt;
&lt;li&gt;Run the application end-to-end.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Introduction
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://en.wikipedia.org/wiki/Microservices" rel="noopener noreferrer"&gt;Microservices&lt;/a&gt; architecture is a popular approach for building scalable and maintainable applications. By breaking down applications into smaller, independent services, developers can enhance flexibility and streamline deployment cycles. We will implement Kafka producer and consumer applications in Go and demonstrate how to run Kafka in a Docker container with Docker Compose to create a seamless microservices environment.&lt;/p&gt;

&lt;h3&gt;
  
  
  Go
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://go.dev/" rel="noopener noreferrer"&gt;Go&lt;/a&gt; is an open-source, statically typed, compiled language designed at Google for simplicity, reliability, and efficiency. It ships with a rich standard library, first-class concurrency primitives (goroutines and channels), and produces single, statically-linked binaries — making it an excellent fit for microservices and containerised workloads.&lt;/p&gt;

&lt;h3&gt;
  
  
  Apache Kafka
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://kafka.apache.org/" rel="noopener noreferrer"&gt;Kafka&lt;/a&gt; is a distributed streaming platform used to build real-time data pipelines and streaming applications. It allows producers to send messages to topics, which are then consumed by various consumers, making it ideal for event-driven architectures.&lt;/p&gt;

&lt;h3&gt;
  
  
  Docker
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://www.docker.com/" rel="noopener noreferrer"&gt;Docker&lt;/a&gt; is a platform that allows developers to automate the deployment of applications inside lightweight, portable containers. With Docker Compose, you can manage services like Kafka in isolated containers, making it easy to build and maintain microservices architectures.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Go 1.22 or later. If you haven't installed Go yet, &lt;a href="https://dev.to/blog/all-you-need-to-start-go/"&gt;this guide&lt;/a&gt; walks you through the setup.&lt;/li&gt;
&lt;li&gt;Docker&lt;/li&gt;
&lt;li&gt;Docker Compose&lt;/li&gt;
&lt;li&gt;Basic knowledge of Go and Kafka&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Download Code from Github
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://github.com/azam-akram/dev-toolkit-go/tree/master/kafka-docker-go" rel="noopener noreferrer"&gt;Download full code&lt;/a&gt; from this github repository.&lt;/p&gt;

&lt;h2&gt;
  
  
  Project Overview
&lt;/h2&gt;

&lt;p&gt;We will create two Go applications:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Kafka Producer&lt;/strong&gt;: Exposes a &lt;code&gt;GET /send&lt;/code&gt; HTTP endpoint that publishes a message to a Kafka topic.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Kafka Consumer&lt;/strong&gt;: Connects to the topic, processes each message, and retries on failure with dead-letter support.
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;kafka-docker-go/
├── kafka-producer/
│   ├── go.mod
│   ├── main.go
│   ├── producer.go
│   ├── handler.go
│   └── model.go
├── kafka-consumer/
│   ├── go.mod
│   ├── main.go
│   ├── consumer.go
│   └── model.go
└── docker-compose.yml
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Building and Running the Application
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Docker Compose for Kafka Setup
&lt;/h3&gt;

&lt;p&gt;Before we start building the Go producer and consumer, we first need a running Kafka instance locally. Instead of manually installing Kafka and managing all its dependencies, we'll use Docker Compose to spin up a ready-to-use environment in a single command.&lt;/p&gt;

&lt;p&gt;We'll be using Apache Kafka running in KRaft mode (Kafka without ZooKeeper). Traditionally, Kafka relied on Apache ZooKeeper for cluster coordination, but modern Kafka versions (3.3+) have introduced KRaft (Kafka Raft metadata mode), which simplifies the architecture by removing ZooKeeper entirely.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;docker-compose.yml&lt;/code&gt;:&lt;/p&gt;

&lt;p&gt;This configuration provisions a single Kafka broker that also acts as a controller (KRaft mode):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;services&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;kafka&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;apache/kafka:latest&lt;/span&gt;
    &lt;span class="na"&gt;container_name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kafka&lt;/span&gt;
    &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;9092:9092"&lt;/span&gt;
    &lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_NODE_ID&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_PROCESS_ROLES&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;broker,controller'&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_CONTROLLER_QUORUM_VOTERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;1@localhost:9093'&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_LISTENERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;PLAINTEXT://:9092,CONTROLLER://:9093'&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_ADVERTISED_LISTENERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;PLAINTEXT://localhost:9092'&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_LISTENER_SECURITY_PROTOCOL_MAP&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT'&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_CONTROLLER_LISTENER_NAMES&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;CONTROLLER'&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;code&gt;KAFKA_PROCESS_ROLES: 'broker,controller'&lt;/code&gt;&lt;br&gt;
Instructs this single container to act simultaneously as both the metadata controller and the message broker, completely eliminating the need for ZooKeeper.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;code&gt;KAFKA_ADVERTISED_LISTENERS&lt;/code&gt;&lt;br&gt;
Defines the network address and port (&lt;code&gt;localhost:9092&lt;/code&gt;) that external clients, such as your Go producers and consumers, will use to connect to the cluster.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;code&gt;KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1&lt;/code&gt;&lt;br&gt;
Sets the replication factor for the internal offsets topic to 1. This is ideal for single-node local development, ensuring the cluster stays healthy without requiring peer nodes.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;To launch your local Kafka environment in detached mode, run:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;docker-compose up &lt;span class="nt"&gt;-d&lt;/span&gt;
&lt;span class="o"&gt;[&lt;/span&gt;+] up 2/2
 ✔ Network golang-kafka-docker_default  Created                          0.0s
 ✔ Container kafka                      Started
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Verify the container is running:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;docker ps
CONTAINER ID   IMAGE                COMMAND                  CREATED         STATUS         PORTS                    NAMES
ae58e1252135   apache/kafka:latest  &lt;span class="s2"&gt;"/__cacert_entrypoin…"&lt;/span&gt;   3 minutes ago   Up 3 minutes   0.0.0.0:9092-&amp;gt;9092/tcp   kafka
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now that Kafka is running, let's quickly verify it using the built-in console tools.&lt;/p&gt;

&lt;p&gt;In a terminal, start a Kafka console consumer that reads all messages from the beginning of the topic:&lt;/p&gt;

&lt;p&gt;I wrote following command in git Bash. Git Bash on Windows automatically converts paths starting with / to Windows paths, turning /opt/kafka/bin/... into C:/Program Files/Git/opt/kafka/bin/....&lt;br&gt;
SO I wrote path starting with double slashes, &lt;code&gt;//opt/kafka/bin/kafka-console-consumer.sh&lt;/code&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker &lt;span class="nb"&gt;exec&lt;/span&gt; &lt;span class="nt"&gt;-it&lt;/span&gt; kafka //opt/kafka/bin/kafka-console-consumer.sh &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--topic&lt;/span&gt; demo-kafka-topic &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--from-beginning&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In a second terminal, start a producer to send a test message:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker &lt;span class="nb"&gt;exec&lt;/span&gt; &lt;span class="nt"&gt;-it&lt;/span&gt; kafka //opt/kafka/bin/kafka-console-producer.sh &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--topic&lt;/span&gt; demo-kafka-topic
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Type a message and press Enter:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Hello from kafka Producer
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The consumer terminal will immediately display:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Hello from kafka Producer
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;
  src="/static/images/golang-kafka-producer-consumer-with-docker/kafka-producer-consumer-manual-test.webp"&lt;br&gt;
  alt="kafka-producer-consumer-manual-test"&lt;br&gt;
  className="mx-auto my-6 h-5/6 w-5/6 rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;


&lt;h2&gt;
  
  
  Kafka Consumer Service (Go)
&lt;/h2&gt;

&lt;p&gt;Now that our Kafka broker is running, let's build the &lt;strong&gt;consumer service&lt;/strong&gt; that listens to messages from the topic and processes them in real time.&lt;/p&gt;

&lt;p&gt;We'll use &lt;a href="https://github.com/segmentio/kafka-go" rel="noopener noreferrer"&gt;kafka-go&lt;/a&gt; — a pure Go Kafka client that is a light-weight dependency, and gives fine-grained control over offset commits.&lt;/p&gt;
&lt;h3&gt;
  
  
  &lt;code&gt;go.mod&lt;/code&gt;
&lt;/h3&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="n"&gt;module&lt;/span&gt; &lt;span class="n"&gt;github&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;com&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;azam&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;akram&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;docker&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="k"&gt;go&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;consumer&lt;/span&gt;

&lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="m"&gt;1.22&lt;/span&gt;

&lt;span class="n"&gt;require&lt;/span&gt; &lt;span class="n"&gt;github&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;com&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;segmentio&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="n"&gt;v0&lt;/span&gt;&lt;span class="m"&gt;.4.51&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Run &lt;code&gt;go mod tidy&lt;/code&gt; after creating this file to download the dependency.&lt;/p&gt;
&lt;h3&gt;
  
  
  Message Model (&lt;code&gt;model.go&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;The &lt;code&gt;Message&lt;/code&gt; struct is the shared data contract between producer and consumer. It is marshalled to JSON by the producer and unmarshalled back by the consumer.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;package&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;

&lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="n"&gt;Message&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;UUID&lt;/span&gt;    &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"uuid"`&lt;/span&gt;
    &lt;span class="n"&gt;From&lt;/span&gt;    &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"from"`&lt;/span&gt;
    &lt;span class="n"&gt;To&lt;/span&gt;      &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"to"`&lt;/span&gt;
    &lt;span class="n"&gt;Message&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"message"`&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Kafka Consumer (&lt;code&gt;consumer.go&lt;/code&gt;)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;package&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="s"&gt;"context"&lt;/span&gt;
    &lt;span class="s"&gt;"encoding/json"&lt;/span&gt;
    &lt;span class="s"&gt;"fmt"&lt;/span&gt;
    &lt;span class="s"&gt;"log"&lt;/span&gt;
    &lt;span class="s"&gt;"time"&lt;/span&gt;

    &lt;span class="s"&gt;"github.com/segmentio/kafka-go"&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;maxRetries&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
    &lt;span class="n"&gt;retryDelay&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Second&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="n"&gt;KafkaConsumer&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;reader&lt;/span&gt;    &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Reader&lt;/span&gt;
    &lt;span class="n"&gt;dlqWriter&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Writer&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="n"&gt;NewKafkaConsumer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;broker&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;groupID&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaConsumer&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;dlqTopic&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;topic&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="s"&gt;"-dlt"&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;KafkaConsumer&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;reader&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NewReader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ReaderConfig&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;Brokers&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;     &lt;span class="p"&gt;[]&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;broker&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="n"&gt;Topic&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;       &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;GroupID&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;     &lt;span class="n"&gt;groupID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;MinBytes&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;    &lt;span class="m"&gt;10e3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;MaxBytes&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;    &lt;span class="m"&gt;10e6&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;StartOffset&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;FirstOffset&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;}),&lt;/span&gt;
        &lt;span class="n"&gt;dlqWriter&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Writer&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;Addr&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;                   &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;TCP&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;broker&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;Topic&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;                  &lt;span class="n"&gt;dlqTopic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;Balancer&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;               &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;LeastBytes&lt;/span&gt;&lt;span class="p"&gt;{},&lt;/span&gt;
            &lt;span class="n"&gt;AllowAutoTopicCreation&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="no"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c"&gt;// Run fetches messages in a loop until ctx is cancelled.&lt;/span&gt;
&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaConsumer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;Run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt; &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Context&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;reader&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;FetchMessage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Err&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="c"&gt;// clean shutdown on SIGTERM / SIGINT&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt;
            &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"ERROR fetch: %v"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;continue&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;

        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;processWithRetry&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"ERROR retries exhausted, routing to DLQ — uuid=%s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Key&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
            &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sendToDLQ&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;

        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;reader&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;CommitMessages&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"ERROR commit: %v"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaConsumer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;processWithRetry&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="n"&gt;lastErr&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="n"&gt;maxRetries&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;process&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;lastErr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;
            &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"WARN attempt %d/%d failed: %v"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;maxRetries&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;maxRetries&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retryDelay&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt;
            &lt;span class="k"&gt;continue&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;lastErr&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="n"&gt;process&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt; &lt;span class="n"&gt;Message&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Unmarshal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Value&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Errorf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"unmarshal: %w"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"================================="&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Received message:"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"  UUID:    %s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;UUID&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"  From:    %s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;From&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"  To:      %s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;To&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"  Message: %s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"================================="&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaConsumer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;sendToDLQ&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt; &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Context&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dlqWriter&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WriteMessages&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;Key&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;   &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;Value&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Value&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"ERROR DLQ write: %v"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaConsumer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;Close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;reader&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dlqWriter&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Key design decisions explained:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;code&gt;kafka.NewReader&lt;/code&gt; with &lt;code&gt;GroupID&lt;/code&gt;&lt;/strong&gt;: registering a &lt;code&gt;GroupID&lt;/code&gt; enables consumer-group semantics — Kafka assigns partitions to group members and tracks committed offsets per group. Multiple consumer instances sharing the same &lt;code&gt;GroupID&lt;/code&gt; can scale horizontally across partitions.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;code&gt;FetchMessage&lt;/code&gt; vs &lt;code&gt;ReadMessage&lt;/code&gt;&lt;/strong&gt;: &lt;code&gt;kafka-go&lt;/code&gt; provides two fetch APIs. &lt;code&gt;ReadMessage&lt;/code&gt; automatically commits the offset after each fetch, which risks marking a message as processed even if your handler panics. &lt;code&gt;FetchMessage&lt;/code&gt; leaves the offset uncommitted; we call &lt;code&gt;CommitMessages&lt;/code&gt; only after the message has been fully processed (or routed to the DLQ). This gives us &lt;strong&gt;at-least-once delivery&lt;/strong&gt; semantics.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;code&gt;StartOffset: kafka.FirstOffset&lt;/code&gt;&lt;/strong&gt;: equivalent to &lt;code&gt;auto-offset-reset: earliest&lt;/code&gt; in a Spring Boot config. On first start (no committed offset for the group), the consumer reads from the beginning of the topic.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;code&gt;processWithRetry&lt;/code&gt;&lt;/strong&gt;: wraps &lt;code&gt;process&lt;/code&gt; in a simple retry loop — up to &lt;code&gt;maxRetries&lt;/code&gt; attempts with a &lt;code&gt;retryDelay&lt;/code&gt; between each. If a transient error causes &lt;code&gt;process&lt;/code&gt; to fail, the message is retried in place before being sent to the DLQ.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Dead-letter topic (&lt;code&gt;-dlt&lt;/code&gt; suffix)&lt;/strong&gt;: if all retry attempts are exhausted, the raw message bytes are forwarded to &lt;code&gt;demo-kafka-topic-dlt&lt;/code&gt;. Failed messages are captured for inspection or manual reprocessing rather than silently dropped — the same pattern as &lt;code&gt;@DltHandler&lt;/code&gt; in Spring Kafka.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Application Entry Point (&lt;code&gt;main.go&lt;/code&gt;)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;package&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="s"&gt;"context"&lt;/span&gt;
    &lt;span class="s"&gt;"log"&lt;/span&gt;
    &lt;span class="s"&gt;"net/http"&lt;/span&gt;
    &lt;span class="s"&gt;"os"&lt;/span&gt;
    &lt;span class="s"&gt;"os/signal"&lt;/span&gt;
    &lt;span class="s"&gt;"syscall"&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;broker&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="s"&gt;"localhost:9092"&lt;/span&gt;
    &lt;span class="n"&gt;topic&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="s"&gt;"demo-kafka-topic"&lt;/span&gt;
    &lt;span class="n"&gt;groupID&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="s"&gt;"demo-kafka-consumer"&lt;/span&gt;
    &lt;span class="n"&gt;port&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="s"&gt;"5555"&lt;/span&gt;

    &lt;span class="n"&gt;consumer&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;NewKafkaConsumer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;broker&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;groupID&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="c"&gt;// Simple health endpoint&lt;/span&gt;
    &lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;mux&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NewServeMux&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;mux&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;HandleFunc&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"/health"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ResponseWriter&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Header&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Content-Type"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"application/json"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Write&lt;/span&gt;&lt;span class="p"&gt;([]&lt;/span&gt;&lt;span class="kt"&gt;byte&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;`{"status":"up"}`&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="p"&gt;})&lt;/span&gt;
        &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Health endpoint on :%s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ListenAndServe&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;":"&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;mux&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}()&lt;/span&gt;

    &lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cancel&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;signal&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NotifyContext&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Background&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Interrupt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;syscall&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SIGTERM&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;cancel&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Consumer started topic=%s group=%s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;groupID&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Println&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Consumer stopped"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;signal.NotifyContext&lt;/code&gt;&lt;/strong&gt;: creates a context that is cancelled when the process receives &lt;code&gt;SIGINT&lt;/code&gt; (Ctrl+C) or &lt;code&gt;SIGTERM&lt;/code&gt; (e.g. from &lt;code&gt;docker stop&lt;/code&gt;). When the context is cancelled, &lt;code&gt;FetchMessage&lt;/code&gt; returns immediately, allowing the &lt;code&gt;Run&lt;/code&gt; loop to exit cleanly.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Health goroutine&lt;/strong&gt;: a minimal &lt;code&gt;GET /health&lt;/code&gt; endpoint runs on port 5555 in a background goroutine, so the consumer can be probed by Docker health checks or an orchestrator without interfering with the main consume loop.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Kafka Producer Service (Go)
&lt;/h2&gt;

&lt;p&gt;The producer service exposes a simple REST endpoint. When called, it builds a &lt;code&gt;Message&lt;/code&gt; and publishes it to the Kafka topic.&lt;/p&gt;

&lt;h3&gt;
  
  
  &lt;code&gt;go.mod&lt;/code&gt;
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="n"&gt;module&lt;/span&gt; &lt;span class="n"&gt;github&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;com&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;azam&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;akram&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;docker&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="k"&gt;go&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;producer&lt;/span&gt;

&lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="m"&gt;1.22&lt;/span&gt;

&lt;span class="n"&gt;require&lt;/span&gt; &lt;span class="n"&gt;github&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;com&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;segmentio&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="k"&gt;go&lt;/span&gt; &lt;span class="n"&gt;v0&lt;/span&gt;&lt;span class="m"&gt;.4.47&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Message Model (&lt;code&gt;model.go&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;The producer's &lt;code&gt;Message&lt;/code&gt; model mirrors the consumer's exactly, ensuring the JSON wire format is compatible on both sides.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;package&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;

&lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="n"&gt;Message&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;UUID&lt;/span&gt;    &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"uuid"`&lt;/span&gt;
    &lt;span class="n"&gt;From&lt;/span&gt;    &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"from"`&lt;/span&gt;
    &lt;span class="n"&gt;To&lt;/span&gt;      &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"to"`&lt;/span&gt;
    &lt;span class="n"&gt;Message&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"message"`&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;In a larger system you would extract this into a shared Go module to avoid duplication.&lt;br&gt;
For this self-contained example, both services define their own copy.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h3&gt;
  
  
  Kafka Producer (&lt;code&gt;producer.go&lt;/code&gt;)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;package&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="s"&gt;"context"&lt;/span&gt;
    &lt;span class="s"&gt;"encoding/json"&lt;/span&gt;
    &lt;span class="s"&gt;"fmt"&lt;/span&gt;
    &lt;span class="s"&gt;"log"&lt;/span&gt;

    &lt;span class="s"&gt;"github.com/segmentio/kafka-go"&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="n"&gt;KafkaProducer&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;writer&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Writer&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="n"&gt;NewKafkaProducer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;broker&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;topic&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaProducer&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;KafkaProducer&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;writer&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Writer&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="n"&gt;Addr&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;                   &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;TCP&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;broker&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;Topic&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;                  &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;Balancer&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;               &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;LeastBytes&lt;/span&gt;&lt;span class="p"&gt;{},&lt;/span&gt;
            &lt;span class="n"&gt;RequiredAcks&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;           &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;RequireOne&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;AllowAutoTopicCreation&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="no"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaProducer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;Send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt; &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Context&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt; &lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Marshal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Errorf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"marshal: %w"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;writer&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WriteMessages&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;kafka&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;Key&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;   &lt;span class="p"&gt;[]&lt;/span&gt;&lt;span class="kt"&gt;byte&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;UUID&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="n"&gt;Value&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Errorf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"write: %w"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Sent  uuid=%s topic=%s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;UUID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;writer&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Topic&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaProducer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;Close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;writer&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Key design decisions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;RequiredAcks: kafka.RequireOne&lt;/code&gt;&lt;/strong&gt;: the broker acknowledges the write once the leader partition has persisted the message. This balances throughput and durability for a single-node development setup. In production, use &lt;code&gt;kafka.RequireAll&lt;/code&gt; to wait for all in-sync replicas.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;AllowAutoTopicCreation: true&lt;/code&gt;&lt;/strong&gt;: if &lt;code&gt;demo-kafka-topic&lt;/code&gt; does not exist yet, the broker creates it automatically on first write — no need to pre-create topics manually.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Message key set to UUID&lt;/strong&gt;: using the UUID as the partition key ensures messages with the same UUID always land on the same partition. For random distribution across partitions, omit the key.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;fmt.Errorf("...: %w", err)&lt;/code&gt;&lt;/strong&gt;: wraps errors with context using the &lt;code&gt;%w&lt;/code&gt; verb so callers can use &lt;code&gt;errors.Is&lt;/code&gt; / &lt;code&gt;errors.As&lt;/code&gt; for structured error handling.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  HTTP Handler (&lt;code&gt;handler.go&lt;/code&gt;)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;package&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="s"&gt;"context"&lt;/span&gt;
    &lt;span class="s"&gt;"fmt"&lt;/span&gt;
    &lt;span class="s"&gt;"log"&lt;/span&gt;
    &lt;span class="s"&gt;"net/http"&lt;/span&gt;
    &lt;span class="s"&gt;"time"&lt;/span&gt;

    &lt;span class="s"&gt;"github.com/google/uuid"&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="n"&gt;SendHandler&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;producer&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;KafkaProducer&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;h&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;SendHandler&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;ServeHTTP&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ResponseWriter&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;msg&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;UUID&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;    &lt;span class="n"&gt;uuid&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;New&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
        &lt;span class="n"&gt;From&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;    &lt;span class="s"&gt;"Alice"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;To&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;      &lt;span class="s"&gt;"Bob"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;Message&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="s"&gt;"Hello from Go producer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cancel&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WithTimeout&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Context&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Second&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;cancel&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;h&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;producer&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"ERROR send: %v"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"failed to send message"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;StatusInternalServerError&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Fprintln&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"Message sent!"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;ServeHTTP&lt;/code&gt;&lt;/strong&gt;: implementing the &lt;code&gt;http.Handler&lt;/code&gt; interface directly (rather than a plain &lt;code&gt;func&lt;/code&gt;) lets us inject the &lt;code&gt;*KafkaProducer&lt;/code&gt; dependency cleanly and register the handler with &lt;code&gt;mux.Handle("/send", &amp;amp;SendHandler{...})&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;context.WithTimeout&lt;/code&gt;&lt;/strong&gt;: the send is bound to a 5-second deadline. If the Kafka broker is unavailable, &lt;code&gt;WriteMessages&lt;/code&gt; returns promptly with a deadline-exceeded error rather than hanging the HTTP request indefinitely.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;newUUID&lt;/code&gt;&lt;/strong&gt;: generates a RFC 4122 v4 UUID via &lt;code&gt;github.com/google/uuid&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Application Entry Point (&lt;code&gt;main.go&lt;/code&gt;)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;package&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="s"&gt;"log"&lt;/span&gt;
    &lt;span class="s"&gt;"net/http"&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;broker&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="s"&gt;"localhost:9092"&lt;/span&gt;
    &lt;span class="n"&gt;topic&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="s"&gt;"demo-kafka-topic"&lt;/span&gt;
    &lt;span class="n"&gt;port&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="s"&gt;"4444"&lt;/span&gt;

    &lt;span class="n"&gt;producer&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;NewKafkaProducer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;broker&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;producer&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="n"&gt;mux&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NewServeMux&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;mux&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Handle&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"/send"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;SendHandler&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;producer&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;producer&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="n"&gt;mux&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;HandleFunc&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"/health"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ResponseWriter&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Header&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Content-Type"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"application/json"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Write&lt;/span&gt;&lt;span class="p"&gt;([]&lt;/span&gt;&lt;span class="kt"&gt;byte&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;`{"status":"up"}`&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="p"&gt;})&lt;/span&gt;

    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Printf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Producer listening on :%s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;log&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Fatal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ListenAndServe&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;":"&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;mux&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The producer uses Go's built-in &lt;code&gt;net/http&lt;/code&gt; server, no third-party web framework needed. A &lt;code&gt;GET /health&lt;/code&gt; endpoint is registered alongside &lt;code&gt;/send&lt;/code&gt; so that the service can be health-checked by Docker or an orchestrator.&lt;/p&gt;




&lt;h2&gt;
  
  
  Running the Application End-to-End
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Step 1 — Start Kafka
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker-compose up &lt;span class="nt"&gt;-d&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 2 — Start the Consumer
&lt;/h3&gt;

&lt;p&gt;From the &lt;code&gt;kafka-consumer&lt;/code&gt; directory:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;go mod tidy
go run &lt;span class="nb"&gt;.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The consumer starts on port &lt;strong&gt;5555&lt;/strong&gt; and begins listening to &lt;code&gt;demo-kafka-topic&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;2026/05/29 10:00:00 Health endpoint on :5555
2026/05/29 10:00:00 Consumer started topic=demo-kafka-topic group=demo-kafka-consumer
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 3 — Start the Producer
&lt;/h3&gt;

&lt;p&gt;From the &lt;code&gt;kafka-producer&lt;/code&gt; directory:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;go mod tidy
go run &lt;span class="nb"&gt;.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The producer starts on port &lt;strong&gt;4444&lt;/strong&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;2026/05/29 10:00:05 Producer listening on :4444
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 4 — Send a Message
&lt;/h3&gt;

&lt;p&gt;Trigger the producer's REST endpoint:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl http://localhost:4444/send
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You should see the response:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Message sent!
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 5 — Observe the Consumer Logs
&lt;/h3&gt;

&lt;p&gt;In the consumer terminal you will see:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;=================================
Received message:
  UUID:    3f2504e0-4f89-11d3-9a0c-0305e82c3301
  From:    Alice
  To:      Bob
  Message: Hello from Go producer
=================================
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And in the producer terminal, the send confirmation:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;2026/05/29 10:00:10 Sent  uuid=3f2504e0-4f89-11d3-9a0c-0305e82c3301 topic=demo-kafka-topic
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;
  src="/static/images/golang-kafka-producer-consumer-with-docker/kafka-docker-go-e2e-test.png"&lt;br&gt;
  alt="kafka-docker-go-e2e-test"&lt;br&gt;
  className="mx-auto my-6 h-5/6 w-5/6 rounded-lg shadow-lg"&lt;br&gt;
/&amp;gt;&lt;/p&gt;
&lt;h3&gt;
  
  
  Step 6 — Check Health Endpoints
&lt;/h3&gt;

&lt;p&gt;Both services expose a &lt;code&gt;/health&lt;/code&gt; endpoint:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Producer health&lt;/span&gt;
curl http://localhost:4444/health

&lt;span class="c"&gt;# Consumer health&lt;/span&gt;
curl http://localhost:5555/health
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Both return:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"status"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"up"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Step 7 — Graceful Shutdown
&lt;/h3&gt;

&lt;p&gt;Press &lt;code&gt;Ctrl+C&lt;/code&gt; in the consumer terminal. The &lt;code&gt;signal.NotifyContext&lt;/code&gt; cancels the context, &lt;code&gt;FetchMessage&lt;/code&gt; returns, and the consumer exits cleanly:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;2026/05/29 10:01:00 Consumer stopped
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;p&gt;In this blog we built two Go microservices that communicate asynchronously through Apache Kafka running in Docker:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;Responsibility&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;docker-compose.yml&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Runs a single-node Kafka broker in KRaft mode (no ZooKeeper)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;KafkaProducer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Marshals a &lt;code&gt;Message&lt;/code&gt; to JSON and publishes it to a topic&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;SendHandler&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Exposes a &lt;code&gt;GET /send&lt;/code&gt; HTTP endpoint to trigger the producer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;KafkaConsumer&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Unmarshals and processes messages with manual offset commits&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;processWithRetry&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Retries failed messages up to 3 times with a 1-second backoff&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;sendToDLQ&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Routes exhausted messages to a dead-letter topic (&lt;code&gt;-dlt&lt;/code&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The key production-ready features we added beyond a minimal example:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Manual offset commit&lt;/strong&gt; — &lt;code&gt;FetchMessage&lt;/code&gt; + &lt;code&gt;CommitMessages&lt;/code&gt; ensures an offset is committed only after successful processing, giving at-least-once delivery semantics.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Retry with backoff&lt;/strong&gt; — transient failures are retried up to 3 times before escalating to the DLQ.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dead-letter topic&lt;/strong&gt; — messages that exhaust all retries are forwarded to &lt;code&gt;demo-kafka-topic-dlt&lt;/code&gt; for inspection rather than silently dropped.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Graceful shutdown&lt;/strong&gt; — &lt;code&gt;signal.NotifyContext&lt;/code&gt; cancels the consume loop on &lt;code&gt;SIGTERM&lt;/code&gt;/&lt;code&gt;SIGINT&lt;/code&gt;, preventing data loss on process stop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Health endpoints&lt;/strong&gt; — both services expose &lt;code&gt;GET /health&lt;/code&gt; for Docker and orchestrator health checks.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Environment-driven config&lt;/strong&gt; — all connection parameters (&lt;code&gt;KAFKA_BROKER&lt;/code&gt;, &lt;code&gt;KAFKA_TOPIC&lt;/code&gt;, etc.) are read from environment variables, making both services container-ready out of the box.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>go</category>
      <category>kafka</category>
      <category>docker</category>
      <category>microservices</category>
    </item>
  </channel>
</rss>
