<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community</title>
    <description>The most recent home feed on DEV Community.</description>
    <link>https://dev.to</link>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/rss"/>
    <language>en</language>
    <item>
      <title>Kafka for Data Engineers: Core Concepts, KRaft, and the Patterns That Actually Work</title>
      <dc:creator>De' Clerke</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:36:14 +0000</pubDate>
      <link>https://dev.to/de_clerke/kafka-for-data-engineers-core-concepts-kraft-and-the-patterns-that-actually-work-3d0m</link>
      <guid>https://dev.to/de_clerke/kafka-for-data-engineers-core-concepts-kraft-and-the-patterns-that-actually-work-3d0m</guid>
      <description>&lt;p&gt;If your Kafka Docker Compose still has a ZooKeeper service in it, your setup is already legacy. As of Kafka 4.0 (released March 2025), ZooKeeper is gone. The architecture changed, the config changed, and the setup you learned two years ago will not work with any Kafka 4.x image.&lt;/p&gt;

&lt;p&gt;This guide covers Kafka from the ground up — what it is, how it works, how to run it locally in 2026 with KRaft, and the Python patterns that hold up in production. It also covers the gotchas that waste days when you're new to it.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Kafka Is and Why Data Engineers Use It
&lt;/h2&gt;

&lt;p&gt;Kafka is a distributed event streaming platform. The core abstraction is a log: an ordered, append-only sequence of records. Producers write to the log. Consumers read from it. The log is retained for a configurable period (default 7 days), so multiple consumers can independently read the same data at their own pace.&lt;/p&gt;

&lt;p&gt;For data engineers, this matters in three scenarios:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Real-time ingestion.&lt;/strong&gt; When data arrives faster than a batch pipeline can process it — flight events, sensor readings, stock ticks, user clicks — Kafka buffers it reliably. Your pipeline reads at whatever pace it can sustain without losing data.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Decoupling producers from consumers.&lt;/strong&gt; Your ingestion pipeline doesn't need to know who consumes the data or how fast they process it. Add a new downstream system and it starts reading from whatever offset it needs. Nothing about the producer changes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Change Data Capture (CDC).&lt;/strong&gt; Kafka Connect + Debezium reads PostgreSQL's write-ahead log and streams every row-level insert, update, and delete to a Kafka topic. Downstream systems get a live feed of database changes without polling.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Architecture: What You Actually Need to Know
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Brokers, Topics, and Partitions
&lt;/h3&gt;

&lt;p&gt;A Kafka &lt;strong&gt;broker&lt;/strong&gt; is a server that stores messages and serves producers and consumers. A production cluster has multiple brokers for fault tolerance.&lt;/p&gt;

&lt;p&gt;A &lt;strong&gt;topic&lt;/strong&gt; is a named stream of messages — like a database table, but append-only. Topics are divided into &lt;strong&gt;partitions&lt;/strong&gt;, which are the actual unit of storage and parallelism. Each partition is an ordered, immutable log on disk.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Topic: flight-events (3 partitions)

Partition 0: [offset 0] [offset 1] [offset 2] [offset 3] ...
Partition 1: [offset 0] [offset 1] [offset 2] ...
Partition 2: [offset 0] [offset 1] ...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Messages within a partition are strictly ordered. Messages across partitions are not. If you need strict global ordering, use one partition. If you need parallelism, use more.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Partition count rule of thumb:&lt;/strong&gt; &lt;code&gt;partitions = max(target throughput / single-partition throughput, number of consumers)&lt;/code&gt;. You can increase partition count later, but you can never decrease it. Start conservative.&lt;/p&gt;

&lt;h3&gt;
  
  
  Offsets
&lt;/h3&gt;

&lt;p&gt;An &lt;strong&gt;offset&lt;/strong&gt; is the unique sequential ID of a message within a partition. Offset 0 is the first message. Offset 47 is the 48th. Consumers track their position by storing ("committing") the last offset they processed. This is how they resume after a restart without reprocessing everything or missing anything.&lt;/p&gt;

&lt;h3&gt;
  
  
  Consumer Groups
&lt;/h3&gt;

&lt;p&gt;A &lt;strong&gt;consumer group&lt;/strong&gt; is a set of consumers that split a topic's partitions between them. Each partition is assigned to exactly one consumer in the group at a time.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Topic: flight-events (4 partitions)
Consumer group: pipeline-processors (2 consumers)

Consumer A: handles Partition 0 + Partition 1
Consumer B: handles Partition 2 + Partition 3
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is the horizontal scaling model. To process faster, add consumers to the group — up to the number of partitions. Beyond that, extra consumers sit idle.&lt;/p&gt;

&lt;p&gt;A &lt;strong&gt;rebalance&lt;/strong&gt; happens when consumers join or leave the group. Partitions get redistributed. In older Kafka, rebalances were disruptive — all consumers stopped processing, all partitions were revoked, and they were reassigned from scratch. With the new rebalance protocol (KIP-848, GA in Kafka 4.0), rebalances are incremental. Only the partitions that need to move actually move. For large consumer groups, this is a significant operational improvement.&lt;/p&gt;

&lt;h3&gt;
  
  
  Delivery Semantics
&lt;/h3&gt;

&lt;p&gt;Three guarantees, each with a different tradeoff:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Semantic&lt;/th&gt;
&lt;th&gt;Description&lt;/th&gt;
&lt;th&gt;Risk&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;At-most-once&lt;/td&gt;
&lt;td&gt;Commit before processing&lt;/td&gt;
&lt;td&gt;May lose messages on crash&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;At-least-once&lt;/td&gt;
&lt;td&gt;Commit after processing&lt;/td&gt;
&lt;td&gt;May process duplicates on retry&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exactly-once&lt;/td&gt;
&lt;td&gt;Transactions + idempotent producer&lt;/td&gt;
&lt;td&gt;Highest complexity and latency&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For most data engineering pipelines, &lt;strong&gt;at-least-once&lt;/strong&gt; is the right default. Make your processing idempotent (upsert into PostgreSQL with &lt;code&gt;ON CONFLICT&lt;/code&gt;, for example) and duplicates become harmless.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Big Change in 2025: KRaft Replaces ZooKeeper
&lt;/h2&gt;

&lt;p&gt;Every Kafka deployment before Kafka 4.0 required ZooKeeper — a separate distributed coordination service that managed cluster metadata (which broker is the controller, partition leaders, topic configs, consumer group state).&lt;/p&gt;

&lt;p&gt;ZooKeeper was the most operationally painful part of running Kafka. It required its own cluster, its own monitoring, its own tuning. It had its own failure modes that were separate from Kafka's.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;KRaft&lt;/strong&gt; (Kafka Raft Metadata) replaces ZooKeeper entirely. Kafka manages its own metadata using the Raft consensus algorithm, built directly into the broker. There is no separate service. The cluster self-manages.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Kafka 3.9.x was the last release to support ZooKeeper.&lt;/strong&gt; If you're on Kafka 3.9 with ZooKeeper and want to upgrade to 4.0+, you must migrate to KRaft first using the migration tooling in 3.9. You cannot skip it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Kafka 4.0+ is KRaft-only, no exceptions.&lt;/strong&gt; The current stable release is &lt;a href="https://kafka.apache.org/blog/2026/05/22/apache-kafka-4.3.0-release-announcement/" rel="noopener noreferrer"&gt;4.3.0&lt;/a&gt; (May 2026).&lt;/p&gt;

&lt;p&gt;The practical impact for a new project: your Docker Compose is simpler. No ZooKeeper service. No &lt;code&gt;KAFKA_ZOOKEEPER_CONNECT&lt;/code&gt;. Just Kafka, configured to run as both broker and controller.&lt;/p&gt;




&lt;h2&gt;
  
  
  Running Kafka Locally: KRaft Docker Compose
&lt;/h2&gt;

&lt;p&gt;The Confluent Platform Docker images are the easiest way to run Kafka locally. Confluent Platform 8.1 maps to Kafka 4.1.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# docker-compose.yml — Kafka 4.1 (Confluent Platform 8.1.2), KRaft mode&lt;/span&gt;
&lt;span class="na"&gt;version&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;3.8'&lt;/span&gt;
&lt;span class="na"&gt;services&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;kafka&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;confluentinc/cp-kafka:8.1.2&lt;/span&gt;
    &lt;span class="na"&gt;container_name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kafka&lt;/span&gt;
    &lt;span class="na"&gt;hostname&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kafka&lt;/span&gt;
    &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;9092:9092"&lt;/span&gt;
    &lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_NODE_ID&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_PROCESS_ROLES&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;broker,controller&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_LISTENERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_ADVERTISED_LISTENERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PLAINTEXT://localhost:9092&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_LISTENER_SECURITY_PROTOCOL_MAP&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_CONTROLLER_LISTENER_NAMES&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;CONTROLLER&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_CONTROLLER_QUORUM_VOTERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;1@kafka:9093&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_TRANSACTION_STATE_LOG_MIN_ISR&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_AUTO_CREATE_TOPICS_ENABLE&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;true'&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_LOG_RETENTION_HOURS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;168&lt;/span&gt;
      &lt;span class="na"&gt;CLUSTER_ID&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;MkU3OEVBNTcwNTJENDM2Qk"&lt;/span&gt;
    &lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;kafka_data:/var/lib/kafka/data&lt;/span&gt;
    &lt;span class="na"&gt;healthcheck&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;test&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kafka-topics --bootstrap-server localhost:9092 --list || exit &lt;/span&gt;&lt;span class="m"&gt;1&lt;/span&gt;
      &lt;span class="na"&gt;interval&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;10s&lt;/span&gt;
      &lt;span class="na"&gt;timeout&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;10s&lt;/span&gt;
      &lt;span class="na"&gt;retries&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;

  &lt;span class="na"&gt;kafka-ui&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;provectuslabs/kafka-ui:latest&lt;/span&gt;
    &lt;span class="na"&gt;container_name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kafka-ui&lt;/span&gt;
    &lt;span class="na"&gt;depends_on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;kafka&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;condition&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;service_healthy&lt;/span&gt;
    &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;8080:8080"&lt;/span&gt;
    &lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_CLUSTERS_0_NAME&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;local&lt;/span&gt;
      &lt;span class="na"&gt;KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kafka:9092&lt;/span&gt;

&lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;kafka_data&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Start it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker compose up &lt;span class="nt"&gt;-d&lt;/span&gt;
docker compose logs &lt;span class="nt"&gt;-f&lt;/span&gt; kafka    &lt;span class="c"&gt;# watch for "started (kafka.server.KafkaServer)"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The Kafka UI at &lt;code&gt;http://localhost:8080&lt;/code&gt; shows topics, consumer groups, lag, and messages visually. Use it while you learn.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The ADVERTISED_LISTENERS gotcha (the most common Docker error):&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092&lt;/code&gt; works when your producer/consumer runs on the host machine. If your producer also runs inside Docker (a different container), use the service name instead:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;KAFKA_ADVERTISED_LISTENERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PLAINTEXT://kafka:9092&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The rule: &lt;code&gt;localhost&lt;/code&gt; for host-to-container access, &lt;code&gt;kafka&lt;/code&gt; (the service name) for container-to-container. Mixing them produces the error &lt;code&gt;kafka: client has run out of available brokers to talk to&lt;/code&gt; with no further explanation.&lt;/p&gt;

&lt;p&gt;For multi-service Docker setups (producer container + Kafka container + consumer container), add two listeners:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;KAFKA_LISTENERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PLAINTEXT://0.0.0.0:9092,PLAINTEXT_HOST://0.0.0.0:29092&lt;/span&gt;
&lt;span class="na"&gt;KAFKA_ADVERTISED_LISTENERS&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092&lt;/span&gt;
&lt;span class="na"&gt;KAFKA_LISTENER_SECURITY_PROTOCOL_MAP&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Internal containers connect on &lt;code&gt;kafka:9092&lt;/code&gt;. Your host machine connects on &lt;code&gt;localhost:29092&lt;/code&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  CLI: The Commands You'll Use Constantly
&lt;/h2&gt;

&lt;p&gt;Get into the container:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker &lt;span class="nb"&gt;exec&lt;/span&gt; &lt;span class="nt"&gt;-it&lt;/span&gt; kafka bash
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Topics:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Create a topic&lt;/span&gt;
kafka-topics &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--create&lt;/span&gt; &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--partitions&lt;/span&gt; 3 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--replication-factor&lt;/span&gt; 1

&lt;span class="c"&gt;# List all topics&lt;/span&gt;
kafka-topics &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="nt"&gt;--list&lt;/span&gt;

&lt;span class="c"&gt;# Describe a topic (partitions, leaders, replicas, ISR)&lt;/span&gt;
kafka-topics &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="nt"&gt;--describe&lt;/span&gt; &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events

&lt;span class="c"&gt;# Delete a topic&lt;/span&gt;
kafka-topics &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="nt"&gt;--delete&lt;/span&gt; &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events

&lt;span class="c"&gt;# Change retention to 1 day&lt;/span&gt;
kafka-configs &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--alter&lt;/span&gt; &lt;span class="nt"&gt;--entity-type&lt;/span&gt; topics &lt;span class="nt"&gt;--entity-name&lt;/span&gt; flight-events &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--add-config&lt;/span&gt; retention.ms&lt;span class="o"&gt;=&lt;/span&gt;86400000
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Quick produce/consume for testing:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Produce — type messages, Ctrl+C to stop&lt;/span&gt;
kafka-console-producer &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events

&lt;span class="c"&gt;# Consume from the beginning&lt;/span&gt;
kafka-console-consumer &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--from-beginning&lt;/span&gt;

&lt;span class="c"&gt;# Consume as part of a named group&lt;/span&gt;
kafka-console-consumer &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--group&lt;/span&gt; my-test-group &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--from-beginning&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Consumer group inspection (critical for debugging lag):&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# List all consumer groups&lt;/span&gt;
kafka-consumer-groups &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="nt"&gt;--list&lt;/span&gt;

&lt;span class="c"&gt;# Describe a group — shows current offset, log end offset, and lag per partition&lt;/span&gt;
kafka-consumer-groups &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--describe&lt;/span&gt; &lt;span class="nt"&gt;--group&lt;/span&gt; flight-processor

&lt;span class="c"&gt;# Output:&lt;/span&gt;
&lt;span class="c"&gt;# TOPIC          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG  CONSUMER-ID&lt;/span&gt;
&lt;span class="c"&gt;# flight-events  0          1234            1236            2    ...&lt;/span&gt;
&lt;span class="c"&gt;# flight-events  1          890             890             0    ...&lt;/span&gt;
&lt;span class="c"&gt;# flight-events  2          445             450             5    ...&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Resetting offsets:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Reset to beginning (reprocess all messages)&lt;/span&gt;
kafka-consumer-groups &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--group&lt;/span&gt; flight-processor &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--reset-offsets&lt;/span&gt; &lt;span class="nt"&gt;--to-earliest&lt;/span&gt; &lt;span class="nt"&gt;--execute&lt;/span&gt;

&lt;span class="c"&gt;# Reset to latest (skip existing, process only new messages)&lt;/span&gt;
kafka-consumer-groups &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--group&lt;/span&gt; flight-processor &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--reset-offsets&lt;/span&gt; &lt;span class="nt"&gt;--to-latest&lt;/span&gt; &lt;span class="nt"&gt;--execute&lt;/span&gt;

&lt;span class="c"&gt;# Reset to a specific datetime&lt;/span&gt;
kafka-consumer-groups &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--group&lt;/span&gt; flight-processor &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--topic&lt;/span&gt; flight-events &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--reset-offsets&lt;/span&gt; &lt;span class="nt"&gt;--to-datetime&lt;/span&gt; 2025-01-01T00:00:00.000 &lt;span class="nt"&gt;--execute&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Python: Producer Patterns with confluent-kafka
&lt;/h2&gt;

&lt;p&gt;Install:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;confluent-kafka&lt;span class="o"&gt;==&lt;/span&gt;2.14.0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Basic producer:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;confluent_kafka&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Producer&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;KafkaException&lt;/span&gt;

&lt;span class="n"&gt;KAFKA_CONFIG&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;bootstrap.servers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;localhost:9092&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;acks&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;all&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;                           &lt;span class="c1"&gt;# wait for all in-sync replicas
&lt;/span&gt;    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;enable.idempotence&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;              &lt;span class="c1"&gt;# exactly-once per partition, retries safe
&lt;/span&gt;    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;max.in.flight.requests.per.connection&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;retries&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;retry.backoff.ms&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;compression.type&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;snappy&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;batch.size&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;32768&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;                     &lt;span class="c1"&gt;# 32 KB batches
&lt;/span&gt;    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;linger.ms&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;                         &lt;span class="c1"&gt;# wait 10ms to fill a batch before sending
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;delivery_callback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Delivery failed: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Delivered to &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; [&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;partition&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;] @ offset &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;producer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Producer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;KAFKA_CONFIG&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;produce_message&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;producer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;produce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="n"&gt;callback&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;delivery_callback&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;producer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;poll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;    &lt;span class="c1"&gt;# trigger callbacks without blocking
&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;produce_batch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;records&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;key_field&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;record&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;records&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;record&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;key_field&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;''&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="n"&gt;producer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;produce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;record&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;callback&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;delivery_callback&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;producer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;flush&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;    &lt;span class="c1"&gt;# block until all messages confirmed delivered
&lt;/span&gt;    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Flushed &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;records&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; messages to &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Why &lt;code&gt;enable.idempotence=True&lt;/code&gt;:&lt;/strong&gt; Without it, a retry after a network timeout might deliver the same message twice because the broker received it but the acknowledgment was lost. With idempotence enabled, Kafka assigns each message a sequence number and deduplicates retries at the broker level. Always enable it unless you have a specific reason not to.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why &lt;code&gt;linger.ms=10&lt;/code&gt;:&lt;/strong&gt; By default, Kafka sends messages as soon as they're produced, one batch at a time. Setting &lt;code&gt;linger.ms&lt;/code&gt; tells the producer to wait up to 10ms for more messages before sending, filling batches more efficiently. For high-throughput pipelines this meaningfully reduces network round trips.&lt;/p&gt;




&lt;h2&gt;
  
  
  Python: Consumer Patterns
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;confluent_kafka&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Consumer&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;KafkaError&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;KafkaException&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;signal&lt;/span&gt;

&lt;span class="n"&gt;CONSUMER_CONFIG&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;bootstrap.servers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;localhost:9092&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;group.id&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;flight-processor&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;auto.offset.reset&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;earliest&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;     &lt;span class="c1"&gt;# start from the beginning if no committed offset
&lt;/span&gt;    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;enable.auto.commit&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;         &lt;span class="c1"&gt;# manual commit — never let Kafka auto-commit
&lt;/span&gt;    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;max.poll.interval.ms&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;300000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;      &lt;span class="c1"&gt;# 5 min max between polls before rebalance triggered
&lt;/span&gt;    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;session.timeout.ms&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;45000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;heartbeat.interval.ms&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="n"&gt;consumer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Consumer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;CONSUMER_CONFIG&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;subscribe&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;flight-events&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;

&lt;span class="n"&gt;running&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;shutdown&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;signum&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;frame&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;global&lt;/span&gt; &lt;span class="n"&gt;running&lt;/span&gt;
    &lt;span class="n"&gt;running&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;

&lt;span class="n"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SIGTERM&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;shutdown&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SIGINT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;shutdown&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="n"&gt;running&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;msg&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;poll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;1.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;continue&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;error&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;error&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;code&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;KafkaError&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_PARTITION_EOF&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="c1"&gt;# Reached end of partition — not an error, just informational
&lt;/span&gt;                &lt;span class="k"&gt;continue&lt;/span&gt;
            &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="nc"&gt;KafkaException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;error&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;

        &lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;key&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;decode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;key&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;
        &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;value&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;decode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;utf-8&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

        &lt;span class="c1"&gt;# Process the message
&lt;/span&gt;        &lt;span class="nf"&gt;process_event&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

        &lt;span class="c1"&gt;# Commit after processing — at-least-once delivery
&lt;/span&gt;        &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;commit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;finally&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;The &lt;code&gt;auto.offset.reset&lt;/code&gt; gotcha — the mistake everyone makes first:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;auto.offset.reset&lt;/code&gt; only applies when a consumer group has &lt;strong&gt;no committed offsets&lt;/strong&gt; — meaning it's brand new or its offsets were deleted. It does not override existing committed offsets.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;earliest&lt;/code&gt;: start from the oldest available message (what you want for a new consumer seeing existing data)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;latest&lt;/code&gt;: start from the newest message — everything before the consumer started is invisible&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The common mistake: you write a new consumer, set &lt;code&gt;auto.offset.reset='latest'&lt;/code&gt;, start it, and wonder why it receives nothing. It's because there were no new messages after it started. All the existing messages are already "past" the latest offset. Change to &lt;code&gt;earliest&lt;/code&gt; and reset the consumer group offsets, or use &lt;code&gt;--to-earliest&lt;/code&gt; on the CLI.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why &lt;code&gt;enable.auto.commit=False&lt;/code&gt;:&lt;/strong&gt; Auto-commit fires on a timer (every 5 seconds by default). If your process crashes after the commit but before processing finishes, messages are lost. If it crashes after processing but before the commit, you reprocess on restart. Manual commit after processing gives you at-least-once: possible duplicates, never lost. For idempotent sinks (PostgreSQL upserts), this is the right tradeoff.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Batch consumer (for loading into a database):&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Instead of committing after every single message, accumulate a batch and commit once:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;consume_and_load&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;batch_size&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;subscribe&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
    &lt;span class="n"&gt;batch&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;msg&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;poll&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;5.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;error&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
                &lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;value&lt;/span&gt;&lt;span class="p"&gt;()))&lt;/span&gt;

            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="n"&gt;batch_size&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="nf"&gt;load_to_database&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;    &lt;span class="c1"&gt;# your upsert logic
&lt;/span&gt;                &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;commit&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
                &lt;span class="n"&gt;batch&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
    &lt;span class="k"&gt;finally&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="nf"&gt;load_to_database&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;batch&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;commit&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This pattern is typical for Kafka-to-PostgreSQL pipelines: poll until you have 500 messages, bulk-insert with &lt;code&gt;ON CONFLICT DO NOTHING&lt;/code&gt;, commit, repeat.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Rebalance Problem and How KIP-848 Fixes It
&lt;/h2&gt;

&lt;p&gt;The most disruptive thing in a Kafka consumer setup is a rebalance. Before Kafka 4.0, every time a consumer joined or left a group, the broker would:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Tell all consumers to stop processing and revoke their partitions&lt;/li&gt;
&lt;li&gt;Wait for all consumers to acknowledge&lt;/li&gt;
&lt;li&gt;Reassign all partitions from scratch&lt;/li&gt;
&lt;li&gt;Tell consumers to resume&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For a consumer group with 20 consumers and 60 partitions, a single consumer restart triggered a full stop-the-world for every consumer. At high message volume, this caused visible processing gaps.&lt;/p&gt;

&lt;p&gt;The new rebalance protocol (KIP-848, GA in Kafka 4.0) works incrementally. Only the partitions that need to move actually move. Consumers that don't need to change keep processing. The impact of a single consumer restart is isolated to that consumer's partitions.&lt;/p&gt;

&lt;p&gt;To use the new protocol, set the &lt;code&gt;group.protocol&lt;/code&gt; consumer config:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;CONSUMER_CONFIG&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;bootstrap.servers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;localhost:9092&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;group.id&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;flight-processor&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;group.protocol&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;consumer&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;      &lt;span class="c1"&gt;# new KIP-848 protocol; 'classic' is the old one
&lt;/span&gt;    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;auto.offset.reset&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;earliest&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;enable.auto.commit&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;consumer&lt;/code&gt; protocol is the default in Kafka 4.0+. If you're connecting to a 4.0+ broker, it will be used automatically. If you're connecting to a 3.x broker, it falls back to &lt;code&gt;classic&lt;/code&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  Monitoring Consumer Lag
&lt;/h2&gt;

&lt;p&gt;Lag is the difference between the latest message in a partition and the last message your consumer committed. Lag = 0 means you're fully caught up. Lag growing means your consumer is slower than the producer.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;CLI:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kafka-consumer-groups &lt;span class="nt"&gt;--bootstrap-server&lt;/span&gt; localhost:9092 &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--describe&lt;/span&gt; &lt;span class="nt"&gt;--group&lt;/span&gt; flight-processor
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;LAG&lt;/code&gt; column is what you watch. If it's consistently growing, your consumer needs more threads, more instances, or a faster processing path.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Python:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;confluent_kafka.admin&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;AdminClient&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ConsumerGroupTopicPartitions&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;TopicPartition&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;confluent_kafka&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Consumer&lt;/span&gt;

&lt;span class="n"&gt;admin&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;AdminClient&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;bootstrap.servers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;localhost:9092&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;check_lag&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;group_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;num_partitions&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;partitions&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;TopicPartition&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;num_partitions&lt;/span&gt;&lt;span class="p"&gt;)]&lt;/span&gt;
    &lt;span class="n"&gt;fut&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;admin&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;list_consumer_group_offsets&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;ConsumerGroupTopicPartitions&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;group_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;partitions&lt;/span&gt;&lt;span class="p"&gt;)]&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;committed&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;fut&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;group_id&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="nf"&gt;result&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="n"&gt;topic_partitions&lt;/span&gt;

    &lt;span class="n"&gt;temp_consumer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Consumer&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;bootstrap.servers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;localhost:9092&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;group.id&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;_lag-check&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="n"&gt;lag_report&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{}&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;tp&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;committed&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;low&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;high&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;temp_consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_watermark_offsets&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="nc"&gt;TopicPartition&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;partition&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;lag&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;high&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;tp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;offset&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="n"&gt;low&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;lag_report&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;tp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;[&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;tp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;partition&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;]&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;lag&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;lag&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;10000&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;WARNING: high lag on &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;tp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;[&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;tp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;partition&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;]: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;lag&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;temp_consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;lag_report&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Common Errors and What They Actually Mean
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;"LEADER_NOT_AVAILABLE"&lt;/strong&gt;&lt;br&gt;
The topic was just created and leader election is still in progress. Wait 2–3 seconds and retry. This is not a real error for newly created topics.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Consumer receives no messages despite messages existing in the topic&lt;/strong&gt;&lt;br&gt;
Two causes: (1) &lt;code&gt;auto.offset.reset='latest'&lt;/code&gt; and the consumer started after those messages were written — reset offsets to &lt;code&gt;earliest&lt;/code&gt; for the group; (2) another consumer in the same group already committed those offsets — use a different &lt;code&gt;group.id&lt;/code&gt; or reset the group.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;"Rebalancing too often"&lt;/strong&gt;&lt;br&gt;
Your consumer is taking too long between &lt;code&gt;poll()&lt;/code&gt; calls. If processing one batch takes longer than &lt;code&gt;max.poll.interval.ms&lt;/code&gt; (default 5 minutes), Kafka assumes the consumer is dead and triggers a rebalance. Fix: increase &lt;code&gt;max.poll.interval.ms&lt;/code&gt;, process fewer records per poll, or move slow processing to a background thread.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;"Message too large" (broker rejects)&lt;/strong&gt;&lt;br&gt;
&lt;code&gt;message.max.bytes&lt;/code&gt; on the broker (default 1MB) is smaller than your message. Align &lt;code&gt;message.max.bytes&lt;/code&gt; on the broker and &lt;code&gt;max.request.size&lt;/code&gt; on the producer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# In Docker Compose:&lt;/span&gt;
&lt;span class="na"&gt;KAFKA_MESSAGE_MAX_BYTES&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;10485760&lt;/span&gt;         &lt;span class="c1"&gt;# 10 MB broker limit&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# In producer config:
&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;max.request.size&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;10485760&lt;/span&gt;             &lt;span class="c1"&gt;# must match
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Dead Letter Queue — when you can't fix a bad message:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Some messages will always fail processing — malformed JSON, missing fields, unexpected types. Don't let them block your pipeline. Route them to a DLQ topic:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;process_with_dlq&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;dlq_producer&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;value&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
        &lt;span class="nf"&gt;process_event&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;commit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;Exception&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;dlq_producer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;produce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;flight-events-dlq&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;key&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
            &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;value&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
            &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;error&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
                &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;original-topic&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
                &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;original-partition&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;partition&lt;/span&gt;&lt;span class="p"&gt;()).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
                &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;original-offset&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;offset&lt;/span&gt;&lt;span class="p"&gt;()).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
            &lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;dlq_producer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;flush&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;commit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;    &lt;span class="c1"&gt;# commit anyway — bad message goes to DLQ, not back to queue
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The DLQ lets you inspect failed messages later, fix the processing logic, and replay them.&lt;/p&gt;




&lt;h2&gt;
  
  
  Kafka with Airflow
&lt;/h2&gt;

&lt;p&gt;The &lt;code&gt;apache-airflow-providers-apache-kafka&lt;/code&gt; package adds native Kafka operators and sensors to Airflow:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;apache-airflow-providers-apache-kafka
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Add a Kafka connection in the Airflow UI: Conn Type → Apache Kafka, Bootstrap Servers → &lt;code&gt;kafka:9092&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;airflow.decorators&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dag&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;task&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;airflow.providers.apache.kafka.sensors.kafka&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;AwaitMessageSensor&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;airflow.providers.apache.kafka.operators.consume&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;ConsumeFromTopicOperator&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;

&lt;span class="nd"&gt;@dag&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;schedule&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;@daily&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;start_date&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2025&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;kafka_pipeline&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;

    &lt;span class="c1"&gt;# Wait for a message matching a condition before proceeding
&lt;/span&gt;    &lt;span class="n"&gt;wait_for_data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;AwaitMessageSensor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;task_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;wait_for_flight_data&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;kafka_config_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;kafka_default&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;topics&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;flight-events&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
        &lt;span class="n"&gt;apply_function&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;my_module.check_message&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;poll_timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;1.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;poll_interval&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# Consume and process a batch from the topic
&lt;/span&gt;    &lt;span class="n"&gt;consume&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ConsumeFromTopicOperator&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;task_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;consume_flights&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;kafka_config_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;kafka_default&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;topics&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;flight-events&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
        &lt;span class="n"&gt;apply_function&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;my_module.process_message&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;max_messages&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;max_batch_size&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;commit_cadence&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;end_of_batch&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;wait_for_data&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;consume&lt;/span&gt;

&lt;span class="nf"&gt;kafka_pipeline&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# my_module.py
&lt;/span&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;check_message&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;value&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;ready&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;process_message&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;value&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
    &lt;span class="c1"&gt;# transform and return
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;processed_at&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;ts&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For simpler use cases — just producing from a Python task — use the &lt;code&gt;confluent_kafka&lt;/code&gt; library directly inside a &lt;code&gt;@task&lt;/code&gt; function:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;confluent_kafka&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Producer&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;

&lt;span class="nd"&gt;@task&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;produce_events&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;records&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Producer&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;bootstrap.servers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;kafka:9092&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;acks&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;all&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;record&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;records&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;produce&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;flight-events&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;record&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
    &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;flush&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;records&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  &lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fj1gb8zci3a8aitbqjt01.jpg" alt=" "&gt;
&lt;/h2&gt;

&lt;h2&gt;
  
  
  Quick Reference: Key Config Decisions
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Config&lt;/th&gt;
&lt;th&gt;Recommended value&lt;/th&gt;
&lt;th&gt;Why&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;acks&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;all&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Safest — waits for all in-sync replicas&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;enable.idempotence&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;True&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Deduplicates retries at broker level&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;enable.auto.commit&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;False&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Manual commit for reliable at-least-once&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;auto.offset.reset&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;earliest&lt;/code&gt; (new consumers)&lt;/td&gt;
&lt;td&gt;Don't silently miss existing messages&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;compression.type&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;snappy&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Good ratio, fast encode/decode&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;linger.ms&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Fill batches efficiently; minimal latency impact&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;group.protocol&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;consumer&lt;/code&gt; (Kafka 4.0+)&lt;/td&gt;
&lt;td&gt;New incremental rebalance protocol&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  Version Reference (June 2026)
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Version&lt;/th&gt;
&lt;th&gt;Notes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Apache Kafka&lt;/td&gt;
&lt;td&gt;&lt;a href="https://kafka.apache.org/blog/2026/05/22/apache-kafka-4.3.0-release-announcement/" rel="noopener noreferrer"&gt;4.3.0&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;Latest stable (May 2026)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Confluent Platform&lt;/td&gt;
&lt;td&gt;8.1.2&lt;/td&gt;
&lt;td&gt;Maps to Kafka 4.1; use for Docker images&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;confluent-kafka (Python)&lt;/td&gt;
&lt;td&gt;2.14.0&lt;/td&gt;
&lt;td&gt;Preferred Python client&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Last ZooKeeper release&lt;/td&gt;
&lt;td&gt;3.9.2&lt;/td&gt;
&lt;td&gt;Dead end — do not build new projects on it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Java requirement (brokers)&lt;/td&gt;
&lt;td&gt;17+&lt;/td&gt;
&lt;td&gt;From Kafka 4.0 onward&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;If you're migrating an existing ZooKeeper-based cluster, the path is: upgrade to Kafka 3.9 first (the bridge version with the best migration tooling), run the ZooKeeper-to-KRaft migration, then upgrade to 4.0+.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Follow me on &lt;a href="https://dev.to/de_clerke"&gt;dev.to&lt;/a&gt; for more data engineering content, or browse the project code at &lt;a href="https://github.com/declerke" rel="noopener noreferrer"&gt;github.com/declerke&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>dataengineering</category>
      <category>kafka</category>
      <category>airflow</category>
      <category>streaming</category>
    </item>
    <item>
      <title>Migrando uma Aplicação Vue 2 Legada de Webpack 2 para Vite: Um Guia Prático Baseado em Problemas Reais</title>
      <dc:creator>Camila Rody</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:29:31 +0000</pubDate>
      <link>https://dev.to/camilasrody/migrando-uma-aplicacao-vue-2-legada-de-webpack-2-para-vite-um-guia-pratico-baseado-em-problemas-1k1e</link>
      <guid>https://dev.to/camilasrody/migrando-uma-aplicacao-vue-2-legada-de-webpack-2-para-vite-um-guia-pratico-baseado-em-problemas-1k1e</guid>
      <description>&lt;p&gt;Quando falamos sobre migração para Vite, a maioria dos artigos parte de um cenário ideal: projetos relativamente modernos, dependências atualizadas, versões recentes do Node e uma arquitetura preparada para evoluir. Na prática, porém, muitas empresas ainda mantêm aplicações Vue 2 que surgiram anos atrás, construídas sobre Webpack 2, carregando uma grande quantidade de bibliotecas descontinuadas, loaders obsoletos e configurações que foram sendo adaptadas por diferentes equipes ao longo do tempo.&lt;/p&gt;

&lt;p&gt;Foi exatamente esse cenário que encontrei. O objetivo não era migrar para Vue 3, reescrever a aplicação ou modernizar toda a stack de uma vez. O desafio era muito mais delicado: substituir o Webpack 2 por Vite mantendo a aplicação funcionando, preservando compatibilidade com bibliotecas legadas e sem interromper o desenvolvimento do produto.&lt;/p&gt;

&lt;p&gt;Ao longo desse processo, ficou evidente que a maior dificuldade não estava na configuração do Vite em si. O verdadeiro desafio era descobrir tudo aquilo que o Webpack estava fazendo silenciosamente há anos e garantir que esse comportamento fosse reproduzido no novo ambiente.&lt;/p&gt;

&lt;h2&gt;
  
  
  Antes de Instalar o Vite, Entenda o Que o Webpack Faz Pelo Seu Projeto
&lt;/h2&gt;

&lt;p&gt;Um erro bastante comum é iniciar a migração instalando o Vite e tentando fazer a aplicação subir imediatamente. Em aplicações pequenas isso pode funcionar. Em sistemas legados, normalmente gera uma sequência interminável de erros difíceis de rastrear.&lt;/p&gt;

&lt;p&gt;Antes de qualquer alteração, é fundamental mapear as responsabilidades atuais do Webpack. Em muitos projetos ele não atua apenas como bundler. Frequentemente também é responsável por resolver aliases, processar imagens, transpilar JavaScript, injetar variáveis de ambiente, tratar arquivos Sass, gerar chunks dinâmicos e fornecer polyfills para APIs que os navegadores não implementam nativamente.&lt;/p&gt;

&lt;p&gt;Por isso, o primeiro passo deve ser uma auditoria da configuração existente.&lt;/p&gt;

&lt;p&gt;Analise cuidadosamente arquivos como:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;webpack.config.js
webpack.dev.js
webpack.prod.js
.babelrc
package.json
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Durante essa análise, documente:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Aliases utilizados&lt;/li&gt;
&lt;li&gt;Loaders instalados&lt;/li&gt;
&lt;li&gt;Plugins do Webpack&lt;/li&gt;
&lt;li&gt;Configurações do Babel&lt;/li&gt;
&lt;li&gt;Variáveis de ambiente&lt;/li&gt;
&lt;li&gt;Polyfills&lt;/li&gt;
&lt;li&gt;Estratégias de code splitting&lt;/li&gt;
&lt;li&gt;Tratamento de assets&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Essa documentação servirá como roteiro para toda a migração.&lt;/p&gt;

&lt;h2&gt;
  
  
  Estabilize a Versão do Node Antes de Trocar o Bundler
&lt;/h2&gt;

&lt;p&gt;Em projetos antigos, é comum encontrar versões do Node extremamente defasadas. Muitas vezes o sistema foi desenvolvido utilizando Node 8, Node 10 ou Node 12, e diversas dependências foram escritas considerando limitações dessas versões.&lt;/p&gt;

&lt;p&gt;Tentar atualizar Node e substituir Webpack ao mesmo tempo costuma gerar um problema clássico: quando algo quebra, torna-se difícil descobrir a causa.&lt;/p&gt;

&lt;p&gt;Por isso, recomendo separar completamente essas etapas.&lt;/p&gt;

&lt;p&gt;Primeiro, valide até qual versão do Node a aplicação consegue evoluir sem alterações significativas. Durante essa fase você provavelmente encontrará dependências problemáticas como:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;node-sass
extract-text-webpack-plugin
uglify-js
babel-preset-es2015
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Em muitos casos, a melhor decisão não é atualizar essas bibliotecas, mas congelá-las temporariamente para reduzir riscos.&lt;/p&gt;

&lt;p&gt;O objetivo inicial não é modernizar tudo. É criar um ambiente estável sobre o qual a migração poderá acontecer.&lt;/p&gt;

&lt;h2&gt;
  
  
  Instale o Vite Sem Remover o Webpack
&lt;/h2&gt;

&lt;p&gt;Uma das decisões mais importantes durante minha migração foi evitar uma substituição imediata.&lt;/p&gt;

&lt;p&gt;Em vez de remover o Webpack, o Vite foi introduzido paralelamente.&lt;/p&gt;

&lt;p&gt;A instalação básica para Vue 2 é relativamente simples:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npm &lt;span class="nb"&gt;install &lt;/span&gt;vite vite-plugin-vue2 &lt;span class="nt"&gt;--save-dev&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Depois disso, crie um arquivo &lt;code&gt;vite.config.js&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;defineConfig&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;vite&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;createVuePlugin&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;vite-plugin-vue2&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;

&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="k"&gt;default&lt;/span&gt; &lt;span class="nf"&gt;defineConfig&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
  &lt;span class="na"&gt;plugins&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="nf"&gt;createVuePlugin&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
  &lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;})&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Nesse momento o Webpack continua existindo normalmente.&lt;/p&gt;

&lt;p&gt;Os scripts passam a coexistir:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"scripts"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"dev"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"webpack-dev-server"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"dev:vite"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"vite"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"build"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"webpack"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"build:vite"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"vite build"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Essa abordagem permite comparar comportamentos, validar funcionalidades e criar um plano de rollback caso algo inesperado aconteça.&lt;/p&gt;

&lt;h2&gt;
  
  
  Migrando os Aliases
&lt;/h2&gt;

&lt;p&gt;Quase toda aplicação Vue 2 de médio ou grande porte utiliza aliases para evitar imports relativos extensos.&lt;/p&gt;

&lt;p&gt;Um exemplo comum no Webpack:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;alias&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;@&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;__dirname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;src&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;@components&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;__dirname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;src/components&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Esses aliases precisam ser reproduzidos no Vite.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;path&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;

&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="k"&gt;default&lt;/span&gt; &lt;span class="nf"&gt;defineConfig&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
  &lt;span class="na"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;alias&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;@&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;__dirname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;./src&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
      &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;@components&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;__dirname&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;./src/components&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;})&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Pode parecer um detalhe pequeno, mas em aplicações grandes isso afeta centenas ou milhares de imports.&lt;/p&gt;

&lt;p&gt;Por isso, essa costuma ser uma das primeiras configurações que implemento durante a migração.&lt;/p&gt;

&lt;h2&gt;
  
  
  Corrigindo Imports Dinâmicos Que Funcionavam Apenas no Webpack
&lt;/h2&gt;

&lt;p&gt;Uma das incompatibilidades mais frequentes está relacionada aos famosos &lt;code&gt;require()&lt;/code&gt; dinâmicos.&lt;/p&gt;

&lt;p&gt;Durante anos o Webpack permitiu padrões como:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;component&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;require&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;./pages/&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="nx"&gt;pageName&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;ou&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;service&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;require&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;path&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;O problema é que o Vite utiliza análise estática para resolver módulos e não consegue interpretar caminhos gerados dinamicamente em tempo de execução.&lt;/p&gt;

&lt;p&gt;Nesses casos é necessário utilizar &lt;code&gt;import.meta.glob()&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Antes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;component&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;require&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`./pages/&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;pageName&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;.vue`&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Depois:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;pages&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;meta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;glob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;./pages/*.vue&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;component&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;pages&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;`./pages/&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;pageName&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;.vue`&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Em projetos legados, essa costuma ser uma das etapas mais trabalhosas da migração, pois exige revisão de arquitetura em diversos pontos da aplicação.&lt;/p&gt;

&lt;h2&gt;
  
  
  Migrando Variáveis de Ambiente
&lt;/h2&gt;

&lt;p&gt;Outro ajuste obrigatório envolve as variáveis de ambiente.&lt;/p&gt;

&lt;p&gt;Projetos baseados em Webpack normalmente utilizam:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;API_URL&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No Vite o padrão é diferente.&lt;/p&gt;

&lt;p&gt;Antes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;apiUrl&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;process&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;API_URL&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Depois:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;apiUrl&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;meta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;env&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;VITE_API_URL&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Além disso, o arquivo &lt;code&gt;.env&lt;/code&gt; precisa ser adaptado:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;VITE_API_URL=https://api.minhaempresa.com
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Um detalhe importante é que o Vite só expõe para o frontend variáveis prefixadas com:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;VITE_
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Ignorar esse detalhe pode gerar erros difíceis de identificar em ambientes de homologação e produção.&lt;/p&gt;

&lt;h2&gt;
  
  
  Lidando com Bibliotecas CommonJS e Dependências Abandonadas
&lt;/h2&gt;

&lt;p&gt;Uma das maiores preocupações em sistemas legados é a quantidade de bibliotecas que não acompanham mais a evolução do ecossistema JavaScript.&lt;/p&gt;

&lt;p&gt;Muitas delas ainda utilizam:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;module&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;exports&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;library&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;ou&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;exports&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;library&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Embora o Vite possua mecanismos de compatibilidade, alguns pacotes exigem otimizações explícitas:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="k"&gt;default&lt;/span&gt; &lt;span class="nf"&gt;defineConfig&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
  &lt;span class="na"&gt;optimizeDeps&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;include&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
      &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;legacy-library&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;old-plugin&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;})&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Quando isso não é suficiente, uma alternativa eficiente é criar wrappers de compatibilidade para isolar o problema e evitar alterações massivas no código.&lt;/p&gt;

&lt;p&gt;Essa abordagem reduz significativamente o impacto da migração.&lt;/p&gt;

&lt;h2&gt;
  
  
  Polyfills: O Problema Que Geralmente Só Aparece Depois
&lt;/h2&gt;

&lt;p&gt;Durante anos o Webpack forneceu diversos polyfills automaticamente.&lt;/p&gt;

&lt;p&gt;Quando migramos para Vite, muitos deles desaparecem.&lt;/p&gt;

&lt;p&gt;É comum encontrar erros como:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Buffer is not defined
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;process is not defined
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;global is not defined
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Um exemplo de correção para Buffer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;Buffer&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;buffer&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;

&lt;span class="nb"&gt;window&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;Buffer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;Buffer&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Esses problemas normalmente surgem apenas durante testes mais profundos, por isso é importante validar cuidadosamente todas as funcionalidades críticas da aplicação.&lt;/p&gt;

&lt;h2&gt;
  
  
  Revisando Loaders e Assets
&lt;/h2&gt;

&lt;p&gt;Grande parte da complexidade de projetos Webpack antigos está concentrada nos loaders.&lt;/p&gt;

&lt;p&gt;É comum encontrar configurações envolvendo:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;file-loader&lt;/li&gt;
&lt;li&gt;url-loader&lt;/li&gt;
&lt;li&gt;raw-loader&lt;/li&gt;
&lt;li&gt;svg-loader&lt;/li&gt;
&lt;li&gt;font-loader&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A boa notícia é que muitos desses recursos são tratados nativamente pelo Vite.&lt;/p&gt;

&lt;p&gt;Entretanto, não assuma que todos os comportamentos serão reproduzidos automaticamente.&lt;/p&gt;

&lt;p&gt;Sempre valide:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;SVGs inline&lt;/li&gt;
&lt;li&gt;Fontes customizadas&lt;/li&gt;
&lt;li&gt;Arquivos estáticos&lt;/li&gt;
&lt;li&gt;Assets importados dinamicamente&lt;/li&gt;
&lt;li&gt;Recursos utilizados por bibliotecas de terceiros&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Muitos bugs de produção surgem exatamente nessa etapa.&lt;/p&gt;

&lt;h2&gt;
  
  
  Validando a Build de Produção
&lt;/h2&gt;

&lt;p&gt;Fazer a aplicação funcionar em desenvolvimento não significa que a migração terminou.&lt;/p&gt;

&lt;p&gt;Na verdade, a parte mais importante começa depois.&lt;/p&gt;

&lt;p&gt;Uma validação completa deve incluir:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Navegação entre rotas&lt;/li&gt;
&lt;li&gt;Lazy loading&lt;/li&gt;
&lt;li&gt;Autenticação&lt;/li&gt;
&lt;li&gt;Upload de arquivos&lt;/li&gt;
&lt;li&gt;Integrações externas&lt;/li&gt;
&lt;li&gt;Variáveis de ambiente&lt;/li&gt;
&lt;li&gt;Source maps&lt;/li&gt;
&lt;li&gt;Geração de bundles&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Além disso, recomendo comparar o comportamento da aplicação construída pelo Webpack e pelo Vite simultaneamente durante algum tempo.&lt;/p&gt;

&lt;p&gt;Essa estratégia facilita a identificação de regressões e reduz significativamente os riscos da implantação.&lt;/p&gt;

&lt;h2&gt;
  
  
  Conclusão
&lt;/h2&gt;

&lt;p&gt;Depois de concluir essa migração, uma percepção ficou muito clara para mim: trocar Webpack por Vite é a parte fácil.&lt;/p&gt;

&lt;p&gt;O trabalho real está em compreender todas as decisões arquiteturais acumuladas ao longo dos anos e identificar quais delas dependem diretamente do comportamento do Webpack.&lt;/p&gt;

&lt;p&gt;Aliases, imports dinâmicos, polyfills, bibliotecas CommonJS, loaders antigos e configurações herdadas raramente aparecem na documentação do projeto. Muitas vezes, só descobrimos sua importância quando algo deixa de funcionar.&lt;/p&gt;

&lt;p&gt;Por isso, a melhor estratégia não é migrar rápido. É migrar de forma controlada, mantendo Webpack e Vite coexistindo durante o processo, validando cada etapa e reduzindo ao máximo a quantidade de variáveis envolvidas.&lt;/p&gt;

&lt;p&gt;Em aplicações Vue 2 legadas, o sucesso da migração não está em instalar uma ferramenta nova. Está em conseguir modernizar a infraestrutura sem alterar o comportamento que o sistema construiu ao longo dos anos.&lt;/p&gt;

</description>
      <category>webpack</category>
      <category>vite</category>
      <category>migration</category>
      <category>vue</category>
    </item>
    <item>
      <title>5 Angular Features Developers Should Actually Pay Attention to in 2026</title>
      <dc:creator>Niharika Pujari</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:25:47 +0000</pubDate>
      <link>https://dev.to/niharikapujari/5-angular-features-developers-should-actually-pay-attention-to-in-2026-1735</link>
      <guid>https://dev.to/niharikapujari/5-angular-features-developers-should-actually-pay-attention-to-in-2026-1735</guid>
      <description>&lt;h2&gt;
  
  
  Introduction
&lt;/h2&gt;

&lt;p&gt;Angular has changed a lot over the past few releases.&lt;/p&gt;

&lt;p&gt;Between Signals, standalone components, modern template syntax, and rendering improvements, Angular feels very different from the Angular many of us learned a few years ago.&lt;/p&gt;

&lt;p&gt;Some features are incremental. Others genuinely change how we build applications.&lt;/p&gt;

&lt;h3&gt;
  
  
  Here are 5 Angular features that actually matter in 2026 and are worth paying attention to if you're building modern Angular apps.
&lt;/h3&gt;




&lt;h3&gt;
  
  
  1. Signals Are Changing How Angular State Management Works
&lt;/h3&gt;

&lt;p&gt;Signals are probably the biggest shift in Angular’s mental model in years.&lt;/p&gt;

&lt;p&gt;Instead of relying heavily on:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;BehaviorSubject&lt;/li&gt;
&lt;li&gt;manual subscriptions&lt;/li&gt;
&lt;li&gt;async pipes everywhere&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Angular now provides a much simpler reactive primitive:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="nx"&gt;count&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="nf"&gt;increment&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;count&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;v&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;v&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And in the template:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;p&amp;gt;&lt;/span&gt;{{ count() }}&lt;span class="nt"&gt;&amp;lt;/p&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;What makes Signals interesting is that Angular now tracks dependencies automatically and updates only what actually changed.&lt;/p&gt;

&lt;p&gt;For UI state:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;loading flags&lt;/li&gt;
&lt;li&gt;counters&lt;/li&gt;
&lt;li&gt;filters&lt;/li&gt;
&lt;li&gt;selected items&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Signals make Angular code feel dramatically simpler.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Standalone Components Are Becoming the Default
&lt;/h3&gt;

&lt;p&gt;Angular spent years being heavily module-based.&lt;/p&gt;

&lt;p&gt;Now?&lt;br&gt;
Most new Angular apps are moving toward standalone components.&lt;/p&gt;

&lt;p&gt;Example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="p"&gt;@&lt;/span&gt;&lt;span class="nd"&gt;Component&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
  &lt;span class="na"&gt;selector&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;app-home&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;standalone&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;templateUrl&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;./home.component.html&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
&lt;span class="p"&gt;})&lt;/span&gt;
&lt;span class="k"&gt;export&lt;/span&gt; &lt;span class="kd"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;HomeComponent&lt;/span&gt; &lt;span class="p"&gt;{}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Why this matters:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;less boilerplate&lt;/li&gt;
&lt;li&gt;simpler project structure&lt;/li&gt;
&lt;li&gt;easier lazy loading&lt;/li&gt;
&lt;li&gt;cleaner onboarding for new developers&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Honestly, after working with standalone components for a while, going back to large module trees feels painful.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. The New Template Syntax Makes Angular Templates Much Cleaner
&lt;/h3&gt;

&lt;p&gt;Angular’s newer control flow syntax is one of those features that feels small until you actually use it.&lt;/p&gt;

&lt;p&gt;Instead of:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;div&lt;/span&gt; &lt;span class="na"&gt;*ngIf=&lt;/span&gt;&lt;span class="s"&gt;"isLoggedIn"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You now write:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="p"&gt;@&lt;/span&gt;&lt;span class="nd"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;isLoggedIn&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;div&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="nx"&gt;Welcome&lt;/span&gt; &lt;span class="nx"&gt;back&lt;/span&gt;&lt;span class="o"&gt;!&amp;lt;&lt;/span&gt;&lt;span class="sr"&gt;/div&lt;/span&gt;&lt;span class="err"&gt;&amp;gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Same for loops:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="nx"&gt;typescript&lt;/span&gt;
&lt;span class="p"&gt;@&lt;/span&gt;&lt;span class="nd"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;product&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="nx"&gt;products&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nx"&gt;track&lt;/span&gt; &lt;span class="nx"&gt;product&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;li&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;{{&lt;/span&gt; &lt;span class="nx"&gt;product&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt; &lt;span class="p"&gt;}}&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="sr"&gt;/li&lt;/span&gt;&lt;span class="err"&gt;&amp;gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This makes templates:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;easier to read&lt;/li&gt;
&lt;li&gt;closer to normal programming logic&lt;/li&gt;
&lt;li&gt;less cluttered&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It also works really nicely alongside Signals.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Angular Hydration Improved SSR Performance Significantly
&lt;/h3&gt;

&lt;p&gt;Server-side rendering has become much more important for:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;SEO&lt;/li&gt;
&lt;li&gt;performance&lt;/li&gt;
&lt;li&gt;Core Web Vitals&lt;/li&gt;
&lt;li&gt;perceived loading speed&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Angular’s hydration improvements now allow apps to reuse server-rendered DOM instead of fully re-rendering everything on the client.&lt;/p&gt;

&lt;p&gt;For users, this means:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;faster page loads&lt;/li&gt;
&lt;li&gt;smoother startup experience&lt;/li&gt;
&lt;li&gt;less UI flickering&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For developers, SSR setups are becoming much more practical than they used to be.&lt;/p&gt;

&lt;p&gt;Especially for content-heavy or enterprise apps, this is becoming increasingly relevant.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Angular Developer Experience Feels Much Better Now
&lt;/h3&gt;

&lt;p&gt;This one is harder to measure, but honestly important.&lt;/p&gt;

&lt;p&gt;Modern Angular feels:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;lighter&lt;/li&gt;
&lt;li&gt;more reactive&lt;/li&gt;
&lt;li&gt;less verbose&lt;/li&gt;
&lt;li&gt;easier to reason about&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A few years ago, Angular sometimes felt overly complex for smaller applications.&lt;/p&gt;

&lt;p&gt;Today:&lt;/p&gt;

&lt;p&gt;Signals reduce RxJS overload&lt;br&gt;
standalone components reduce structure overhead&lt;br&gt;
modern templates reduce clutter&lt;/p&gt;

&lt;p&gt;The framework is clearly moving toward a simpler developer experience without losing the power Angular is known for.&lt;/p&gt;

&lt;h2&gt;
  
  
  Final Thoughts
&lt;/h2&gt;

&lt;p&gt;Angular in 2026 feels very different from Angular a few years ago.&lt;/p&gt;

&lt;p&gt;The framework is evolving toward:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;fine-grained reactivity&lt;/li&gt;
&lt;li&gt;simpler APIs&lt;/li&gt;
&lt;li&gt;cleaner templates&lt;/li&gt;
&lt;li&gt;better performance&lt;/li&gt;
&lt;li&gt;less boilerplate&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The biggest takeaway for me:&lt;/p&gt;

&lt;p&gt;Angular is becoming easier to work with while still remaining extremely powerful for large-scale applications. And honestly, that's probably the direction many frontend developers were hoping for.&lt;/p&gt;

</description>
      <category>angular</category>
      <category>frontend</category>
      <category>webdev</category>
      <category>typescript</category>
    </item>
    <item>
      <title>How I built a Go middleware for Stripe-style idempotency-key handling</title>
      <dc:creator>Eben</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:22:00 +0000</pubDate>
      <link>https://dev.to/eben-vranken/how-i-built-a-go-middleware-for-stripe-style-idempotency-key-handling-4nlh</link>
      <guid>https://dev.to/eben-vranken/how-i-built-a-go-middleware-for-stripe-style-idempotency-key-handling-4nlh</guid>
      <description>&lt;p&gt;Retries in payment and order APIs are a classic footgun. Your client times out, retries the request, and you've just charged someone twice. The fix is idempotency-key handling, but getting it right is harder than it looks.&lt;/p&gt;

&lt;h2&gt;
  
  
  The naive approach breaks under load
&lt;/h2&gt;

&lt;p&gt;The obvious solution is Redis SETNX: claim a key before running the handler, release it after. Works fine on the happy path. Breaks in at least three ways:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Two identical requests arrive simultaneously before either has claimed the key. Both get through and execute.&lt;/li&gt;
&lt;li&gt;Your handler panics or returns an error. The lock never gets released.&lt;/li&gt;
&lt;li&gt;A client retries with a slightly different payload (a timestamp in the body, a different amount). You execute both and silently diverge.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Idempotency is one of those things that feels solved until you actually test it under concurrency.&lt;/p&gt;

&lt;h2&gt;
  
  
  What idempo does
&lt;/h2&gt;

&lt;p&gt;idempo is a &lt;code&gt;net/http middleware&lt;/code&gt; that implements the &lt;a href="https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/" rel="noopener noreferrer"&gt;IETF Idempotency-Key draft&lt;/a&gt; with Stripe-compatible semantics. A client sends a unique Idempotency-Key header with a request. The middleware:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Claims the key atomically before the handler runs&lt;/li&gt;
&lt;li&gt;If the key is new, runs the handler and stores the full response (status, headers, body)&lt;/li&gt;
&lt;li&gt;If the same key arrives again, replays the stored response without running the handler, and adds Idempotency-Replayed: true so you know it happened&lt;/li&gt;
&lt;li&gt;If the key is still in flight: 409 Conflict, same as Stripe&lt;/li&gt;
&lt;li&gt;If the key is reused with a different payload: 422 Unprocessable Entity&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Wiring it up
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="n"&gt;store&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;inmem&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;New&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;24&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Hour&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Minute&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;mw&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;idempo&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;New&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;idempo&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Options&lt;/span&gt;&lt;span class="p"&gt;{})&lt;/span&gt;

&lt;span class="n"&gt;mux&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NewServeMux&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;mux&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;HandleFunc&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"POST /charge"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;chargeHandler&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ListenAndServe&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;":8080"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;mw&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Handler&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mux&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="n"&gt;Because&lt;/span&gt; &lt;span class="n"&gt;it&lt;/span&gt;&lt;span class="err"&gt;'&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="n"&gt;just&lt;/span&gt; &lt;span class="k"&gt;func&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Handler&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Handler&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;it&lt;/span&gt; &lt;span class="n"&gt;drops&lt;/span&gt; &lt;span class="n"&gt;into&lt;/span&gt; &lt;span class="n"&gt;any&lt;/span&gt; &lt;span class="n"&gt;router&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;


&lt;span class="c"&gt;// chi&lt;/span&gt;
&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;chi&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NewRouter&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Use&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mw&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Handler&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Pluggable backends
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="c"&gt;// In-memory (single instance or testing)&lt;/span&gt;
&lt;span class="n"&gt;store&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;inmem&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;New&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;24&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Hour&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Minute&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c"&gt;// Redis (distributed)&lt;/span&gt;
&lt;span class="n"&gt;store&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;redis&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;New&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;goredis&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Options&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;Addr&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="s"&gt;"localhost:6379"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="m"&gt;24&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Hour&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Minute&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c"&gt;// Postgres (durable, ACID)&lt;/span&gt;
&lt;span class="n"&gt;pg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;RunMigration&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;connStr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;pg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;New&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;connStr&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="m"&gt;24&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Hour&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Minute&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each backend enforces exactly-once execution at the storage layer: a mutex in memory, an atomic Lua script in Redis, and INSERT ... ON CONFLICT in Postgres.&lt;/p&gt;

&lt;h2&gt;
  
  
  The concurrency guarantee
&lt;/h2&gt;

&lt;p&gt;The test suite fires 50 simultaneous identical requests and asserts the handler ran exactly once. The whole suite runs under -race in CI. If you've been burned by a race between SETNX and the actual handler execution, this is the part that matters.&lt;/p&gt;

&lt;h2&gt;
  
  
  One design decision worth calling out
&lt;/h2&gt;

&lt;p&gt;When a duplicate arrives while the first request is still in flight, idempo returns 409 rather than blocking and waiting to replay. This matches Stripe's behavior.&lt;/p&gt;

&lt;p&gt;My reasoning: holding connections open while waiting for an upstream to finish is a resource problem that compounds under load. Better to push the retry logic back to the client where it belongs. That said, if you need blocking behavior for your use case, the Store interface is just three methods (Claim, Complete, Abandon) and you can implement it yourself.&lt;/p&gt;

&lt;h2&gt;
  
  
  Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;GitHub: &lt;a href="https://github.com/eben-vranken/idempo" rel="noopener noreferrer"&gt;https://github.com/eben-vranken/idempo&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Docs: &lt;a href="https://eben-vranken.github.io/idempo-docs/" rel="noopener noreferrer"&gt;https://eben-vranken.github.io/idempo-docs/&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;pkg.go.dev: &lt;a href="https://pkg.go.dev/github.com/eben-vranken/idempo" rel="noopener noreferrer"&gt;https://pkg.go.dev/github.com/eben-vranken/idempo&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Curious what edge cases people have run into trying to implement this themselves.&lt;/p&gt;

</description>
      <category>go</category>
      <category>webdev</category>
      <category>opensource</category>
      <category>backend</category>
    </item>
    <item>
      <title>Why My Handwriting Font Didn't Show Up in Microsoft Word</title>
      <dc:creator>ramya Thirunavukkarasu</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:19:37 +0000</pubDate>
      <link>https://dev.to/ramya_thirunavukkarasu_0a/why-my-handwriting-font-didnt-show-up-in-microsoft-word-279j</link>
      <guid>https://dev.to/ramya_thirunavukkarasu_0a/why-my-handwriting-font-didnt-show-up-in-microsoft-word-279j</guid>
      <description>&lt;p&gt;In my previous blog, I explained how I converted my handwritten characters into a TrueType font using Python and FontForge. After generating the font file, I expected the process to be complete.&lt;br&gt;
The script executed successfully, the .ttf file was created, and Windows installed the font without any errors. Everything appeared to be working as expected.&lt;br&gt;
However, when I opened Microsoft Word to test the font, it was nowhere to be found.&lt;br&gt;
This was an unexpected result. Since the font had been generated and installed successfully, I initially assumed the issue was related to Microsoft Word or the Windows font cache. After several attempts, including restarting applications and reinstalling the font, it became clear that the problem originated from the font generation process itself.&lt;br&gt;
Reviewing the Font Generation Script&lt;br&gt;
The script was designed to read handwritten character images, create glyphs for each character, and generate a TrueType font.&lt;br&gt;
For each character, the script:&lt;br&gt;
Created a glyph using its Unicode value.&lt;br&gt;
Loaded the corresponding handwritten image.&lt;br&gt;
Converted the image into vector outlines.&lt;br&gt;
Applied glyph cleanup operations.&lt;br&gt;
Assigned character spacing.&lt;br&gt;
Added the glyph to the font.&lt;br&gt;
Once all characters were processed, the font was validated and exported as a .ttf file.&lt;br&gt;
While the workflow seemed straightforward, I learned that generating a font file is only one part of the process. The generated font must also contain valid glyph data and proper metadata to be recognized consistently by different applications.&lt;br&gt;
Improvements Made&lt;br&gt;
To improve the quality and compatibility of the generated font, I made several changes to the script.&lt;br&gt;
Converting Images to Vector Outlines&lt;br&gt;
Handwritten characters were stored as PNG images. Before they could be used in a font, they needed to be converted into vector outlines.&lt;br&gt;
Using FontForge's tracing functionality ensured that each character contained scalable vector data rather than simple bitmap information.&lt;br&gt;
Cleaning Glyph Data&lt;br&gt;
After tracing, I applied additional cleanup operations to simplify paths and correct outline directions.&lt;br&gt;
These steps helped create cleaner glyphs and reduced the likelihood of rendering issues.&lt;br&gt;
Defining Character Width&lt;br&gt;
Character spacing plays an important role in readability.&lt;br&gt;
By assigning a width value to each glyph, I ensured that letters would display with consistent spacing when typed.&lt;br&gt;
Adding Font Metadata&lt;br&gt;
I also defined the font's family name, font name, and full name.&lt;br&gt;
Proper metadata helps operating systems and applications identify and register fonts correctly.&lt;br&gt;
Validating Before Export&lt;br&gt;
Before generating the final font file, I included a validation step.&lt;br&gt;
This allowed FontForge to check for common issues and helped ensure the generated font met basic requirements.&lt;br&gt;
Next Blog:&lt;br&gt;
This project started with English characters, but my next goal is much bigger—creating a Tamil handwriting font. I'm looking forward to exploring the unique challenges that come with Tamil script development and sharing what I learn along the way.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
      <category>beginners</category>
      <category>python</category>
    </item>
    <item>
      <title>7 Best SaaS Courses for Developers Who Want to Launch a Product in 2026</title>
      <dc:creator>Esimit Karlgusta</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:19:02 +0000</pubDate>
      <link>https://dev.to/thekarlesi/7-best-saas-courses-for-developers-who-want-to-launch-a-product-in-2026-4829</link>
      <guid>https://dev.to/thekarlesi/7-best-saas-courses-for-developers-who-want-to-launch-a-product-in-2026-4829</guid>
      <description>&lt;p&gt;Building a SaaS has never been easier.&lt;/p&gt;

&lt;p&gt;Launching one has never been harder.&lt;/p&gt;

&lt;p&gt;Most developers don't struggle with coding. They struggle with turning knowledge into a finished product.&lt;/p&gt;

&lt;p&gt;That's why SaaS-focused courses have exploded in popularity. The right course can save weeks of confusion around architecture, authentication, payments, deployment, and launch strategy.&lt;/p&gt;

&lt;p&gt;After reviewing popular options, here are some of the best SaaS courses and learning resources available in 2026.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Zero to SaaS
&lt;/h2&gt;

&lt;p&gt;Best for: Developers who want to build and launch a SaaS quickly&lt;/p&gt;

&lt;p&gt;Many courses spend dozens of hours teaching concepts without helping students ship.&lt;/p&gt;

&lt;p&gt;Zero to SaaS takes a different approach.&lt;/p&gt;

&lt;p&gt;The blueprint is designed around a simple outcome:&lt;/p&gt;

&lt;p&gt;Build and launch a SaaS in 14 focused days.&lt;/p&gt;

&lt;p&gt;Instead of overwhelming students with endless theory, it walks through the critical decisions involved in creating a launch-ready product, including:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Next.js architecture&lt;/li&gt;
&lt;li&gt;Database design&lt;/li&gt;
&lt;li&gt;Authentication&lt;/li&gt;
&lt;li&gt;Stripe payments&lt;/li&gt;
&lt;li&gt;Deployment&lt;/li&gt;
&lt;li&gt;Launch preparation&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The focus is execution rather than information consumption.&lt;/p&gt;

&lt;p&gt;If you've spent months learning but haven't shipped anything, this practical approach is refreshing.&lt;/p&gt;

&lt;p&gt;Website:&lt;br&gt;
&lt;a href="https://zero-to-saas.collabtower.com/" rel="noopener noreferrer"&gt;https://zero-to-saas.collabtower.com/&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Full Stack Open
&lt;/h2&gt;

&lt;p&gt;Best for: Deep technical learning&lt;/p&gt;

&lt;p&gt;Full Stack Open is one of the most respected free developer programs available.&lt;/p&gt;

&lt;p&gt;It covers modern web development, React, APIs, testing, and backend systems in significant depth.&lt;/p&gt;

&lt;p&gt;The downside is that it's designed more for learning engineering concepts than launching products.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. The Odin Project
&lt;/h2&gt;

&lt;p&gt;Best for: Beginners starting from scratch&lt;/p&gt;

&lt;p&gt;The Odin Project remains one of the best free resources for aspiring developers.&lt;/p&gt;

&lt;p&gt;It provides a structured curriculum covering frontend and backend development.&lt;/p&gt;

&lt;p&gt;For SaaS founders, it's an excellent foundation before moving into product building.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Buildspace
&lt;/h2&gt;

&lt;p&gt;Best for: Community-driven builders&lt;/p&gt;

&lt;p&gt;Buildspace became popular for helping creators build projects alongside other developers.&lt;/p&gt;

&lt;p&gt;The community aspect is one of its biggest strengths.&lt;/p&gt;

&lt;p&gt;Students gain accountability and feedback while working on real products.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Indie Hackers
&lt;/h2&gt;

&lt;p&gt;Best for: Learning from founders&lt;/p&gt;

&lt;p&gt;While not technically a course, Indie Hackers offers an enormous amount of practical startup knowledge.&lt;/p&gt;

&lt;p&gt;You'll find interviews, case studies, launch stories, and lessons from bootstrapped founders.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. Y Combinator Startup School
&lt;/h2&gt;

&lt;p&gt;Best for: Startup fundamentals&lt;/p&gt;

&lt;p&gt;Startup School focuses less on coding and more on company building.&lt;/p&gt;

&lt;p&gt;Topics include:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Validation&lt;/li&gt;
&lt;li&gt;Distribution&lt;/li&gt;
&lt;li&gt;Customer discovery&lt;/li&gt;
&lt;li&gt;Growth&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It's valuable once you've moved beyond the technical stage.&lt;/p&gt;

&lt;h2&gt;
  
  
  7. freeCodeCamp
&lt;/h2&gt;

&lt;p&gt;Best for: Expanding technical skills&lt;/p&gt;

&lt;p&gt;freeCodeCamp provides thousands of hours of free programming content.&lt;/p&gt;

&lt;p&gt;It's an excellent supplement for developers looking to strengthen specific skills while building products.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Makes a Great SaaS Course?
&lt;/h2&gt;

&lt;p&gt;The best SaaS courses do more than teach code.&lt;/p&gt;

&lt;p&gt;They help students answer questions like:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;What should I build?&lt;/li&gt;
&lt;li&gt;How much should I build?&lt;/li&gt;
&lt;li&gt;When should I launch?&lt;/li&gt;
&lt;li&gt;How do I charge users?&lt;/li&gt;
&lt;li&gt;How do I avoid overengineering?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The goal isn't knowledge accumulation.&lt;/p&gt;

&lt;p&gt;The goal is product creation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Which Course Should You Choose?
&lt;/h2&gt;

&lt;p&gt;If you're completely new to development, start with foundational resources like The Odin Project or freeCodeCamp.&lt;/p&gt;

&lt;p&gt;If you're already comfortable with React and modern web development, choose a resource focused on shipping products.&lt;/p&gt;

&lt;p&gt;That's where execution-focused programs such as Zero to SaaS provide the most value.&lt;/p&gt;

&lt;p&gt;The difference between learning and launching often comes down to having a clear roadmap.&lt;/p&gt;

&lt;p&gt;And for many developers, that's exactly what's missing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Final Thoughts
&lt;/h2&gt;

&lt;p&gt;Most aspiring founders don't need more information.&lt;/p&gt;

&lt;p&gt;They need a system that helps them finish.&lt;/p&gt;

&lt;p&gt;The internet is full of tutorials.&lt;/p&gt;

&lt;p&gt;Launched products are much harder to find.&lt;/p&gt;

&lt;p&gt;Choose resources that move you closer to shipping, not just studying.&lt;/p&gt;

&lt;p&gt;Because the fastest way to learn SaaS is still to build one.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
      <category>productivity</category>
      <category>beginners</category>
    </item>
    <item>
      <title>After 4 months solo: shipping a Windows tray AI hotkey on .NET 8 + WPF (and the Win32 paste-back rabbit hole)</title>
      <dc:creator>Роман Тихоненко</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:18:52 +0000</pubDate>
      <link>https://dev.to/phantasmat2018/after-4-months-solo-shipping-a-windows-tray-ai-hotkey-on-net-8-wpf-and-the-win32-paste-back-50hm</link>
      <guid>https://dev.to/phantasmat2018/after-4-months-solo-shipping-a-windows-tray-ai-hotkey-on-net-8-wpf-and-the-win32-paste-back-50hm</guid>
      <description>&lt;p&gt;I spent 4 months of nights and weekends building &lt;a href="https://capybro.app" rel="noopener noreferrer"&gt;CapyBro&lt;/a&gt; — a Windows tray app that runs AI on any selected text via a global hotkey. Native .NET 8 + WPF (not Electron), MIT-licensed, ~49 MB installer. Two backends: cloud (OpenRouter) or fully local (Ollama). The hardest technical problem turned out to be Win32 paste-back into child controls. This post walks through that rabbit hole + the architecture decisions that paid off.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why I built this
&lt;/h2&gt;

&lt;p&gt;For most of 2025, my AI workflow was this loop:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;1. Read something in Slack / a doc / a Telegram message
2. Alt+Tab → ChatGPT tab → paste
3. Type my prompt
4. Wait
5. Copy result
6. Alt+Tab back → paste over the original
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I caught myself doing this &lt;strong&gt;30+ times a day&lt;/strong&gt; for trivial things — fixing one comma, translating a paragraph for a client email, rewording a DM so it doesn't sound passive-aggressive.&lt;/p&gt;

&lt;p&gt;Then it hit me: AI is currently trapped in a browser tab. But every other utility on my PC — clipboard manager, screenshot tool, voice typer, password manager, snippet expander — is &lt;strong&gt;one hotkey away&lt;/strong&gt;. Why isn't AI?&lt;/p&gt;

&lt;p&gt;So I built it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What CapyBro does
&lt;/h2&gt;

&lt;p&gt;You select text &lt;strong&gt;anywhere&lt;/strong&gt; on Windows. Word, Telegram, VS Code, your browser, Notepad, Discord, an email draft, a YAML file in your terminal — doesn't matter. The OS-level selection is the input.&lt;/p&gt;

&lt;p&gt;You press &lt;strong&gt;Ctrl+Shift+E&lt;/strong&gt;. A small popup appears. You pick a prompt (or type a custom one). AI runs. The result &lt;strong&gt;replaces the original text in the same app&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;That's the entire product. The whole magic is in step 3 — &lt;em&gt;replacing text in the source app&lt;/em&gt;. Sounds trivial. Took me three iterations of Win32 plumbing to get right.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Win32 paste-back rabbit hole
&lt;/h2&gt;

&lt;p&gt;This is the most undervalued part of the project. I expected ~100 lines of code. Got 130 lines of comments around a 50-line method that does two things: &lt;strong&gt;capture focused-child HWND before showing UI&lt;/strong&gt;, then &lt;strong&gt;restore foreground + focus&lt;/strong&gt; when user clicks Accept.&lt;/p&gt;

&lt;h3&gt;
  
  
  Naive solution (doesn't work)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="n"&gt;SendKeys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;SendWait&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"^v"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Why it fails:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;No control over which window receives the Ctrl+V&lt;/li&gt;
&lt;li&gt;If your modal dialog is still open, Ctrl+V goes THERE&lt;/li&gt;
&lt;li&gt;If foreground changed between Accept and send → paste lands somewhere random&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  Better but still broken
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="n"&gt;IntPtr&lt;/span&gt; &lt;span class="n"&gt;originalForeground&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;GetForegroundWindow&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="nf"&gt;ShowDialog&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="c1"&gt;// User clicks Accept&lt;/span&gt;
&lt;span class="nf"&gt;SetForegroundWindow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;originalForeground&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="n"&gt;Thread&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;50&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="n"&gt;SendKeys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;SendWait&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"^v"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Works for Notepad. Fails for Notepad++, VS Code, Office. Why?&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;code&gt;SetForegroundWindow&lt;/code&gt; is gated by &lt;a href="https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setforegroundwindow" rel="noopener noreferrer"&gt;OS focus rules&lt;/a&gt; — caller must have received last input event, no active foreground lock, target not minimized. Any check fails → it silently returns false.&lt;/li&gt;
&lt;li&gt;The actual editor lives in a child control (e.g. Scintilla inside Notepad++). &lt;code&gt;SetForegroundWindow&lt;/code&gt; activates the top-level frame, but keyboard focus stays elsewhere. &lt;code&gt;SendInput Ctrl+V&lt;/code&gt; then lands on the WindowProc of a non-input frame.&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  Working solution: AttachThreadInput sandwich
&lt;/h3&gt;

&lt;p&gt;The trick is &lt;strong&gt;AttachThreadInput&lt;/strong&gt; — an API that lets your thread temporarily share input state with another thread. While attached, the OS treats both threads as one for focus/foreground purposes, bypassing the "didn't receive last input event" check.&lt;/p&gt;

&lt;p&gt;Plus: I need to know &lt;strong&gt;which child HWND had keyboard focus&lt;/strong&gt; before my modal stole it. &lt;code&gt;GetGUIThreadInfo.hwndFocus&lt;/code&gt; returns exactly that.&lt;/p&gt;

&lt;p&gt;Here's the production code (extracted from &lt;a href="https://github.com/phantasmat2018/capy-bro/blob/main/src/CapyBro/Platform/ForegroundRestorer.cs" rel="noopener noreferrer"&gt;&lt;code&gt;Platform/ForegroundRestorer.cs&lt;/code&gt;&lt;/a&gt;):&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Phase 1: capture, BEFORE showing UI&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;static&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;IntPtr&lt;/span&gt; &lt;span class="n"&gt;TopLevel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;IntPtr&lt;/span&gt; &lt;span class="n"&gt;FocusedChild&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nf"&gt;CaptureForegroundFocus&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;topLevel&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;GetForegroundWindow&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="n"&gt;IntPtr&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Zero&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;IntPtr&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Zero&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;IntPtr&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Zero&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;targetThreadId&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;GetWindowThreadProcessId&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;out&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;targetThreadId&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;IntPtr&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Zero&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;info&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;GUITHREADINFO&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;CbSize&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;uint&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;Marshal&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SizeOf&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;GUITHREADINFO&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;(),&lt;/span&gt;
    &lt;span class="p"&gt;};&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;GetGUIThreadInfo&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;targetThreadId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;ref&lt;/span&gt; &lt;span class="n"&gt;info&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;info&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;HwndFocus&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;IntPtr&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Zero&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Phase 2: restore, AFTER user clicks Accept&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;static&lt;/span&gt; &lt;span class="kt"&gt;bool&lt;/span&gt; &lt;span class="nf"&gt;RestoreToForeground&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;IntPtr&lt;/span&gt; &lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;IntPtr&lt;/span&gt; &lt;span class="n"&gt;focusedChild&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="n"&gt;IntPtr&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Zero&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;targetThreadId&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;GetWindowThreadProcessId&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;out&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;targetThreadId&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;SetForegroundWindow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="c1"&gt;// window died, best-effort&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;IsIconic&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ShowWindowAsync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SwRestore&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;ourThreadId&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;GetCurrentThreadId&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ourThreadId&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="n"&gt;targetThreadId&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;SetForegroundWindow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="c1"&gt;// AttachThreadInput on same thread is undefined&lt;/span&gt;

    &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;attached&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;AttachThreadInput&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ourThreadId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;targetThreadId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;BringWindowToTop&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;fgOk&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;SetForegroundWindow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

        &lt;span class="c1"&gt;// CRITICAL: SetFocus on the FOCUSED CHILD, not the top-level frame.&lt;/span&gt;
        &lt;span class="c1"&gt;// Without this, SendInput Ctrl+V echoes into the non-input frame.&lt;/span&gt;
        &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;focusTarget&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;focusedChild&lt;/span&gt; &lt;span class="p"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;IntPtr&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Zero&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="n"&gt;focusedChild&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;topLevel&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;SetFocus&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;focusTarget&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;fgOk&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;finally&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c1"&gt;// ALWAYS detach. Forget this, and the user loses keyboard control&lt;/span&gt;
        &lt;span class="c1"&gt;// system-wide for the lifetime of your process.&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;attached&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;NativeMethods&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;AttachThreadInput&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ourThreadId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;targetThreadId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;false&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Pieces that earned their place
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;AttachThreadInput&lt;/code&gt;&lt;/strong&gt; — bypasses focus-stealing protection. Without it, &lt;code&gt;SetForegroundWindow&lt;/code&gt; silently no-ops.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;SetFocus&lt;/code&gt; on child HWND&lt;/strong&gt; — Notepad++'s actual edit is a Scintilla control nested inside its frame. &lt;code&gt;SetFocus&lt;/code&gt; on the frame leaves keyboard focus on the wrong WindowProc.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;BringWindowToTop&lt;/code&gt; before &lt;code&gt;SetForegroundWindow&lt;/code&gt;&lt;/strong&gt; — raises z-order even when the foreground call is rejected. Belt-and-braces.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;IsIconic&lt;/code&gt; + &lt;code&gt;ShowWindowAsync(SW_RESTORE)&lt;/code&gt;&lt;/strong&gt; — &lt;code&gt;SetForegroundWindow&lt;/code&gt; no-ops on minimized targets. Restore first.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;finally&lt;/code&gt; + &lt;code&gt;AttachThreadInput(false)&lt;/code&gt;&lt;/strong&gt; — I forgot this once during development. Lost system-wide keyboard input until I rebooted. Don't be me.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;SendInput&lt;/code&gt; not &lt;code&gt;SendKeys&lt;/code&gt;&lt;/strong&gt; — &lt;code&gt;SendKeys&lt;/code&gt; uses scan codes that break on non-Latin layouts. &lt;code&gt;SendInput&lt;/code&gt; works with virtual-key codes.&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  Bonus rabbit hole: the clipboard is single-owner
&lt;/h3&gt;

&lt;p&gt;Win32 clipboard is a single-owner resource. Clipboard managers, RDP virtual channels, antivirus, even the OS shell briefly hold it. Without retry, any concurrent open throws &lt;code&gt;CLIPBRD_E_CANT_OPEN&lt;/code&gt; (HRESULT &lt;code&gt;0x800401D0&lt;/code&gt;) and you lose either the AI result or the user's original selection.&lt;/p&gt;

&lt;p&gt;I wrap every clipboard call in an &lt;strong&gt;async&lt;/strong&gt; retry loop (not sync — sync &lt;code&gt;Thread.Sleep&lt;/code&gt; between attempts freezes the WPF UI for up to 500ms):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;private&lt;/span&gt; &lt;span class="k"&gt;static&lt;/span&gt; &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="n"&gt;Task&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;T&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;RetryAsync&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;T&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;(&lt;/span&gt;&lt;span class="n"&gt;Func&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;T&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;action&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;CancellationToken&lt;/span&gt; &lt;span class="n"&gt;ct&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;RetryAttempts&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="m"&gt;10&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;retryDelay&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;TimeSpan&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;FromMilliseconds&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;50&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="p"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="n"&gt;RetryAttempts&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt;&lt;span class="p"&gt;++)&lt;/span&gt;
    &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;ct&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ThrowIfCancellationRequested&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;action&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;COMException&lt;/span&gt; &lt;span class="n"&gt;ex&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;when&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ex&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;HResult&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="k"&gt;unchecked&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="m"&gt;0x800401D0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
                &lt;span class="p"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="p"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;RetryAttempts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;Task&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Delay&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;retryDelay&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ct&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;ConfigureAwait&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nf"&gt;InvalidOperationException&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The Win32 calls themselves are synchronous (the API has no cancellation hook), but the gaps between retries release the dispatcher so WPF can pump messages, repaint, and respond to input.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two AI backends, one interface
&lt;/h2&gt;

&lt;p&gt;CapyBro supports OpenRouter (cloud — one API key, ~300 models) and Ollama (local — text never leaves the machine). Both stream responses.&lt;/p&gt;

&lt;p&gt;The pain: OpenRouter speaks &lt;strong&gt;SSE&lt;/strong&gt; (&lt;code&gt;data: {...}\n\n&lt;/code&gt;, terminated by &lt;code&gt;data: [DONE]&lt;/code&gt;), Ollama speaks &lt;strong&gt;NDJSON&lt;/strong&gt; (one JSON object per line, terminated by &lt;code&gt;{"done": true}&lt;/code&gt;). Different error shapes, different rate-limit signaling.&lt;/p&gt;

&lt;p&gt;The abstraction:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;interface&lt;/span&gt; &lt;span class="nc"&gt;ILlmProvider&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;IAsyncEnumerable&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;ImproveStreamAsync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="n"&gt;apiKey&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;         &lt;span class="c1"&gt;// OpenRouter uses; Ollama ignores&lt;/span&gt;
        &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="n"&gt;promptText&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="n"&gt;userText&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;TimeSpan&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="kt"&gt;bool&lt;/span&gt; &lt;span class="n"&gt;preserveLanguage&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;CancellationToken&lt;/span&gt; &lt;span class="n"&gt;ct&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;default&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="n"&gt;Task&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;IReadOnlyList&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;GetModelsAsync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="n"&gt;apiKey&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;CancellationToken&lt;/span&gt; &lt;span class="n"&gt;ct&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;default&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="kt"&gt;bool&lt;/span&gt; &lt;span class="n"&gt;RequiresApiKey&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="k"&gt;get&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;RequiresApiKey&lt;/code&gt; is the cute bit — it lets TextProcessor pre-flight the request. If &lt;code&gt;Provider=OpenRouter&lt;/code&gt; and key is empty, show an actionable toast ("set your key in Settings") instead of round-tripping to a 401.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;ILlmProviderFactory.Resolve&lt;/code&gt; is a switch that &lt;strong&gt;throws on unknown enum values&lt;/strong&gt;, not a fall-back to OpenRouter. A future 3rd provider added without matching switch arm + DI registration will crash on first user interaction instead of silently routing to the wrong backend. That's intentional — silent fallbacks are how you ship "why does my Anthropic key work everywhere except CapyBro?" bug reports.&lt;/p&gt;

&lt;h3&gt;
  
  
  Ollama edge case: stream truncated vs empty result
&lt;/h3&gt;

&lt;p&gt;A subtle bug I found while testing: Ollama can complete a request &lt;strong&gt;without&lt;/strong&gt; a &lt;code&gt;done:true&lt;/code&gt; frame — connection drop, proxy timeout, antivirus interception. The total content length is 0, but it's not "the model returned nothing" — it's "the network died."&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;sawDoneFrame&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;totalContentLength&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;foreach&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;frame&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;ReadNdjsonFramesAsync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Token&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frame&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Done&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;sawDoneFrame&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frame&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Delta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Length&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;continue&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;totalContentLength&lt;/span&gt; &lt;span class="p"&gt;+=&lt;/span&gt; &lt;span class="n"&gt;frame&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Delta&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Length&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;frame&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Delta&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;totalContentLength&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nf"&gt;OpenRouterException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;sawDoneFrame&lt;/span&gt;
            &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="n"&gt;_translator&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"api_empty_result"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;   &lt;span class="c1"&gt;// model returned ""&lt;/span&gt;
            &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;_translator&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"api_server_error"&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt; &lt;span class="c1"&gt;// stream interrupted&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two different toasts: "the model didn't produce output for your prompt" vs "check whether &lt;code&gt;ollama serve&lt;/code&gt; is running." Different remediation paths for the user.&lt;/p&gt;

&lt;h2&gt;
  
  
  Installer size: 150 MB → 49 MB
&lt;/h2&gt;

&lt;p&gt;Self-contained .NET 8 + WPF publish folder is ~150 MB. Single-file &lt;code&gt;.exe&lt;/code&gt; is tempting but:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Single-file decompresses into memory on every launch → visible cold-start latency&lt;/li&gt;
&lt;li&gt;Self-extracting native libraries unpack to &lt;code&gt;%TEMP%&lt;/code&gt; on first run → first-launch hit&lt;/li&gt;
&lt;li&gt;For a tray app the user opens dozens of times a day, that's noticeable&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So: &lt;strong&gt;folder build&lt;/strong&gt; + NSIS LZMA SOLID compression. The csproj:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight xml"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;PropertyGroup&lt;/span&gt; &lt;span class="na"&gt;Condition=&lt;/span&gt;&lt;span class="s"&gt;"'$(_IsPublishing)' == 'true'"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;SelfContained&amp;gt;&lt;/span&gt;true&lt;span class="nt"&gt;&amp;lt;/SelfContained&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;RuntimeIdentifier&amp;gt;&lt;/span&gt;win-x64&lt;span class="nt"&gt;&amp;lt;/RuntimeIdentifier&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;PublishReadyToRun&amp;gt;&lt;/span&gt;true&lt;span class="nt"&gt;&amp;lt;/PublishReadyToRun&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;DebugType&amp;gt;&lt;/span&gt;none&lt;span class="nt"&gt;&amp;lt;/DebugType&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;/PropertyGroup&amp;gt;&lt;/span&gt;

&lt;span class="c"&gt;&amp;lt;!-- Strip 13 culture-specific satellite assembly folders. --&amp;gt;&lt;/span&gt;
&lt;span class="c"&gt;&amp;lt;!-- Our UI translations live in Translator.cs, not satellite assemblies. --&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;SatelliteResourceLanguages&amp;gt;&lt;/span&gt;en&lt;span class="nt"&gt;&amp;lt;/SatelliteResourceLanguages&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;NSIS:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;SetCompressor /SOLID lzma
SetCompressorDictSize 64
File /r "..\publish\win-x64\*.*"
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;LZMA SOLID archives everything as one stream rather than per-file. Repeated bytes across files compress much better. ~49 MB installer, ~150 MB unpacked.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I didn't do: &lt;code&gt;PublishTrimmed&lt;/code&gt;.&lt;/strong&gt; WPF heavily uses reflection for XAML binding + resource lookup. Trimmer eagerly removes "unused" types, then runtime XAML lookup explodes with &lt;code&gt;Type not found&lt;/code&gt;. I tried &lt;code&gt;TrimMode=partial&lt;/code&gt; and got 25 MB savings + 12 runtime regressions. Reverted.&lt;/p&gt;

&lt;h2&gt;
  
  
  Open source as a trust mechanism, not ideology
&lt;/h2&gt;

&lt;p&gt;This is a utility that reads my text — sometimes confidential (client emails, draft docs). Would I trust it if it were closed-source from an unknown indie dev?&lt;/p&gt;

&lt;p&gt;No.&lt;/p&gt;

&lt;p&gt;So why should other people trust me?&lt;/p&gt;

&lt;p&gt;Answer: &lt;strong&gt;open the source&lt;/strong&gt;. Remove "trust me bro" and show what happens.&lt;/p&gt;

&lt;p&gt;I picked MIT. Almost went "source-available" (popular among indie SaaS right now) but decided:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;If someone forks and fixes a bug I missed → that saves me work, doesn't "compete" with my product&lt;/li&gt;
&lt;li&gt;If someone forks and sells their own version → they still don't have my community, my support, my updates. The product isn't the code.&lt;/li&gt;
&lt;li&gt;"Source-available" gets a negative reaction in the dev community. MIT gets a positive one.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;API keys live in &lt;strong&gt;Windows Credential Manager&lt;/strong&gt; via &lt;a href="https://www.nuget.org/packages/Meziantou.Framework.Win32.CredentialManager" rel="noopener noreferrer"&gt;&lt;code&gt;Meziantou.Framework.Win32.CredentialManager&lt;/code&gt;&lt;/a&gt;, not &lt;code&gt;config.json&lt;/code&gt;. DPAPI encryption under the hood, bound to the user account, non-portable across machines by design.&lt;/p&gt;

&lt;h2&gt;
  
  
  Lessons after 4 months
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Win32 is alive.&lt;/strong&gt; Microsoft didn't replace it — they hid it behind WPF/WinUI. Build anything non-trivial system-side, and you're back to user32.dll. My P/Invoke list for the core workflow: &lt;code&gt;RegisterHotKey&lt;/code&gt;, &lt;code&gt;SendInput&lt;/code&gt;, &lt;code&gt;GetForegroundWindow&lt;/code&gt;, &lt;code&gt;GetGUIThreadInfo&lt;/code&gt;, &lt;code&gt;AttachThreadInput&lt;/code&gt;, &lt;code&gt;SetFocus&lt;/code&gt;, &lt;code&gt;BringWindowToTop&lt;/code&gt;, &lt;code&gt;IsIconic&lt;/code&gt;, &lt;code&gt;ShowWindowAsync&lt;/code&gt;. That's just the baseline.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Native beats Electron for tray utilities.&lt;/strong&gt; Not ideology — pragmatism. 49 MB vs 250 MB installer, &amp;lt;1s cold start vs 3-5s, ~80 MB RAM idle vs ~400 MB. For something that lives in the background, that's the difference between "I don't notice it" and "oh there you are."&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;code&gt;Local\&lt;/code&gt; Mutex, not &lt;code&gt;Global\&lt;/code&gt;.&lt;/strong&gt; Singletons usually use &lt;code&gt;Global\&lt;/code&gt; namespace, which requires &lt;code&gt;SeCreateGlobalPrivilege&lt;/code&gt;. That right is granted to interactive users by default but &lt;strong&gt;stripped on locked-down domain machines&lt;/strong&gt; (kiosks, AppLocker configs). On those systems, my app crashed at startup with &lt;code&gt;UnauthorizedAccessException&lt;/code&gt;. &lt;code&gt;Local\&lt;/code&gt; (per-session) has no such restriction and matches the semantics I actually want (one instance per user session, not per machine):&lt;br&gt;
&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;   &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="n"&gt;DefaultMutexName&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;@"Local\CapyBroV2"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
   &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;mutex&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nf"&gt;Mutex&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;initiallyOwned&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;mutexName&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;createdNew&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;out&lt;/span&gt; &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;createdNew&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Also: &lt;code&gt;initiallyOwned: false&lt;/code&gt;. With &lt;code&gt;true&lt;/code&gt;, I'd get &lt;code&gt;AbandonedMutexException&lt;/code&gt; after every crash. With &lt;code&gt;false&lt;/code&gt;, process death cleans up silently.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Foreground-poller for popup dismiss, not Mouse.Capture.&lt;/strong&gt; My first prompt-picker used &lt;code&gt;Mouse.Capture(this, SubTree)&lt;/code&gt; to detect clicks outside. WPF ListBox grabs &lt;code&gt;Mouse.Capture&lt;/code&gt; internally for click-drag selection — my &lt;code&gt;LostMouseCapture&lt;/code&gt; handler closed the popup BEFORE the user's MouseLeftButtonUp reached the ListBox. Final version uses a 100ms &lt;code&gt;DispatcherTimer&lt;/code&gt; + &lt;code&gt;GetForegroundWindow()&lt;/code&gt; poll. If foreground isn't my popup → close. Cross-process clicks (browser tabs, Notepad, Telegram) are invisible to WPF's input system — polling Win32 is the only reliable catch-all.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;STJ with source generation.&lt;/strong&gt; Not &lt;code&gt;JsonSerializer.Deserialize&amp;lt;T&amp;gt;(json)&lt;/code&gt; (reflection-based). Instead:&lt;br&gt;
&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csharp"&gt;&lt;code&gt;   &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nf"&gt;JsonSerializable&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;typeof&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;AppConfig&lt;/span&gt;&lt;span class="p"&gt;))]&lt;/span&gt;
   &lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;partial&lt;/span&gt; &lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;AppConfigJsonContext&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;JsonSerializerContext&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;

   &lt;span class="kt"&gt;var&lt;/span&gt; &lt;span class="n"&gt;config&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;JsonSerializer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Deserialize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;AppConfigJsonContext&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Default&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;AppConfig&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One day to set up &lt;code&gt;[JsonSerializable]&lt;/code&gt; attrs for each DTO. Result: AOT-friendly, no runtime reflection, faster parsing, cleaner stack traces on &lt;code&gt;JsonException&lt;/code&gt;.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Tech is ~30% of the work.&lt;/strong&gt; The other 70% is marketing, docs, screenshots, localizations, SEO, GitHub issue triage, replying on Reddit. As a solo dev, that's not "side activity" — it's &lt;strong&gt;the&lt;/strong&gt; activity after MVP.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Stack receipts
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;~12,000 lines of C# (the WPF app)&lt;/li&gt;
&lt;li&gt;~3,000 lines of Next.js (marketing site)&lt;/li&gt;
&lt;li&gt;~4 months, nights + weekends&lt;/li&gt;
&lt;li&gt;~$130 spent (domain, OpenRouter test credits, stock icons I didn't end up using)&lt;/li&gt;
&lt;li&gt;Coffee: uncountable&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What's next
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;macOS port via Avalonia (~2 months)&lt;/li&gt;
&lt;li&gt;Browser extension companion for web apps with shadow DOM&lt;/li&gt;
&lt;li&gt;Native AOT once WPF + AOT become compatible (would shave 49 MB → ~25 MB)&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Code + links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Source:&lt;/strong&gt; &lt;a href="https://github.com/phantasmat2018/capy-bro" rel="noopener noreferrer"&gt;https://github.com/phantasmat2018/capy-bro&lt;/a&gt; (MIT)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Site:&lt;/strong&gt; &lt;a href="https://capybro.app" rel="noopener noreferrer"&gt;https://capybro.app&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Installer:&lt;/strong&gt; &lt;a href="https://github.com/phantasmat2018/capy-bro/releases/tag/v2.0.0" rel="noopener noreferrer"&gt;https://github.com/phantasmat2018/capy-bro/releases/tag/v2.0.0&lt;/a&gt; (Win 10/11 x64, 49 MB)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you've built a similar Windows-side AI tool, I'd love to hear what Win32 weirdness you ran into. The Office (Word, Excel) paste-back behavior is something I still haven't 100% nailed — Word works, Excel works only via the F2/Esc edit-mode dance. If anyone has a clean solution, drop it in the comments 🙏&lt;/p&gt;

&lt;p&gt;Thanks for reading.&lt;/p&gt;

</description>
      <category>dotnet</category>
      <category>csharp</category>
      <category>opensource</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Seu modelo de regressão mente quando X não varia — e você provavelmente não percebe</title>
      <dc:creator>Ana Carolina Neumann Rodrigues</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:16:45 +0000</pubDate>
      <link>https://dev.to/anacneumann/seu-modelo-de-regressao-mente-quando-x-nao-varia-e-voce-provavelmente-nao-percebe-1id</link>
      <guid>https://dev.to/anacneumann/seu-modelo-de-regressao-mente-quando-x-nao-varia-e-voce-provavelmente-nao-percebe-1id</guid>
      <description>&lt;p&gt;Tem uma armadilha clássica em projetos de ciência de dados com regressão linear que pega muita gente: o modelo treina, a loss parece ok, o R² até aparece razoável — mas as estimativas de coeficiente são uma bagunça.&lt;/p&gt;

&lt;p&gt;O motivo, quase sempre, é simples: &lt;strong&gt;X não varia o suficiente&lt;/strong&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  O problema em 30 segundos
&lt;/h2&gt;

&lt;p&gt;Na regressão linear simples:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Y = β₀ + β₁X + ε
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A variância do coeficiente estimado é:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Var(β̂₁) = σ² / Σ(xᵢ - x̄)²
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Lê assim:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Var(β̂₁) = ruído do modelo / variação de X
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Duas conclusões diretas:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Muito ruído em Y&lt;/strong&gt; → estimativa instável&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pouca variação em X&lt;/strong&gt; → estimativa instável&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;O denominador é o ponto que costuma ser ignorado.&lt;/p&gt;




&lt;h2&gt;
  
  
  Exemplo concreto: previsão de lead time
&lt;/h2&gt;

&lt;p&gt;Você trabalha com supply chain e quer prever o &lt;strong&gt;lead time de entrega&lt;/strong&gt; (em dias) com base na &lt;strong&gt;distância percorrida&lt;/strong&gt; (em km).&lt;/p&gt;

&lt;h3&gt;
  
  
  Cenário A: dados de uma só rota regional
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Distância (km)&lt;/th&gt;
&lt;th&gt;Lead Time (dias)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;480&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;490&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;500&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;510&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;505&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Todo mundo está na mesma rota, percorrendo praticamente a mesma distância.&lt;/p&gt;

&lt;p&gt;O modelo olha pra isso e pensa:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;"X quase não mudou. Como vou saber o efeito de X em Y?"&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Qualquer variação no lead time pode ser atraso no porto, problema do fornecedor, feriado — não necessariamente distância. A inclinação estimada vai ser instável e pouco confiável.&lt;/p&gt;




&lt;h3&gt;
  
  
  Cenário B: dados de múltiplas rotas
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Distância (km)&lt;/th&gt;
&lt;th&gt;Lead Time (dias)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;80&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;250&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;600&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1.200&lt;/td&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2.800&lt;/td&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4.500&lt;/td&gt;
&lt;td&gt;18&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Agora o modelo tem "evidência horizontal" de verdade. Ele vê embarques curtos, médios e longos — e consegue separar o efeito da distância do ruído aleatório.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhhmung7uv3qv0a8ld2tl.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhhmung7uv3qv0a8ld2tl.gif" alt="GIF mostrando uma reta ajustando bem em dados espalhados" width="760" height="570"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  Por que variação em X importa tanto?
&lt;/h2&gt;

&lt;p&gt;A fórmula do coeficiente estimado é:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;β̂₁ = Σ(xᵢ - x̄)(yᵢ - ȳ) / Σ(xᵢ - x̄)²
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;O denominador é o mesmo que aparece na variância: &lt;strong&gt;quanto X varia&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Quando X mal se move, o denominador fica pequeno. Qualquer ruído em Y distorce muito a razão. O resultado é um coeficiente que parece razoável num treino mas oscila absurdamente entre diferentes amostras.&lt;/p&gt;




&lt;h2&gt;
  
  
  Três cuidados que ninguém te conta
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Variação causada por outlier não conta como variação boa
&lt;/h3&gt;

&lt;p&gt;Imagina que seus dados de distância são assim:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight csvs"&gt;&lt;code&gt;&lt;span class="mf"&gt;480&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;490&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;510&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;4800&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Matematicamente, X tem muita variação. Na prática, ela vem de &lt;strong&gt;um único ponto extremo&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Esse ponto tem alta alavancagem — ele puxa a reta inteira. O modelo fica "confiante" nos cálculos, mas essa confiança é falsa.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F72s8rhskk4d70ou8al0q.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F72s8rhskk4d70ou8al0q.png" alt="Variações de X" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h3&gt;
  
  
  2. Variação em X não resolve relação não-linear
&lt;/h3&gt;

&lt;p&gt;Se o lead time cresce exponencialmente com a distância (armazém regional → cross-border), uma reta pode não capturar o padrão.&lt;/p&gt;

&lt;p&gt;Ter bastante variação em X ajuda, mas não substitui escolher o modelo certo.&lt;/p&gt;




&lt;h3&gt;
  
  
  3. Em regressão múltipla, X precisa variar &lt;em&gt;independentemente&lt;/em&gt;
&lt;/h3&gt;

&lt;p&gt;Adicionou &lt;code&gt;distância&lt;/code&gt; e &lt;code&gt;tempo em trânsito&lt;/code&gt; no mesmo modelo? Elas andam juntas — embarques mais longos tendem a ter mais tempo em trânsito.&lt;/p&gt;

&lt;p&gt;Isso é &lt;strong&gt;multicolinearidade&lt;/strong&gt;. O modelo não consegue separar:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;O lead time aumenta por causa da distância ou do tempo em trânsito?&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Em regressão múltipla a pergunta vira: &lt;strong&gt;existe variação em X₁ que não seja só repetição de X₂?&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  Resumo mental para guardar
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Var(β̂₁) = ruído / variação de X
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Situação&lt;/th&gt;
&lt;th&gt;Efeito&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Pouca variação em X&lt;/td&gt;
&lt;td&gt;Estimativa instável ⚠️&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Variação só por outlier&lt;/td&gt;
&lt;td&gt;Confiança falsa ⚠️&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;X e X₂ colineares&lt;/td&gt;
&lt;td&gt;Multicolinearidade ⚠️&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Variação ampla e útil&lt;/td&gt;
&lt;td&gt;Estimativa confiável ✅&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;p&gt;A ideia central é simples:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Para estimar o efeito de X, o modelo precisa observar X mudando.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Se seus dados de supply chain vêm de uma janela temporal curta, de uma região só, ou de um perfil de fornecedor muito homogêneo — revise antes de confiar nos coeficientes.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Curtiu? Me segue para mais conteúdo de estatística aplicada a dados de supply chain.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linearregression</category>
      <category>machinelearning</category>
      <category>statistics</category>
    </item>
    <item>
      <title>Finally fixed Linux Bluetooth audio stutter - Realtek Chip</title>
      <dc:creator>PtchNote</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:15:25 +0000</pubDate>
      <link>https://dev.to/ptchnote/finally-fixed-linux-bluetooth-audio-stutter-realtek-chip-4fmp</link>
      <guid>https://dev.to/ptchnote/finally-fixed-linux-bluetooth-audio-stutter-realtek-chip-4fmp</guid>
      <description>&lt;p&gt;After 2 days of digging through forums, testing kernel parameters, and trying multiple distros, I finally eliminated the infuriating audio stutter on my setup. Posting this here so others don't have to suffer through the same trial and error.&lt;/p&gt;

&lt;p&gt;Hardware:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;HP Pavilion 14‑dv1xxx

Realtek RTL8822CE Wi‑Fi/Bluetooth combo card

Sonos Roam (Bluetooth speaker)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;The problem: Bluetooth audio (especially during video playback) stuttered constantly. Music was sometimes okay, but YouTube, VLC, and any video content was a mess. The stutter was caused by Wi‑Fi power management interfering with Bluetooth — a known issue with this Realtek chipset.&lt;/p&gt;

&lt;p&gt;The fix (tested on Debian 13 and Kubuntu 26.04):&lt;/p&gt;

&lt;p&gt;Disable Wi‑Fi power management at the driver level:&lt;br&gt;
bash&lt;/p&gt;

&lt;p&gt;echo "options rtw88_core disable_lps_deep=Y" | sudo tee /etc/modprobe.d/rtw88_core.conf&lt;br&gt;
sudo update-initramfs -u&lt;br&gt;
sudo reboot&lt;/p&gt;

&lt;p&gt;Optional but helpful tweaks (applied at the same time):&lt;br&gt;
bash&lt;/p&gt;

&lt;h1&gt;
  
  
  Prevent Bluetooth USB resets
&lt;/h1&gt;

&lt;p&gt;echo "options btusb reset=0" | sudo tee /etc/modprobe.d/btusb.conf&lt;/p&gt;

&lt;h1&gt;
  
  
  Increase PipeWire buffer for video playback
&lt;/h1&gt;

&lt;p&gt;mkdir -p ~/.config/pipewire/pipewire.conf.d/&lt;br&gt;
cat &amp;lt;&amp;lt; EOF &amp;gt; ~/.config/pipewire/pipewire.conf.d/99-bluetouch.conf&lt;br&gt;
context.properties = {&lt;br&gt;
    default.clock.rate = 48000&lt;br&gt;
    default.clock.allowed-rates = [ 48000 ]&lt;br&gt;
    default.clock.quantum = 2048&lt;br&gt;
    default.clock.min-quantum = 1024&lt;br&gt;
}&lt;br&gt;
pipewire.tx = {&lt;br&gt;
    link.max-buffers = 256&lt;br&gt;
}&lt;br&gt;
EOF&lt;/p&gt;

&lt;h1&gt;
  
  
  Apply all changes
&lt;/h1&gt;

&lt;p&gt;sudo update-initramfs -u&lt;br&gt;
systemctl --user restart pipewire pipewire-pulse&lt;br&gt;
sudo reboot&lt;/p&gt;

&lt;p&gt;Why this works:&lt;br&gt;
The Realtek RTL8822CE aggressively manages Wi‑Fi power states (lps_deep = Low Power State Deep). Every time Wi‑Fi wakes or sleeps, it creates interference with Bluetooth. Disabling this keeps the Wi‑Fi radio in a more consistent state, eliminating the bursty interference that causes audio stutters — especially during video playback (which is bursty by nature).&lt;/p&gt;

&lt;p&gt;Verification:&lt;br&gt;
To confirm the fix is active after reboot:&lt;br&gt;
bash&lt;/p&gt;

&lt;p&gt;cat /sys/module/rtw88_core/parameters/disable_lps_deep&lt;/p&gt;

&lt;p&gt;If it returns Y, the fix is applied. If it returns N, something went wrong — double-check the config file and that you ran sudo update-initramfs -u.&lt;/p&gt;

&lt;p&gt;Tested with The Matrix lobby scene in VLC and YouTube. Zero stutters after the fix. Music (Spotify) also improved.&lt;/p&gt;

&lt;p&gt;One more tip:&lt;br&gt;
If you have access to 5 GHz Wi‑Fi, use it. Bluetooth operates on 2.4 GHz, so 5 GHz eliminates interference entirely. But the driver fix above makes 2.4 GHz usable too.&lt;/p&gt;

&lt;p&gt;Hope this saves someone else the two days I lost. 🎧🐧&lt;/p&gt;

</description>
      <category>linux</category>
      <category>performance</category>
      <category>systems</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Docker vs Kubernetes: Stop Comparing Them Like They Compete</title>
      <dc:creator>Mahendra Singh</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:14:59 +0000</pubDate>
      <link>https://dev.to/akoode_tech/docker-vs-kubernetes-stop-comparing-them-like-they-compete-4c4e</link>
      <guid>https://dev.to/akoode_tech/docker-vs-kubernetes-stop-comparing-them-like-they-compete-4c4e</guid>
      <description>&lt;p&gt;Every developer hitting their first production deployment runs into this question: &lt;strong&gt;Do I need Docker, Kubernetes, or both?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;It's the wrong framing. They don't compete. They operate at completely different layers of your infrastructure.&lt;/p&gt;

&lt;p&gt;Let me be blunt upfront:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Docker&lt;/strong&gt; = packages and runs your app as a container (single host)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Kubernetes&lt;/strong&gt; = manages and scales those containers across many hosts&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;You don't choose between them. You choose &lt;em&gt;when&lt;/em&gt; to graduate from one to the other.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Docker Actually Does
&lt;/h2&gt;

&lt;p&gt;Docker solves the "works on my machine" problem. It bundles your app code, runtime, libraries, and config into a &lt;strong&gt;container image&lt;/strong&gt; — a portable artifact that runs identically everywhere Docker is installed.&lt;/p&gt;

&lt;p&gt;Here's the simplest possible Dockerfile:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight docker"&gt;&lt;code&gt;&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="s"&gt; node:20-alpine&lt;/span&gt;
&lt;span class="k"&gt;WORKDIR&lt;/span&gt;&lt;span class="s"&gt; /app&lt;/span&gt;
&lt;span class="k"&gt;COPY&lt;/span&gt;&lt;span class="s"&gt; package*.json ./&lt;/span&gt;
&lt;span class="k"&gt;RUN &lt;/span&gt;npm ci &lt;span class="nt"&gt;--only&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;production
&lt;span class="k"&gt;COPY&lt;/span&gt;&lt;span class="s"&gt; . .&lt;/span&gt;
&lt;span class="k"&gt;EXPOSE&lt;/span&gt;&lt;span class="s"&gt; 3000&lt;/span&gt;
&lt;span class="k"&gt;CMD&lt;/span&gt;&lt;span class="s"&gt; ["node", "server.js"]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Build it, push it, run it anywhere:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker build &lt;span class="nt"&gt;-t&lt;/span&gt; my-api:1.0 &lt;span class="nb"&gt;.&lt;/span&gt;
docker push my-registry/my-api:1.0
docker run &lt;span class="nt"&gt;-p&lt;/span&gt; 3000:3000 my-registry/my-api:1.0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That image behaves the same on your MacBook, a CI runner, or a prod server. That's the whole value prop.&lt;/p&gt;

&lt;h3&gt;
  
  
  Docker Compose for Multi-Service Dev
&lt;/h3&gt;

&lt;p&gt;For local development with multiple services, Docker Compose is your best friend:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# docker-compose.yml&lt;/span&gt;
&lt;span class="na"&gt;services&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;api&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;build&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;./api&lt;/span&gt;
    &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;3000:3000"&lt;/span&gt;
    &lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;DATABASE_URL=postgres://user:pass@db:5432/myapp&lt;/span&gt;
    &lt;span class="na"&gt;depends_on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;db&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;redis&lt;/span&gt;

  &lt;span class="na"&gt;db&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;postgres:16&lt;/span&gt;
    &lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;pgdata:/var/lib/postgresql/data&lt;/span&gt;
    &lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;POSTGRES_PASSWORD&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;pass&lt;/span&gt;

  &lt;span class="na"&gt;redis&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;redis:7-alpine&lt;/span&gt;

&lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;pgdata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker compose up &lt;span class="nt"&gt;-d&lt;/span&gt;     &lt;span class="c"&gt;# spin everything up&lt;/span&gt;
docker compose logs &lt;span class="nt"&gt;-f&lt;/span&gt;   &lt;span class="c"&gt;# follow logs&lt;/span&gt;
docker compose down      &lt;span class="c"&gt;# tear it all down&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One command spins up your entire local stack. This is where most teams should live until they genuinely need orchestration.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Kubernetes Actually Does
&lt;/h2&gt;

&lt;p&gt;Docker works great on &lt;em&gt;one machine&lt;/em&gt;. The moment you need your app running across &lt;em&gt;multiple&lt;/em&gt; machines — with auto-scaling, self-healing, load balancing, and zero-downtime deploys — Docker alone doesn't cut it.&lt;/p&gt;

&lt;p&gt;Kubernetes (K8s) is the orchestration layer. It doesn't build containers. It &lt;strong&gt;runs and manages them across a cluster&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Here's what K8s handles automatically:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Which node to schedule each container on&lt;/li&gt;
&lt;li&gt;Restarting crashed containers&lt;/li&gt;
&lt;li&gt;Scaling up/down based on CPU/memory load&lt;/li&gt;
&lt;li&gt;Distributing traffic across replicas&lt;/li&gt;
&lt;li&gt;Rolling out updates without downtime&lt;/li&gt;
&lt;li&gt;Managing secrets and config&lt;/li&gt;
&lt;li&gt;Network routing between services&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The mental model: you declare your desired state, and K8s continuously reconciles reality toward it.&lt;/p&gt;

&lt;h3&gt;
  
  
  A Real Kubernetes Deployment
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# deployment.yaml&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;apps/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Deployment&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;replicas&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
  &lt;span class="na"&gt;selector&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;matchLabels&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api&lt;/span&gt;
  &lt;span class="na"&gt;template&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;labels&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api&lt;/span&gt;
    &lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;containers&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;api&lt;/span&gt;
          &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-registry/my-api:1.0&lt;/span&gt;
          &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;containerPort&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3000&lt;/span&gt;
          &lt;span class="na"&gt;resources&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;requests&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
              &lt;span class="na"&gt;cpu&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;100m"&lt;/span&gt;
              &lt;span class="na"&gt;memory&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;128Mi"&lt;/span&gt;
            &lt;span class="na"&gt;limits&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
              &lt;span class="na"&gt;cpu&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;500m"&lt;/span&gt;
              &lt;span class="na"&gt;memory&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;256Mi"&lt;/span&gt;
          &lt;span class="na"&gt;readinessProbe&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;httpGet&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
              &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;/health&lt;/span&gt;
              &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3000&lt;/span&gt;
            &lt;span class="na"&gt;initialDelaySeconds&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;5&lt;/span&gt;
            &lt;span class="na"&gt;periodSeconds&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;10&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Service&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api-svc&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;selector&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api&lt;/span&gt;
  &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;port&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;80&lt;/span&gt;
      &lt;span class="na"&gt;targetPort&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3000&lt;/span&gt;
  &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ClusterIP&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl apply &lt;span class="nt"&gt;-f&lt;/span&gt; deployment.yaml
kubectl get pods
kubectl rollout status deployment/my-api
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If one of those 3 pods crashes, K8s replaces it automatically. Scale on demand:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;kubectl scale deployment my-api &lt;span class="nt"&gt;--replicas&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;10
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Or set up auto-scaling based on CPU:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;autoscaling/v2&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;HorizontalPodAutoscaler&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api-hpa&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;scaleTargetRef&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;apps/v1&lt;/span&gt;
    &lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Deployment&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api&lt;/span&gt;
  &lt;span class="na"&gt;minReplicas&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
  &lt;span class="na"&gt;maxReplicas&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;20&lt;/span&gt;
  &lt;span class="na"&gt;metrics&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Resource&lt;/span&gt;
      &lt;span class="na"&gt;resource&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;cpu&lt;/span&gt;
        &lt;span class="na"&gt;target&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Utilization&lt;/span&gt;
          &lt;span class="na"&gt;averageUtilization&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;70&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Container vs Pod: The Confusion Cleared Up
&lt;/h2&gt;

&lt;p&gt;This trips up everyone new to K8s.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Container&lt;/strong&gt; = one packaged process. Docker's unit.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Pod&lt;/strong&gt; = K8s's deployable unit. Wraps one or more containers that share:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The same network namespace (they talk via &lt;code&gt;localhost&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;The same storage volumes&lt;/li&gt;
&lt;li&gt;The same lifecycle&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Most pods are single-container. But the sidecar pattern is real — you'll use it for logging agents, proxies (like Envoy in a service mesh), or secret injectors:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;containers&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;app&lt;/span&gt;
      &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api:1.0&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;log-shipper&lt;/span&gt;        &lt;span class="c1"&gt;# sidecar&lt;/span&gt;
      &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;fluentd:latest&lt;/span&gt;
      &lt;span class="na"&gt;volumeMounts&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;logs&lt;/span&gt;
          &lt;span class="na"&gt;mountPath&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;/var/log/app&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Why it matters practically&lt;/strong&gt;: K8s schedules pods, not individual containers. Resource requests, limits, and self-healing all operate at the pod level. If a pod crashes, K8s replaces the whole pod — not just the container inside it.&lt;/p&gt;




&lt;h2&gt;
  
  
  How Docker and K8s Fit Together in a Real Pipeline
&lt;/h2&gt;

&lt;p&gt;They're sequential, not overlapping:&lt;br&gt;
Developer writes code&lt;/p&gt;

&lt;p&gt;→ Dockerfile defines the build&lt;br&gt;
→ CI runs: docker build + docker push → registry&lt;br&gt;
→ K8s pulls image from registry&lt;br&gt;
→ K8s schedules pods across nodes&lt;br&gt;
→ K8s manages lifecycle, scaling, health&lt;/p&gt;

&lt;p&gt;A typical GitHub Actions pipeline wiring both together:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;jobs&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;build-and-deploy&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;runs-on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ubuntu-latest&lt;/span&gt;
    &lt;span class="na"&gt;steps&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;actions/checkout@v4&lt;/span&gt;

      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Build and push image&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
          &lt;span class="s"&gt;docker build -t $REGISTRY/my-api:$GITHUB_SHA .&lt;/span&gt;
          &lt;span class="s"&gt;docker push $REGISTRY/my-api:$GITHUB_SHA&lt;/span&gt;

      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Deploy to K8s&lt;/span&gt;
        &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
          &lt;span class="s"&gt;kubectl set image deployment/my-api \&lt;/span&gt;
            &lt;span class="s"&gt;api=$REGISTRY/my-api:$GITHUB_SHA&lt;/span&gt;
          &lt;span class="s"&gt;kubectl rollout status deployment/my-api&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Docker handles the build. Kubernetes handles the deploy. Clean handoff.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note on runtimes&lt;/strong&gt;: Kubernetes uses &lt;code&gt;containerd&lt;/code&gt; as its default runtime, not Docker directly. But Docker-built images follow the OCI standard, so they're fully compatible. You don't need to change your build workflow.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  When to Use Docker Without Kubernetes ✅
&lt;/h2&gt;

&lt;p&gt;Use Docker alone when:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You're in early development or pre-PMF&lt;/li&gt;
&lt;li&gt;Your app runs on a single server&lt;/li&gt;
&lt;li&gt;Your team is 1–3 engineers&lt;/li&gt;
&lt;li&gt;You have fewer than ~10 containers total&lt;/li&gt;
&lt;li&gt;You're serving under ~10k DAU&lt;/li&gt;
&lt;li&gt;You're on a managed PaaS (Railway, Fly.io, Cloud Run) that abstracts orchestration for you&lt;/li&gt;
&lt;li&gt;Fast iteration matters more than scaling headroom right now&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Docker Compose handles multi-service local dev and even modest production deployments cleanly. Adding K8s at this stage is premature optimization with real engineering cost.&lt;/p&gt;

&lt;h2&gt;
  
  
  When to Add Kubernetes ✅
&lt;/h2&gt;

&lt;p&gt;Add K8s when:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You're running 5+ microservices that need coordinated deployment and networking&lt;/li&gt;
&lt;li&gt;Traffic spikes are real and you need auto-scaling&lt;/li&gt;
&lt;li&gt;You have an uptime SLA above 99.9% that requires self-healing&lt;/li&gt;
&lt;li&gt;Multiple teams deploy independently to shared infrastructure&lt;/li&gt;
&lt;li&gt;You need canary releases, blue/green deploys, or sophisticated rollout strategies&lt;/li&gt;
&lt;li&gt;You're serving 100k+ DAU&lt;/li&gt;
&lt;li&gt;Your monthly compute spend is high enough that K8s bin-packing efficiency actually saves money&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A rough rule of thumb: if your compute bill is under $5k/month and you have fewer than 15 services, K8s will likely cost more in engineering time than it saves in ops.&lt;/p&gt;

&lt;h2&gt;
  
  
  When NOT to Use Kubernetes ❌
&lt;/h2&gt;

&lt;p&gt;Avoid K8s when:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;No one on your team has real K8s operational experience&lt;/li&gt;
&lt;li&gt;You're a solo dev or very small startup&lt;/li&gt;
&lt;li&gt;Your stack is a monolith or 2–3 services&lt;/li&gt;
&lt;li&gt;Feature velocity is the priority and K8s YAML would slow you down&lt;/li&gt;
&lt;li&gt;A managed PaaS already gives you what you need&lt;/li&gt;
&lt;li&gt;You're adding it because it looks good on the architecture diagram&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I've seen small teams burn weeks configuring ingress controllers, cert-manager, and RBAC for a 3-service app. The overhead is real. Don't add it until the pain of &lt;em&gt;not&lt;/em&gt; having orchestration is specific and concrete.&lt;/p&gt;

&lt;h2&gt;
  
  
  Quick Comparison
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Factor&lt;/th&gt;
&lt;th&gt;Docker&lt;/th&gt;
&lt;th&gt;Kubernetes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Primary job&lt;/td&gt;
&lt;td&gt;Build + run containers&lt;/td&gt;
&lt;td&gt;Orchestrate at scale&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scope&lt;/td&gt;
&lt;td&gt;Single host&lt;/td&gt;
&lt;td&gt;Multi-host cluster&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Setup time&lt;/td&gt;
&lt;td&gt;Minutes&lt;/td&gt;
&lt;td&gt;Hours to days&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Auto-scaling&lt;/td&gt;
&lt;td&gt;❌&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Self-healing&lt;/td&gt;
&lt;td&gt;❌&lt;/td&gt;
&lt;td&gt;✅&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Rolling updates&lt;/td&gt;
&lt;td&gt;Manual&lt;/td&gt;
&lt;td&gt;Automated&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Learning curve&lt;/td&gt;
&lt;td&gt;Days&lt;/td&gt;
&lt;td&gt;Weeks–months&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Best for&lt;/td&gt;
&lt;td&gt;Dev, small scale&lt;/td&gt;
&lt;td&gt;Production, microservices&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Learning Path
&lt;/h2&gt;

&lt;p&gt;Docker learning curve: &lt;strong&gt;days.&lt;/strong&gt;&lt;br&gt;
Kubernetes learning curve: &lt;strong&gt;weeks to months.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The progression that actually works:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Get fluent with &lt;code&gt;docker build&lt;/code&gt;, &lt;code&gt;docker run&lt;/code&gt;, image layers, multi-stage builds&lt;/li&gt;
&lt;li&gt;Master Docker Compose for local multi-service dev&lt;/li&gt;
&lt;li&gt;Understand container networking (bridge, host, overlay)&lt;/li&gt;
&lt;li&gt;Then introduce K8s concepts: pods → deployments → services → ingress → namespaces → RBAC&lt;/li&gt;
&lt;li&gt;Run locally with &lt;code&gt;kind&lt;/code&gt; or &lt;code&gt;minikube&lt;/code&gt; before touching production
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Spin up a local K8s cluster for learning&lt;/span&gt;
kind create cluster &lt;span class="nt"&gt;--name&lt;/span&gt; dev
kubectl cluster-info &lt;span class="nt"&gt;--context&lt;/span&gt; kind-dev
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Managed K8s options when you're ready for production:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Cloud&lt;/th&gt;
&lt;th&gt;Service&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;AWS&lt;/td&gt;
&lt;td&gt;EKS&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;GCP&lt;/td&gt;
&lt;td&gt;GKE&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Azure&lt;/td&gt;
&lt;td&gt;AKS&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;They manage the control plane. You manage your workloads.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Docker and Kubernetes aren't rivals. They're different layers of the same stack.&lt;/li&gt;
&lt;li&gt;Docker Compose in production is a legitimate, underrated choice for small/medium workloads.&lt;/li&gt;
&lt;li&gt;The Docker → K8s transition is a natural graduation, not a required step.&lt;/li&gt;
&lt;li&gt;Before rolling your own cluster, check if Cloud Run, App Runner, or Container Apps gives you 80% of K8s for 20% of the complexity.&lt;/li&gt;
&lt;li&gt;Master Docker deeply first. The mental models (images, layers, networking, volumes) carry directly into K8s.&lt;/li&gt;
&lt;li&gt;Add K8s when the pain of &lt;em&gt;not&lt;/em&gt; having orchestration is specific and real — not theoretical.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://www.akoode.com/blog/docker-vs-kubernetes" rel="noopener noreferrer"&gt;akoode.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>docker</category>
      <category>kubernetes</category>
      <category>devops</category>
      <category>containers</category>
    </item>
    <item>
      <title>Building WeRemember in Public — First Knots After Starting Over</title>
      <dc:creator>Sandro Hu</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:12:11 +0000</pubDate>
      <link>https://dev.to/oranguengineer/building-weremember-in-public-first-knots-after-starting-over-537</link>
      <guid>https://dev.to/oranguengineer/building-weremember-in-public-first-knots-after-starting-over-537</guid>
      <description>&lt;p&gt;Yesterday I wrote that I was starting over.&lt;/p&gt;

&lt;p&gt;Today was the first real test of that decision: could I restart without rebuilding the same overhead under a cleaner name?&lt;/p&gt;

&lt;p&gt;I did not start with a landing page. I did not start with a VPS. I did not start with monitoring or a content backlog.&lt;/p&gt;

&lt;p&gt;I started with the things that decide whether the project has a spine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Repository split
&lt;/h2&gt;

&lt;p&gt;The community edition lives on GitHub as an AGPL-licensed open source project — the source of truth for the core product.&lt;/p&gt;

&lt;p&gt;Cloud and infrastructure concerns live separately in private repositories.&lt;/p&gt;

&lt;p&gt;The important decision is that the cloud version will not fork the community edition: It depends on it as a Python package, imports core apps, and extends them. When the community edition releases, the cloud version updates a dependency. No manual sync.&lt;/p&gt;

&lt;p&gt;The plan is for the cloud version to eventually become a managed offering — for those who want WeRemember without the self-hosting overhead, or who need enterprise features like SSO, audit logs, or RBAC. The community edition stays the open source core either way.&lt;/p&gt;

&lt;h2&gt;
  
  
  CI/CD boundary
&lt;/h2&gt;

&lt;p&gt;CI builds and pushes images to public registries. It knows nothing about the deployment target.&lt;/p&gt;

&lt;p&gt;Deployment belongs to a separate private pipeline.&lt;/p&gt;

&lt;p&gt;The community repository stays clean. It knows only that an image is ready,not where it goes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Governance scaffold
&lt;/h2&gt;

&lt;p&gt;I opened PR #1 before writing any application code. It includes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;issue and pull request templates&lt;/li&gt;
&lt;li&gt;label taxonomy managed via a committed setup script&lt;/li&gt;
&lt;li&gt;branch naming: &lt;code&gt;&amp;lt;type&amp;gt;/&amp;lt;issue-number&amp;gt;-&amp;lt;short-description&amp;gt;&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Conventional Commits&lt;/li&gt;
&lt;li&gt;branch ruleset documented as &lt;code&gt;gh api&lt;/code&gt; commands&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The issue number is mandatory on every branch. Every branch must point back to tracked work.&lt;/p&gt;

&lt;p&gt;Labels and rulesets are version-controlled and reproducible, not configured once through the GitHub UI and forgotten.&lt;/p&gt;

&lt;p&gt;———&lt;/p&gt;

&lt;p&gt;Short updates on &lt;a href="//x.com/OranguEngineer"&gt;X&lt;/a&gt; | Articles on &lt;a href="//x.com/oranguengineer"&gt;dev.to&lt;/a&gt; | Code on &lt;a href="//github.com/OranguEngineer"&gt;GitHub&lt;/a&gt;&lt;/p&gt;

</description>
      <category>buildinpublic</category>
      <category>django</category>
      <category>opensource</category>
      <category>architecture</category>
    </item>
    <item>
      <title>Refactoring is beyond the code and structure issues...</title>
      <dc:creator>Camila Rody</dc:creator>
      <pubDate>Tue, 02 Jun 2026 20:11:38 +0000</pubDate>
      <link>https://dev.to/camilasrody/refactoring-is-beyond-the-code-and-structure-issues-bk7</link>
      <guid>https://dev.to/camilasrody/refactoring-is-beyond-the-code-and-structure-issues-bk7</guid>
      <description>&lt;p&gt;When we talk about refactoring, we usually think about reducing complexity, eliminating duplication, improving naming, or applying design patterns. All of these are important parts of the process, but over the years I've come to realize that the best refactoring efforts rarely start with code. They start with understanding.&lt;/p&gt;

&lt;p&gt;There's a natural tendency to look at an existing system and judge it based on what we know today. We open a massive component, a complicated business rule, or an architecture that feels outdated and immediately think, &lt;em&gt;"I would do this differently."&lt;/em&gt; And perhaps we would. The problem is that this perspective often ignores a fundamental reality: every software system is a reflection of the circumstances in which it was built.&lt;/p&gt;

&lt;p&gt;Code does not emerge in a vacuum. It is shaped by deadlines, technical constraints, business priorities, budget limitations, organizational pressures, and the experience level of the people involved. A decision that appears questionable today may have been the most rational choice available at the time. Perhaps the team was racing to validate a product, responding to a critical business need, or operating with far fewer resources than we have now. Maybe the technologies, frameworks, or patterns we take for granted today simply didn't exist when those decisions were made.&lt;/p&gt;

&lt;p&gt;This is why good refactoring requires much more than identifying technical flaws. It requires understanding why certain decisions were made in the first place. Before changing a system, it's important to understand the problems it was designed to solve, the constraints that influenced its design, and the business goals it was expected to support. Without that context, we risk confusing evolution with error.&lt;/p&gt;

&lt;p&gt;In many cases, the code itself isn't the problem. What has changed is everything around it. The business has evolved. Requirements have shifted. The team has grown. Technology has advanced. The scale of the application has increased. A solution that worked perfectly for hundreds of users may no longer be suitable for millions. An architecture that was ideal for a startup may struggle within a mature enterprise. In these situations, the original implementation wasn't necessarily wrong—it was simply built for a different reality.&lt;/p&gt;

&lt;p&gt;That's why I increasingly see refactoring as an exercise in investigation. Before changing anything, I want to understand the story behind the system. What challenges was it trying to solve? What compromises were made? What assumptions guided its design? What constraints influenced its architecture? The answers to those questions are often far more valuable than any design pattern or technical framework.&lt;/p&gt;

&lt;p&gt;Another aspect that is often overlooked is that refactoring isn't just about improving technical quality. It's also about preserving knowledge. Every codebase contains years of accumulated decisions, lessons learned, production incidents, business discoveries, and engineering trade-offs. Much of that knowledge never makes it into documentation, tickets, or commit messages. Instead, it becomes embedded in the structure of the system itself.&lt;/p&gt;

&lt;p&gt;When we ignore that history, we risk removing far more than code. We may unintentionally discard valuable business knowledge, hard-earned operational experience, and solutions that exist for reasons no longer immediately visible. A mature refactoring effort doesn't simply ask what should change. It also asks what should be preserved.&lt;/p&gt;

&lt;p&gt;Ultimately, refactoring is far less about rewriting code and far more about understanding decisions. It requires context, empathy, and a long-term perspective. Before building the next version of a system, we need to understand why the current version exists.&lt;/p&gt;

&lt;p&gt;Because at its core, refactoring is the practice of understanding the past in order to build a better future.&lt;/p&gt;

</description>
      <category>discuss</category>
      <category>programming</category>
      <category>softwareengineering</category>
      <category>refactoring</category>
    </item>
  </channel>
</rss>
