DEV Community: M TOQEER ZIA

Horizontal vs. Vertical Scaling: Which One Should You Actually Use?

M TOQEER ZIA — Sat, 11 Jul 2026 11:32:21 +0000

Every growing application eventually hits the same wall: the current infrastructure isn't enough. More users, more data, more requests per second — and suddenly "just add more resources" becomes a real engineering decision, not a throwaway line in a system design interview.

That decision usually comes down to two paths: scale up (vertical) or scale out (horizontal). They sound like a matter of preference, but they come with very different tradeoffs in cost, complexity, and failure modes. Let's break both down.

Vertical Scaling: Make the Machine Bigger

Vertical scaling (scaling up) means adding more power to a single machine — more CPU, more RAM, faster disks (e.g., moving from SSD to NVMe).

Before: 4 vCPU, 16GB RAM
After:  16 vCPU, 64GB RAM

On most cloud providers, this is often as simple as changing an instance type and restarting.

Example (AWS EC2):

# Stop the instance
aws ec2 stop-instances --instance-ids i-0123456789abcdef0

# Resize
aws ec2 modify-instance-attribute \
  --instance-id i-0123456789abcdef0 \
  --instance-type "{\"Value\": \"m5.2xlarge\"}"

# Start it back up
aws ec2 start-instances --instance-ids i-0123456789abcdef0

Pros

Simple. No architecture changes, no distributed systems complexity.
No data consistency issues. One machine, one source of truth.
Great for stateful workloads like traditional relational databases that aren't built for sharding.

Cons

Hard ceiling. Eventually you hit the biggest instance type available.
Single point of failure. If that one machine goes down, everything goes down.
Downtime during resize, in most setups.
Cost grows non-linearly — bigger machines get disproportionately more expensive.

Horizontal Scaling: Add More Machines

Horizontal scaling (scaling out) means adding more instances of your application and distributing load across them, typically behind a load balancer.

Before: 1 server handling all traffic
After:  5 servers behind a load balancer, each handling ~20%

Example (Docker Compose, scaling a service):

docker compose up --scale web=5 -d

Example (Kubernetes, via Horizontal Pod Autoscaler):

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Pros

No practical ceiling. Add as many nodes as your budget and architecture allow.
Better fault tolerance. One node dying doesn't take down the whole system.
Elastic cost. Scale down during low traffic, scale up during peaks — pay for what you use.
Enables rolling deployments with zero downtime.

Cons

Requires stateless design (or careful state management via shared caches, sessions stores like Redis, etc.).
Distributed systems problems appear: data consistency, network latency, race conditions.
More moving parts: load balancers, service discovery, health checks, orchestration.
Debugging is harder — logs and traces are spread across many nodes.

Side-by-Side Comparison

	Vertical Scaling	Horizontal Scaling
Complexity	Low	Higher (distributed systems)
Upper limit	Hardware ceiling	Practically unlimited
Fault tolerance	Single point of failure	Resilient to node failure
Cost curve	Non-linear, gets expensive fast	More linear, elastic
Downtime to scale	Usually yes	No (add/remove nodes live)
Best fit	Traditional RDBMS, legacy monoliths	Stateless services, microservices, web APIs

So Which One Should You Pick?

In practice, most real systems use both, applied to different layers:

Application/API layer → horizontal scaling. Stateless services behind a load balancer scale out cleanly and give you fault tolerance for free.
Database layer → often vertical scaling first (bigger instance, more RAM for caching, faster disks), because re-architecting for horizontal scaling (sharding, read replicas, distributed databases like CockroachDB or Cassandra) is a much bigger investment.
Cache layer (Redis, Memcached) → horizontal scaling via clustering once a single node's memory becomes the bottleneck.

A common growth path looks like this:

Start on a single small server (nothing to scale yet).
Hit load limits → scale vertically (bigger box). Fastest fix, buys time.
Vertical scaling gets expensive or hits a ceiling → move to horizontal scaling for the application tier.
Database becomes the bottleneck → introduce read replicas, then sharding or a distributed database if needed.

The Real Takeaway

Vertical scaling buys you time with minimal complexity. Horizontal scaling buys you resilience and (near) unlimited growth at the cost of architectural complexity. Neither is "better" — the right call depends on where your current bottleneck actually is, how stateful your system is, and how much complexity your team can realistically operate.

If you're not sure which one you need right now: profile first. Find out whether you're CPU-bound, memory-bound, I/O-bound, or connection-bound before reaching for either solution. Scaling the wrong dimension just gets you the same problem with a bigger bill.

Have you had to make this call in production? What tipped the decision one way or the other for you — cost, team size, or the nature of the workload? Drop it in the comments.

Load Balancing Explained: The Complete Guide to Types and Algorithms

M TOQEER ZIA — Sat, 11 Jul 2026 11:27:34 +0000

If you've ever wondered how sites like Amazon or Netflix handle millions of simultaneous users without crashing, the answer usually starts with one unsung hero of system design: the load balancer.

In this article, we'll break down what load balancing is, why it matters, and the different types and algorithms you'll encounter in real-world architectures.

What Is Load Balancing?

Load balancing is the process of distributing incoming network traffic across multiple servers so that no single server becomes overwhelmed. Instead of one server handling every request, a load balancer sits in front of a group of servers (often called a server pool or server farm) and routes each incoming request to the server best equipped to handle it.

Think of it like a checkout line at a grocery store. If everyone lines up at one register, that cashier gets overwhelmed while others sit idle. A good store manager (the load balancer) directs customers to open registers, keeping the whole line moving smoothly.

Why Load Balancing Matters

High availability – if one server fails, traffic is automatically rerouted to healthy servers
Scalability – you can add or remove servers based on demand without downtime
Performance – requests are distributed efficiently, reducing latency
Redundancy – protects against a single point of failure
Flexibility – enables maintenance and deployments without taking the whole system offline

How a Load Balancer Works

At a high level, a load balancer:

Receives an incoming client request
Checks the health/status of servers in the pool
Applies a routing algorithm to pick a server
Forwards the request to that server
Returns the response back to the client

Most load balancers also perform continuous health checks, pinging servers periodically to make sure they're still responsive before sending traffic their way.

Types of Load Balancers

1. Hardware Load Balancers

These are dedicated physical devices designed specifically to distribute traffic. They offer strong performance and reliability but come with a high upfront cost and limited flexibility. Common in large enterprises with strict compliance or performance requirements.

Pros: High throughput, dedicated resources, vendor support
Cons: Expensive, less flexible, harder to scale dynamically

2. Software Load Balancers

These run as applications on standard servers or virtual machines (e.g., NGINX, HAProxy, Traefik). They're cheaper, easier to configure, and integrate well with cloud-native and containerized environments.

Pros: Cost-effective, flexible, easy to automate
Cons: Shares resources with the host machine, may need tuning for extreme scale

3. Cloud-Based / Managed Load Balancers

Services like AWS Elastic Load Balancer (ELB), Google Cloud Load Balancing, and Azure Load Balancer are fully managed offerings. They scale automatically and integrate tightly with other cloud services.

Pros: Auto-scaling, minimal maintenance, pay-as-you-go
Cons: Vendor lock-in, less low-level control

Load Balancing by OSI Layer

Layer 4 (Transport Layer) Load Balancing

Operates at the transport layer, making routing decisions based on IP address and TCP/UDP port information — without inspecting the actual content of the packet.

Faster since it doesn't need to parse application data
Doesn't understand HTTP headers, cookies, or URLs
Good for simple, high-throughput scenarios

Layer 7 (Application Layer) Load Balancing

Operates at the application layer, meaning it can inspect the actual content of requests — HTTP headers, URLs, cookies, and even request bodies.

Enables smart routing, like sending /api/* requests to one set of servers and /images/* to another
Supports SSL termination, content-based routing, and session persistence
Slightly slower than Layer 4 due to deeper packet inspection

Common Load Balancing Algorithms

The algorithm determines how the load balancer chooses which server gets the next request.

1. Round Robin

Requests are distributed sequentially across the server pool, one after another. Simple and effective when all servers have similar capacity.

2. Weighted Round Robin

Similar to round robin, but servers are assigned weights based on their capacity. More powerful servers receive a proportionally larger share of traffic.

3. Least Connections

Routes traffic to the server with the fewest active connections. Ideal when requests vary significantly in processing time.

4. Weighted Least Connections

Combines the "least connections" logic with server capacity weighting — factoring in both current load and server power.

5. IP Hash

Uses the client's IP address to consistently route them to the same server, useful for maintaining session persistence without needing a shared session store.

6. Least Response Time

Sends traffic to the server with the fastest response time and fewest active connections, optimizing for speed.

7. Random / Random with Two Choices

Requests are distributed randomly, sometimes with a "power of two choices" optimization where the balancer picks the less-loaded of two randomly selected servers.

Global Server Load Balancing (GSLB)

Beyond distributing traffic across servers in one data center, GSLB distributes traffic across multiple data centers or geographic regions. This is essential for:

Reducing latency by routing users to the nearest region
Disaster recovery if an entire region goes down
Compliance with data residency requirements

Load Balancing vs. Reverse Proxy

These terms are often confused. A reverse proxy sits in front of servers and forwards client requests, but its primary job may be caching, SSL termination, or security — not necessarily distributing traffic across multiple backend servers. A load balancer is specifically focused on distributing traffic to prevent overload. In practice, many tools (like NGINX and HAProxy) can act as both.

Choosing the Right Load Balancer

Consideration	Recommendation
Simple traffic distribution	Round Robin or Least Connections
Session persistence needed	IP Hash or Layer 7 with sticky sessions
Mixed server capacities	Weighted Round Robin
Content-based routing	Layer 7 load balancer
Global user base	GSLB with regional data centers
Cloud-native infrastructure	Managed cloud load balancer (ELB, GCP LB, etc.)

Final Thoughts

Load balancing is a foundational concept in building scalable, resilient systems. Whether you're running a small side project or architecting infrastructure for millions of users, understanding these types and algorithms helps you make informed decisions about reliability and performance.

The right choice ultimately depends on your traffic patterns, infrastructure, and business requirements — there's no one-size-fits-all solution, but now you have the vocabulary and mental model to reason through it.

Found this useful? Feel free to share your own load balancing setups or war stories in the comments!

The New HTTP QUERY Method: Why It Exists and What Problem It Actually Solves

M TOQEER ZIA — Fri, 10 Jul 2026 19:36:50 +0000

If you've spent any time building backend systems, you know the HTTP verbs by heart: GET, POST, PUT, PATCH, DELETE. For decades, that list felt complete. Recently, though, a new method has entered the conversation — QUERY. Before diving into what it does, it's worth understanding why it was needed in the first place, because the reasoning tells you a lot about how the web actually works under the hood.

HTTP Isn't a Network Protocol — It's an Agreement

A common misconception is that HTTP is a network-level protocol like TCP or UDP. It isn't. HTTP sits at the application layer, built on top of TCP, which handles the actual transport of data between machines.

Here's the thing about TCP: it doesn't care what you send. You could open a TCP connection between two machines and transmit the string "hello," and it would arrive just fine. But that string carries no meaning. Is it a greeting? A command? Data to be saved? TCP has no concept of intent — it just moves bytes from one point to another.

This is the exact problem HTTP was designed to solve. Instead of every developer sending arbitrary, unstructured strings across the wire, HTTP introduced a standardized way to express intent. That's really what an HTTP method is: a declaration of what you're trying to do.

GET → I intend to retrieve something
POST → I intend to create something
PUT → I intend to replace something
PATCH → I intend to partially update something
DELETE → I intend to remove something

After the method comes the resource name (like /users), and after that, optional extras — headers (key-value metadata) and a body (the actual payload). This simple structure is what allows every server and client in the world, regardless of who built them, to understand each other.

Where GET Requests Start to Break Down

GET requests carry their parameters in the URL itself, using query strings — anything after the ?. This works beautifully for simple filtering: /users?name=piyush is clean, readable, and easy to reason about.

The trouble starts when your filtering needs grow. Real-world applications often need to combine multiple conditions — filter by name, restrict by age, match an email domain, sort, paginate, and more, all in a single request. Try to cram that into a query string and you run into real constraints:

URL length limits. Most browsers and servers cap URLs around 8,000 characters. Complex filters can blow past that.
Encoding overhead. Spaces and special characters aren't allowed raw in URLs — everything needs to go through encodeURIComponent or similar, making complex queries messy and error-prone.
Exposure in logs. Since query parameters live in the URL, they show up in server logs, browser history, and proxy logs — not ideal if any of that data is sensitive.

The Workaround Everyone Used (and Why It Was Broken)

Faced with these limits, developers found a workaround: if GET can't hold a complex query, just use POST instead. A POST request has a body, and a body can hold arbitrarily complex, structured JSON — filters, search terms, nested conditions, whatever you need, with no length restrictions and no awkward encoding.

So teams started building endpoints like POST /getUsers, reading the filter criteria from request.body, and running the query on the backend. Functionally, it worked. But it broke the semantic contract of REST.

The resource name says "get users." The method says "I intend to create something." That mismatch isn't just a style nitpick — it has real technical consequences:

POST requests are never cacheable. Caching systems (whether at the CDN layer or elsewhere) assume that identical POST requests might produce different side effects each time, so they're excluded from caching by default.
POST requests are not idempotent. Idempotency means that calling an operation repeatedly with the same input produces the same result every time. GET is naturally idempotent — calling GET /users/1 a thousand times returns the same user a thousand times, with no side effects. POST is the opposite: call POST /users five times with the same payload, and you'll likely end up with five different records, because POST's entire purpose is to create something new each time.

So the "use POST for complex reads" trick solved the encoding problem but created a caching and correctness problem. You couldn't cache these disguised read operations at all, even though they were, semantically, just reads.

Enter the QUERY Method

This is precisely the gap the new HTTP QUERY method fills. Conceptually, it's simple: QUERY behaves like GET — safe, cacheable, and idempotent — but it allows a request body, just like POST.

That combination is exactly what was missing. With QUERY:

Your intent is unambiguous: you're reading data, not creating or mutating it.
You can send arbitrarily complex filter logic in the request body — no URL length limits, no manual encoding gymnastics.
Because the method itself signals "this is a safe, repeatable read," CDNs and caching layers can treat QUERY requests as cacheable, the same way they treat GET.
Because it's idempotent, retry logic, deduplication, and other reliability patterns that depend on idempotency all work correctly.

In short, QUERY resolves a long-standing architectural tension between GET and POST — the need for expressive, complex read requests without sacrificing cacheability or correctness.

When Should You Actually Use It?

This doesn't mean GET is obsolete. For simple lookups and filters — a handful of query parameters — GET with query strings remains perfectly fine and arguably more transparent, since the full request is visible in the URL itself.

QUERY becomes the better choice when your read operations get complex enough that expressing them in a URL is impractical: multi-field search, nested filters, large parameter sets, or anything that previously would have forced you into the POST workaround.

The Bigger Lesson

It's worth remembering that REST conventions — GET fetches, POST creates, PUT updates — aren't laws of physics. They're self-imposed rules the developer community agreed on to keep systems interoperable. Nothing stops your server from working if you send data with a method named "hello" instead of GET. The value of these conventions is entirely in the shared understanding they create between client, server, and every piece of infrastructure (caches, proxies, logging tools) sitting in between.

The QUERY method is a good example of how these conventions evolve: not by breaking the system, but by formalizing a pattern developers were already reaching for through workarounds, and giving it the semantics it always should have had.

Building a Microservices App with Kubernetes: What I Learned the Hard Way

M TOQEER ZIA — Tue, 07 Jul 2026 12:52:32 +0000

From Docker to Minikube, Ingress, Services, and Skaffold in one real project

Introduction

I did not learn Kubernetes by reading theory alone. I learned it by building a real microservices app, breaking things, fixing them, and slowly seeing how all the parts fit together.

The project was a simple blogging app with posts and comments, but the architecture behind it was the real lesson. Instead of putting everything into one big backend, I split the app into small services: Posts, Comments, Moderation, Query, and an Event Bus. The frontend was a React client. Each service had one clear job. That choice forced me to understand containers, service discovery, deployments, networking, ingress, and local Kubernetes with Minikube.

This is my learning journey written in simple English. I am sharing what I built, why I built it this way, what confused me, and what finally made the whole picture click.

Introduction to the Project
Understanding Containers
Kubernetes Basics
Kubernetes Architecture
Deployments
Services
Load Balancing
Communication Between Microservices
Ingress Controller
Host File Configuration
Minikube
Minikube IP
Docker Inside Minikube
Skaffold
Important Commands Explained
Common Problems I Faced
Pros and Cons
Lessons I Learned
Best Practices
Conclusion
What’s Next?
SEO Details

1. Introduction to the Project

What is this project?

I built a small blogging app where users can create posts and add comments. The interesting part is not the app idea itself. The interesting part is how the app behaves behind the scenes.

The system is split into multiple services:

Posts Service creates posts
Comments Service creates comments
Moderation Service checks comments and approves or rejects them
Query Service keeps a read-optimized view for the UI
Event Bus forwards events between services
React Client shows the user interface

This project helped me understand event-driven microservices, CQRS, and eventual consistency. In simple words, one service does not try to do everything. Instead, each service does one thing well and talks to others through events.

Why microservices?

I asked myself a very practical question: why not just build one backend?

The answer became clearer as the project grew. A monolithic app is easier at the beginning, but it can become heavy later. Every change affects the same codebase and one bug can slow down the whole system.

Microservices solved a few real problems for me:

Each service could be built and understood separately
Each service could be deployed separately
One slow service did not need to block the others
It was easier to model real business flow as events
I could practice service-to-service communication the way larger systems do it

Why not a monolithic application?

I did think about a monolith first. For a small demo, that would have been simpler. But my goal was learning, not just shipping the shortest possible code.

If I had used one monolith, I would have missed the hard but useful parts:

How services discover each other
How Kubernetes manages containers
Why ingress matters
Why internal services use ClusterIP
How async events keep the system flexible

So I chose the harder path because it taught me more.

What problems microservices solve

Microservices help when different parts of an application need to move at different speeds.

For example, in my project:

Posts are created quickly
Comments may need moderation
The UI reads from a separate query model
The system should not freeze just because one service is busy

That is the kind of problem microservices handle well.

Architecture overview

Here is the high-level picture I worked with:

React Client
    |
    v
Ingress
    |
    +------------------------+
    |                        |
    v                        v
Posts Service           Query Service
    |                        ^
    v                        |
Event Bus --------------> Moderation Service
    |
    v
Comments Service

The flow is event-based. A request starts in the client, goes to a service, and then that service emits an event to the event bus. The event bus shares that event with the other services that care about it.

That separation was one of the most useful lessons in the whole journey.

2. Understanding Containers

What is Docker?

Docker was my entry point into this whole path. Before Docker, I used to think, “Why does an app work on my machine but fail somewhere else?” Docker made that question smaller.

Docker lets me package an application with everything it needs to run: code, runtime, dependencies, and configuration. That package is called an image. When the image runs, it becomes a container.

Why containers exist

Containers exist because software environments are messy.

One machine may have Node.js 16. Another may have Node.js 20. One machine may have missing packages. Another may have a different operating system. Containers reduce that problem by giving the app a predictable environment.

The easiest way I understood it was this:

An image is the blueprint
A container is the running house built from that blueprint

Images vs containers

This confused me at first, so I kept it simple.

Image: the saved template
Container: the live running instance

If I build the same image ten times, I get ten containers. They all come from the same starting point, but each container runs separately.

Why Kubernetes needs Docker

Strictly speaking, Kubernetes does not require Docker specifically anymore, but in my learning path Docker was the tool that built the container images Kubernetes used.

Kubernetes does not care how the image was built. It cares that the image exists and can be pulled by a node. Docker helped me create those images in a familiar way.

How containers communicate

In the project, containers communicate over the network. Each service listens on a port, and other services call that port using HTTP.

For example:

Posts Service listens on port 8000
Comments Service listens on port 8001
Query Service listens on port 8002
Moderation Service listens on port 8003
Event Bus listens on port 5000

One service sends a request to another service using its network address and port. That is simple in theory, but in Kubernetes it becomes easier when we use Services and DNS names instead of raw IP addresses.

Simple example

If I want the Posts Service to send an event, it posts to the Event Bus endpoint:

Posts Service -> http://event-bus-srv:5000/events

That internal address is much better than hardcoding a Pod IP, because Pods can change.

3. Kubernetes Basics

What is Kubernetes?

Kubernetes is a system that helps me run containers across one or more machines. It handles placement, restarts, scaling, and networking.

The easiest way I can describe it is this: Docker runs containers, but Kubernetes organizes them.

Why Kubernetes exists

I used to think Kubernetes was just a fancy way to run Docker. It is not.

It exists because running one container is easy, but running many containers reliably is hard. Once I had more than one service, I started asking real orchestration questions:

What if a container dies?
What if traffic increases?
Which machine should run which container?
How do services find each other?
How do I update a service without breaking everything?

Kubernetes helps answer those questions.

What problems Kubernetes solves

Kubernetes solves a few practical problems:

It restarts failed containers
It spreads containers across nodes
It scales applications up and down
It gives containers stable network identities through Services
It supports rolling updates
It makes local and production setups more similar

Cluster

A cluster is the whole Kubernetes environment. It is the group of machines that work together.

I like to think of it like a small company. The company is the cluster. Inside it, there are different employees doing different jobs.

Node

A node is one machine inside the cluster. It is where Pods run.

If the cluster is the company, the node is one office building.

Master Node or Control Plane

The master node, also called the control plane, is the brain of Kubernetes.

It decides:

Which node gets a Pod
What happens when a Pod dies
When replicas should be created or removed
How the cluster stays in the desired state

I do not normally run my app there. I let it manage the app.

Worker Node

Worker nodes are the machines that actually run my Pods and containers.

If the control plane is the manager, worker nodes are the people doing the actual work.

Pod

A Pod is the smallest deployable unit in Kubernetes.

This was one of my biggest mental shifts. Kubernetes does not usually run a raw container directly. It runs a Pod, and the Pod contains one or more containers.

I usually used one container per Pod in this project.

Container inside a Pod

A container inside a Pod is the actual process running my app.

The Pod gives it networking and shared storage if needed. The container is still the thing running Node.js or React.

Replica

A replica is another copy of the same Pod.

If one Pod is not enough to handle traffic, Kubernetes can run two, three, or more copies. That helped me understand scaling in a very simple way.

Namespace

A namespace is a way to separate resources inside the same cluster.

I think of it like separate rooms inside the same building. Everything is still inside the cluster, but the rooms help keep things organized.

Service

A Service gives Pods a stable network identity.

This matters because Pods can come and go. If I try to talk to a Pod directly, the address may change. A Service gives me one stable name and forwards traffic to the matching Pods.

That idea became very important later when I explained ClusterIP and internal service communication.

4. Kubernetes Architecture

Once I understood the pieces, I needed to understand how they connect.

Internet
   |
   v
Ingress Controller
   |
   v
Service
   |
   v
Pod
   |
   v
Container

How the connection works

The request usually starts from the browser.

The browser sends traffic to a host name such as posts.com
The Ingress Controller receives the request
Ingress routes the request to the correct Service
The Service finds a matching Pod
The Pod forwards the traffic to the container running the app

That chain is what made Kubernetes feel real to me. It is not random. It is a path.

A larger picture

                  +----------------------+
                  |   Kubernetes Cluster |
                  +----------+-----------+
                             |
           +-----------------+-----------------+
           |                                   |
           v                                   v
     Worker Node 1                        Worker Node 2
           |                                   |
        Pod(s)                              Pod(s)
           |                                   |
      Container(s)                       Container(s)

The cluster decides where the Pods go. The nodes run them. Services give access to them. Ingress gives the outside world one entry point.

5. Deployments

Why Deployments exist

When I first saw a Deployment YAML, I wondered why I could not just create a Pod directly and stop there.

The answer is that Pods are temporary, but Deployments manage Pods for me.

A Deployment makes sure the desired number of Pod replicas exists. If a Pod dies, the Deployment creates a new one. If I want to update the app, it helps perform a rolling update.

Creating Pods the right way

I used Deployments instead of manually creating Pods because I wanted Kubernetes to manage the lifecycle.

For example, the Posts service deployment looks like this in spirit:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: posts-depl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: posts
  template:
    metadata:
      labels:
        app: posts
    spec:
      containers:
        - name: posts
          image: toqeer43553/posts

What each part means

apiVersion: apps/v1 means I am using the apps API group
kind: Deployment means this object manages Pods
metadata.name gives the Deployment a name
spec.replicas: 1 asks Kubernetes to keep one Pod running
spec.selector.matchLabels tells the Deployment which Pods belong to it
spec.template is the Pod template
spec.template.metadata.labels must match the selector
spec.template.spec.containers defines the container inside the Pod
image tells Kubernetes which image to run

Replica management

If I change replicas from 1 to 3, Kubernetes should keep three Pods alive.

That is useful when traffic grows. It is also useful when I want redundancy. If one Pod crashes, the others can still serve traffic.

Rolling updates

Rolling updates mean I can replace old Pods with new ones gradually.

That is safer than shutting down everything at once.

The benefit is simple:

fewer outages
less risk
easier rollback if something breaks

Self healing

Self healing was one of the most satisfying things to watch.

If I deleted a Pod manually, the Deployment created a replacement. That taught me the meaning of “desired state.” I say what I want. Kubernetes tries to keep it true.

Scaling

Scaling becomes easy once a Deployment owns the Pods.

Instead of managing each container by hand, I can ask for more replicas and let Kubernetes handle the rest.

Why I needed Deployments in this project

I had six app components. I did not want to manually babysit each one. Deployments gave me a controlled way to run them all and recover from failures.

6. Services

Services were one of the most important concepts I learned.

Why Services exist

Pods are temporary. Their IP addresses can change. That means one service cannot safely call another by Pod IP.

A Service gives me a stable name and a stable way to reach matching Pods.

Service types

I learned four common Service types.

ClusterIP

ClusterIP is the default service type. It exposes the service only inside the cluster.

I used ClusterIP for almost all internal services in this project.

Use it when:

only other services inside Kubernetes need access
you want internal service discovery
you do not want public access

Advantages:

secure by default
stable internal address
great for microservice-to-microservice communication

Disadvantages:

not reachable directly from outside the cluster

Real example:

posts-clusterip-srv
comments-srv
query-srv
event-bus-srv
moderation-srv

NodePort

NodePort exposes a service on a port on every node.

I used it for the Posts service in the project to help with external access in some learning setups.

Use it when:

you want a simple way to access a service from outside
you are learning or prototyping

Advantages:

simple to understand
easy to test locally

Disadvantages:

less flexible than Ingress
awkward for many services
not ideal for production

LoadBalancer

LoadBalancer asks a cloud provider to create an external load balancer.

Use it when:

you are in a cloud environment
you want a public entry point with managed infrastructure

Advantages:

simple cloud integration
external traffic routing is handled for you

Disadvantages:

depends on cloud support
not very useful in a pure local Minikube setup

ExternalName

ExternalName maps a service name to an external DNS name.

Use it when:

Kubernetes needs to refer to something outside the cluster

Advantages:

simple DNS aliasing
useful for external dependencies

Disadvantages:

does not act like normal proxy routing
not good for every use case

Real examples from my project

The internal services all use ClusterIP so that one service can talk to another by name.

For example:

http://event-bus-srv:5000/events
http://comments-srv:8001/events
http://query-srv:8002/events

That was much cleaner than hardcoding IPs.

Why I chose ClusterIP for internal communication

Because the services talk to each other inside the cluster.

I did not want the browser or the public internet calling my internal moderation service directly. ClusterIP kept those services private while still making them reachable from inside the cluster.

7. Load Balancing

Why load balancing is important

If one service gets a lot of traffic, one Pod may not be enough. Load balancing spreads requests across available Pods.

That matters because it improves:

performance
reliability
availability

How Kubernetes balances traffic

When I send traffic to a Service, Kubernetes forwards it to one of the matching Pods.

If there are multiple replicas, the Service acts like a traffic distributor.

Client
  |
  v
Service
  |
  +-------> Pod 1
  +-------> Pod 2
  +-------> Pod 3

What happens if one Pod crashes

If a Pod crashes, the Deployment replaces it.

If traffic is still flowing, the Service stops sending requests to the dead Pod and keeps using the healthy ones.

That gave me a lot more confidence in Kubernetes. It was not just running containers. It was actively keeping the system alive.

Simple example

If I had three replicas of the Query service, the Service could send requests to any healthy replica.

That means users would not care which exact Pod answered. They only care that the app works.

8. Communication Between Microservices

This is where the project became interesting.

The event flow I built

Posts Service
    |
    v
Event Bus
    |
    v
Query Service

Comments Service
    |
    v
Event Bus
    |
    +-------------------+
    |                   |
    v                   v
Moderation Service   Query Service
    |
    v
Event Bus
    |
    v
Comments Service

The data does not move in a straight line like a normal form submit. It moves as events.

Why ClusterIP is used here

Because these services should talk privately inside the cluster.

The browser does not need to know where Moderation lives. The browser only talks to the public entry point. The internal services talk to each other through internal DNS names such as query-srv or event-bus-srv.

Why not use Pod IPs

Because Pod IPs are temporary.

If a Pod restarts, Kubernetes may assign a new IP. That would break hardcoded service addresses.

Using service names solves that problem.

Service discovery

Service discovery means services can find each other by name.

I did not need to ask, “What is the IP address of the Event Bus today?” I could just use the service DNS name.

That is a huge reason Kubernetes feels manageable once the basics click.

Internal DNS

Inside the cluster, a service name becomes a DNS entry.

That is why code like this works:

http://event-bus-srv:5000/events

The service name is resolved inside Kubernetes, not by my laptop’s normal DNS setup.

Example from the project

When the Posts service creates a post, it stores it locally and then sends a PostCreated event to the Event Bus.

The Event Bus forwards that event to the Query service, which updates the read model.

That is the central idea:

write side changes data
event bus shares the change
read side updates asynchronously

That is event-driven microservices in plain language.

9. Ingress Controller

Why Ingress exists

Before Ingress, each service needed its own exposure strategy.

That becomes messy fast.

Ingress gives me one external entry point and lets me route requests based on host or path.

Problems before Ingress

Without Ingress, I would have needed:

multiple NodePorts
manual routing logic outside the cluster
more open ports
a harder mental model

That is not fun once a project grows.

One entry point

Ingress lets me say:

requests to /create-post go to the Posts service
requests to /get-posts go to the Query service
requests to /post/.../comments go to the Comments service
everything else goes to the React client

Host-based routing

The host in my setup is posts.com.

That means the app can be reached by a friendly domain instead of by a raw IP and port.

Path-based routing

Path-based routing means different URL paths go to different backend services.

Here is the idea in my project:

posts.com/create-post  -> Posts Service
posts.com/get-posts    -> Query Service
posts.com/post/...     -> Comments Service
posts.com/             -> Client Service

My ingress configuration

The real Ingress manifest uses the nginx ingress class and a regex path for the comments route.

That setup lets Kubernetes route requests to the correct service without the browser needing to know internal service names.

How a request travels

Browser
  |
  v
posts.com/get-posts
  |
  v
Ingress Controller
  |
  v
query-srv
  |
  v
Query Pod
  |
  v
Response to browser

That path was one of the clearest signs that I was learning real Kubernetes networking, not just memorizing YAML.

10. Host File Configuration

Why I edited the hosts file

My browser needs to know what posts.com means.

In local development, that name does not exist on the public internet. So I had to teach my laptop to map posts.com to the Minikube IP.

Example

127.0.0.1 posts.com

Or, more commonly in Minikube setups:

<Minikube IP> posts.com

Why browsers need this

When I type a domain into the browser, the browser asks DNS where to send the request.

If the domain is only for local development, I need a manual mapping. The hosts file gives me that mapping.

What I learned

This part felt small at first, but it was very important.

If the host file is wrong, Ingress may be working perfectly and the browser will still fail. That made me realize how many layers exist between “type URL” and “see page.”

11. Minikube

What is Minikube?

Minikube is a local Kubernetes cluster.

It lets me practice Kubernetes on my own machine without needing a cloud account or a production cluster.

Why use Minikube

I used Minikube because it made learning possible.

It gave me a safe place to test things like:

Deployments
Services
Ingress
Pods
DNS
local image handling

Local Kubernetes

Minikube is Kubernetes, but small and local.

That was perfect for me because I could test, break, and retry quickly.

Cluster creation

I learned that starting a cluster is the first step before applying manifests.

minikube start

That command gives me a local cluster to work with.

Cluster deletion

When I wanted a clean restart, I could delete the cluster and create it again.

minikube delete

That saved me many times when I had a messy local state.

Status

I used status checks to make sure the cluster was healthy before blaming my YAML.

Dashboard

The Minikube dashboard helped me see what was running.

Seeing Pods, Services, and Deployments in a UI made the system much less abstract.

Accessing services

Minikube also helped me access Services through IPs and tunnel-like behavior depending on the setup.

That became especially important when I started learning NodePort, Ingress, and the hosts file.

12. Minikube IP

Why Minikube has its own IP

Minikube runs as its own local Kubernetes environment. It is not the same thing as my laptop’s localhost.

That means the cluster may have a separate IP address.

localhost vs Minikube IP

localhost means my own machine.

Minikube IP means the address where the Minikube cluster can be reached.

These are sometimes related, but they are not the same thing.

Why NodePort works

NodePort exposes a port on the node.

If I know the Minikube IP and the NodePort, I can reach the service from outside the cluster.

Why localhost sometimes works

In some setups, Minikube is close enough to my local environment that localhost can work through port forwarding or driver-specific behavior.

That made me realize there is no single universal local Kubernetes setup. The exact access path depends on the driver and cluster configuration.

Internal networking

Inside the cluster, Services and Pods talk on the internal Kubernetes network.

Outside the cluster, I need Ingress, NodePort, port forwarding, or another access method.

Simple diagram

Laptop browser
    |
    v
localhost or Minikube IP
    |
    v
NodePort / Ingress
    |
    v
Service
    |
    v
Pod
    |
    v
Container

That separation helped me understand why some URLs worked and some did not.

13. Docker Inside Minikube

How Docker works inside Minikube

This was a painful but useful lesson.

I first built images on my laptop and expected Minikube to see them automatically.

That is because Minikube often uses its own image environment.

Why images are built inside Minikube

If Kubernetes inside Minikube tries to pull an image, it needs that image to exist in a place it can access.

When I build locally, my host Docker may have the image. But Minikube’s cluster may not see it unless I either push it to a registry or build it in Minikube’s Docker environment.

Why Kubernetes cannot see local Docker images

This was one of my most confusing early mistakes.

I had the image on my machine, so I assumed Kubernetes would just use it. But the cluster is not my laptop shell. The cluster is its own environment.

That is why image visibility matters.

docker-env

The Minikube Docker environment helps me build images directly into Minikube’s image store.

The command looks like this:

eval $(minikube docker-env)

Why this helps

After running that command, my Docker CLI talks to Minikube’s Docker daemon instead of my local one.

That means when I run docker build, the image is built where Kubernetes can see it.

Simple explanation

Before:

My laptop Docker -> local image only
Minikube -> cannot see it

After:

My laptop Docker CLI -> Minikube Docker daemon -> image available to Kubernetes

What I learned

I stopped thinking of Docker as one global thing. It is really a client talking to a Docker environment.

That small realization made Minikube much easier to work with.

14. Skaffold

Skaffold was the tool that made the whole workflow feel smoother.

What problem Skaffold solves

Without Skaffold, my workflow looked like this:

edit code
 -> docker build
 -> docker push
 -> kubectl apply
 -> check rollout
 -> repeat

That is slow and annoying during development.

How Skaffold automates everything

Skaffold watches files, rebuilds images, and redeploys Kubernetes resources for me.

That means I can focus on changing code instead of repeating the same commands all day.

`skaffold dev`

This command starts a development loop.

It watches for file changes and updates the cluster.

That was huge for productivity because I could edit, save, and see changes with much less manual work.

Image rebuild

When source files change, Skaffold can rebuild the matching image.

File watching

It watches my app files so I do not have to trigger everything manually.

Sync

For some files, Skaffold can sync changes directly into a running container instead of rebuilding every time.

That makes the feedback loop faster.

Auto deployment

When images change, Skaffold applies the Kubernetes manifests again.

That closes the loop between code and cluster.

Hot reload

In practice, it feels close to hot reload for the whole stack, even though the exact behavior depends on the app and file syncing setup.

My Skaffold YAML

The config in this project defines:

the Kubernetes manifests to deploy
the local build settings
the images for client and each service
the sync rules for JavaScript files

That was a clean way to keep development repeatable.

Why Skaffold mattered so much

It reduced friction.

When a workflow gets too manual, I spend more time operating tools than learning the system. Skaffold pushed me back toward learning the app itself.

15. Important Commands Explained

This is a compact cheat sheet for the commands I kept using.

kubectl commands

kubectl get pods: shows Pods.
kubectl get services: shows Services.
kubectl get deployments: shows Deployments.
kubectl describe pod: shows Pod events.
kubectl logs: prints logs.
kubectl delete pod: removes a Pod.
kubectl apply -f: applies YAML.
kubectl delete -f: deletes YAML resources.
kubectl rollout restart: restarts a Deployment.
kubectl exec: runs a command inside a container.
kubectl port-forward: forwards a local port.
kubectl get nodes: shows cluster nodes.
kubectl get ingress: shows Ingress resources.
kubectl describe ingress: helps debug routing.

Example:

kubectl apply -f infra/k8s

Minikube commands

minikube start: starts the cluster.
minikube stop: pauses the cluster.
minikube delete: resets the cluster.
minikube ip: prints the cluster IP.
minikube dashboard: opens the dashboard.
minikube docker-env: prints Docker settings.
eval $(minikube docker-env): points my shell at Minikube Docker.

Skaffold commands

skaffold dev: starts watch mode.
skaffold run: does one build and deploy.

Docker commands

docker build: creates an image.
docker images: shows local images.
docker ps: shows running containers.
docker logs: shows container output.
docker exec: runs a command inside a container.
docker tag: renames an image.
docker push: uploads an image.

Once I knew these commands, I could move from confusion to inspection much faster.

16. Common Problems I Faced

This project taught me that Kubernetes is full of small mistakes that look big at first.

Pods not starting

Usually this happened because of an image issue, a bad manifest, or a port mismatch.

How I debugged it:

ran kubectl get pods
used kubectl describe pod
checked kubectl logs

ImagePullBackOff

This often meant Kubernetes could not find the image.

My fix was usually one of these:

build the image in Minikube Docker
make sure the image name matches the manifest
ensure the tag is correct

CrashLoopBackOff

This usually meant the container started and then crashed repeatedly.

Common causes:

app error
missing environment assumptions
wrong port
bad startup code

Service not found

This usually meant I typed the wrong service name or expected the wrong DNS name.

I learned to compare the service name in YAML with the one used in code.

Ingress not working

This happened when the ingress controller was missing, the host file was wrong, or the route path did not match.

I learned to check all three layers:

Ingress resource exists
ingress controller is running
browser hostname resolves correctly

Hosts file issues

If the host mapping was wrong, nothing else mattered.

That was a reminder that local networking issues can look like app issues.

Minikube IP changed

When the Minikube IP changed, my hosts file mapping became stale.

The fix was to run minikube ip again and update the mapping.

Docker image not updating

This happened when I changed code but the cluster kept using an old image.

The fix was usually:

rebuild the image
use Skaffold
verify Minikube Docker context

Skaffold not rebuilding

This usually meant the file sync rules were too narrow or the config did not match the file location.

I learned to check the sync paths carefully.

My debugging habit

The biggest lesson was not to guess.

I tried to inspect one layer at a time:

Is the Pod running?
Is the Service correct?
Is the Ingress correct?
Does the host file point to the right place?
Does the app log show errors?

That simple checklist saved me a lot of time.

17. Pros and Cons

Pros of Kubernetes

It keeps services running and replaces failed Pods
It supports service discovery
It works from local dev to production

Cons of Kubernetes

It has a steep learning curve
YAML can be tedious
Local setups can differ from production

Pros of Skaffold

It speeds up development
It reduces manual commands
It watches files and rebuilds automatically

Cons of Skaffold

The config can be confusing at first
Sync rules need care
It adds another tool to learn

Pros of Ingress

One entry point for many services
Clean routing by host and path

Cons of Ingress

Needs an ingress controller
Debugging can be tricky
Host and path rules must match exactly

Pros of Microservices

Clear service boundaries
Easier to scale parts separately
Better for event-driven systems

Cons of Microservices

More moving parts
More networking problems
More concepts to learn

18. Lessons I Learned

This is the part that matters most to me.

At first, Kubernetes felt like too many objects with strange names. What helped was not reading more definitions. What helped was building something real.

Once I saw the posts app create an event, then watched the query model update, then saw comments go through moderation, I started to understand why these parts exist.

The biggest mental shift was this:

Docker runs containers
Kubernetes manages them
Services make them reachable
Ingress makes them public in a controlled way
Skaffold makes development faster

I also learned that microservices are not just about splitting code. They are about splitting responsibility.

Another thing that became easier was reading YAML. At first it felt dry and repetitive, but later I saw it as a contract between me and Kubernetes.

19. Best Practices

Here are the beginner-friendly practices that helped me most.

Keep services small

Each service should have one clear job.

Use service names, not Pod IPs

Pod IPs can change. Service names are stable.

Separate write and read paths when it makes sense

The CQRS idea helped my app stay organized.

Use ClusterIP for internal services

Do not expose everything publicly.

Use Ingress for the public entry point

This keeps routing clean.

Check logs early

Logs often tell the truth faster than guesswork.

Learn the manifest one piece at a time

Do not try to memorize everything at once.

Keep your local environment clean

If Minikube becomes messy, reset it instead of fighting stale state forever.

Use Skaffold during development

It saves time and keeps the feedback loop short.

Understand the path of a request

Always ask:

Where did the request start?
Which service received it?
Which Service forwarded it?
Which Pod handled it?

That question alone will save you a lot of confusion.

20. Conclusion

I started this project wanting to learn Kubernetes. I ended up learning much more.

I learned how containers package software.

I learned how Kubernetes runs those containers across a cluster.

I learned why Deployments matter, why Services matter, why Ingress matters, and why local tools like Minikube and Skaffold make practice possible.

Most of all, I learned that Kubernetes becomes easier when it is tied to a real project.

On their own, the concepts can feel abstract. But when I used them to run a real microservices app, they started to make sense.

If you are a beginner, do not try to understand everything in one day. Build something small, break it, and repeat.

That is how the pieces connect.

21. What’s Next?

This project gave me a good base, but I know there is more to learn.

The next topics I want to study are:

ConfigMaps
Secrets
Persistent Volumes
Helm
Horizontal Pod Autoscaler
Monitoring
Logging
CI/CD
Production Kubernetes
Cloud Kubernetes such as AWS EKS, GKE, and AKS

I feel more ready for those topics now because I understand the basics from a real app, not just from theory.

SEO Details

SEO title

Building a Microservices App with Kubernetes: My Beginner-Friendly Journey Through Docker, Minikube, Ingress, and Skaffold

Meta description

A beginner-friendly story of building a microservices application with Kubernetes, covering Docker, Pods, Deployments, Services, Ingress, Minikube, Docker inside Minikube, and Skaffold in simple English.

Suggested URL slug

building-microservices-app-with-kubernetes-docker-minikube-ingress-skaffold

Suggested cover image idea

A clean illustration of a laptop connected to a Kubernetes cluster, with small boxes labeled Docker, Pods, Services, Ingress, and Skaffold, styled like a simple system map.

Kubernetes Internals Explained: From Linux Primitives to Ingress

M TOQEER ZIA — Tue, 07 Jul 2026 08:15:51 +0000

If you've ever felt like Kubernetes is a pile of magic YAML files, this article is for you. We're going to build understanding from the ground up — starting with plain Linux, moving to containers, and then climbing all the way to Pods, Nodes, Deployments, Services, and Ingress. By the end, you won't just know the names of these pieces — you'll know why they exist.

Why Start With Linux?

Kubernetes didn't invent containers. It's an orchestrator sitting on top of Linux features that already existed. If you understand the Linux primitives first, every Kubernetes concept afterward becomes "oh, that's just automating this thing I already understand" instead of new magic to memorize.

Two Linux kernel features make containers possible:

1. Namespaces — Isolation

A namespace makes a process think it's the only thing running on the machine. Linux has several kinds:

PID namespace – the process sees itself as PID 1, even though the host sees it as PID 48213.
Network namespace – gives the process its own network interfaces, IP address, and routing table, separate from the host.
Mount namespace – gives the process its own view of the filesystem (its own /).
UTS namespace – lets the process have its own hostname.
IPC namespace – isolates inter-process communication (shared memory, semaphores).
User namespace – lets a process be "root" inside the container but be an unprivileged user on the host.

A container is just a regular Linux process that has been put inside a bundle of these namespaces so it believes it's alone on the machine.

2. Control Groups (cgroups) — Resource Limits

Namespaces isolate what a process can see. Cgroups control how much it can use — CPU shares, memory limits, disk I/O, network bandwidth. This is how you can say "this container gets max 512Mi of RAM" and have the kernel actually enforce it.

Takeaway: Container = Linux process + namespaces (isolation) + cgroups (resource limits) + a filesystem bundle (the image). Nothing more mystical than that.

Container Images

An image is a read-only template for creating containers. It's built in layers — each instruction in a Dockerfile (FROM, RUN, COPY, etc.) creates a new layer stacked on top of the previous one using a union filesystem (like OverlayFS).

Layer 4: COPY . /app          <- your app code
Layer 3: RUN npm install      <- dependencies
Layer 2: RUN apt-get update   <- OS packages
Layer 1: FROM node:20-alpine  <- base OS

Why layers matter:

Layers are cached — if only your app code changes, Docker/containerd reuses the lower layers instead of rebuilding everything.
Layers are shared across images — ten images built FROM node:20-alpine share that base layer on disk instead of duplicating it.

Images live in registries (Docker Hub, GitHub Container Registry, AWS ECR, etc.). When Kubernetes needs to run a container, it pulls the image from a registry to the Node first.

Container Runtime

This is the piece that actually takes an image and turns it into a running, namespaced, cgroup-limited process. Kubernetes doesn't talk to Docker directly anymore — it talks through a standard interface called CRI (Container Runtime Interface).

The common runtime stack today:

kubelet
   │  (talks CRI protocol)
   ▼
containerd  (high-level runtime: manages images, pulls, storage)
   │
   ▼
runc  (low-level runtime: actually calls Linux syscalls to create
       namespaces/cgroups and start the process)

runc is doing the literal work we described in the Linux section: calling clone() with namespace flags, setting up cgroups, and exec()-ing your process. Everything above it is orchestration.

Pods — The Smallest Deployable Unit

Here's a question that trips people up: why doesn't Kubernetes just schedule containers directly? Why the extra layer of a "Pod"?

Answer: because real applications are often more than one process that need to share resources tightly — think of a main app container plus a logging sidecar that both need to read the same files and talk over localhost.

A Pod is a group of one or more containers that share:

The same network namespace — all containers in a Pod share one IP address and can reach each other over localhost. Container A on port 8080 and container B on port 9090 in the same Pod just call localhost:8080 / localhost:9090.
Optionally, storage volumes — a Pod can define a volume that multiple containers mount.
The same lifecycle — they're scheduled together, and they die together.

How is the shared network namespace actually implemented? Kubernetes silently creates a hidden "pause" container for every Pod. This container does nothing except hold open the network namespace. Every other container in the Pod joins that namespace instead of creating its own. That's the real implementation detail behind "Pods share networking."

Pod (shares one network namespace + IP: 10.244.1.7)
├── pause container      (holds the namespace, does nothing)
├── app container        (localhost:8080)
└── sidecar container    (localhost:9090, e.g. log shipper)

This is also why the Pod, not the container, is the atomic unit Kubernetes schedules, scales, and replaces.

Nodes — Where Pods Actually Run

A Node is a worker machine (VM or bare metal) running the components needed to host Pods. Every Node runs:

kubelet – the agent that talks to the control plane, receives instructions ("run this Pod"), and tells the container runtime to do it via CRI. It also continuously reports Pod health back up.
container runtime – containerd + runc, as covered above.
kube-proxy – maintains network rules on the Node (via iptables or IPVS) so that traffic sent to a Service IP gets routed to the correct backend Pod, even though the Pod could be on a completely different Node.

Think of the kubelet as a very obedient local supervisor: the control plane never SSHs into a Node to start something — it just updates a desired state, and kubelet on each Node notices and reconciles reality to match it.

The Control Plane — The Brain

This is the part that decides what should run where, and it's separate from the Nodes that actually run your workloads.

kube-apiserver – the front door. Every single interaction with the cluster — kubectl, the scheduler, controllers, kubelets — goes through the API server. It validates requests and is the only component that talks directly to storage.
etcd – a distributed key-value store that holds the entire cluster state (every object, every desired configuration). If etcd is healthy, the cluster's "memory" is safe.
kube-scheduler – watches for Pods with no Node assigned yet, and decides which Node they should run on, based on resource availability, affinity rules, taints/tolerations, etc.
kube-controller-manager – runs the reconciliation loops. Example: the ReplicaSet controller constantly checks "do I have 3 Pods running as declared? If not, create more."

The core pattern to internalize: you declare desired state (via YAML → API server → etcd). Controllers continuously watch for drift between desired state and actual state, and correct it. Kubernetes never "runs a command once" — it enforces a state forever.

Deployments

A Deployment is a controller that manages ReplicaSets, which in turn manage Pods.

Deployment
   └── ReplicaSet (v1)
           ├── Pod
           ├── Pod
           └── Pod

Why the extra ReplicaSet layer instead of Deployment managing Pods directly? Rolling updates. When you update the image in a Deployment, it doesn't touch existing Pods — it creates a new ReplicaSet with the new Pod template, scales it up gradually, and scales the old ReplicaSet down. This is how you get zero-downtime deploys and easy rollbacks (rolling back = pointing back at the old ReplicaSet).

Services — Solving the Pod IP Problem

Pods are mortal. They get killed and recreated constantly (crashes, scaling, node failures) — and every time, they get a new IP address. So how does anything reliably talk to "the backend"?

This is what a Service solves. A Service gets a stable virtual IP and DNS name, and uses labels and selectors to find which Pods should receive traffic.

selector:
  app: my-backend

Any Pod labeled app: my-backend automatically becomes a backend for this Service — no matter how many times those Pods get replaced. kube-proxy on every Node maintains the routing rules that make "traffic to the Service IP" actually land on one of the healthy matching Pods.

Three Service types matter most:

ClusterIP (default) – a virtual IP reachable only from inside the cluster. Good for internal service-to-service communication (e.g., frontend Pod → backend Service).
NodePort – opens a specific port (30000–32767 range) on every Node's own IP, forwarding it into the Service. Lets external traffic in via <NodeIP>:<NodePort>, but it's clunky for production (you're exposing raw node IPs and non-standard ports).
LoadBalancer – asks the cloud provider (AWS/GCP/Azure) to provision an actual external load balancer that points at the Service. This is the standard way to expose something to the internet on a cloud cluster.

Port Forwarding vs Services

kubectl port-forward pod/my-pod 8080:80 is a developer debugging tool, not a production traffic mechanism. It opens a temporary tunnel from your local machine straight to a single Pod, bypassing Services and kube-proxy entirely. The moment you kill that terminal, the connection is gone. It's for "let me poke this one Pod directly while debugging" — never how real traffic reaches your app.

Ingress — HTTP Routing at the Edge

If Services (specifically LoadBalancer) already expose things externally, why do we need Ingress too?

Problem: if you have 10 HTTP microservices, giving each one its own cloud LoadBalancer means 10 expensive external load balancers, and no shared logic for things like TLS termination, host-based routing (api.example.com vs app.example.com), or path-based routing (/users vs /orders).

Ingress is a set of routing rules — "if the request comes in for api.example.com/users, send it to the users-service ClusterIP Service." But Ingress rules alone do nothing — you need an Ingress Controller (NGINX Ingress Controller, Traefik, etc.) running as Pods in your cluster that actually reads those rules and does the routing. The Ingress Controller is usually the one component that does get a single LoadBalancer Service, and everything else flows through it.

Internet
   │
   ▼
LoadBalancer Service (1 external IP)
   │
   ▼
Ingress Controller Pods (reads Ingress rules)
   │
   ├── /users  → users-service (ClusterIP)  → users Pods
   ├── /orders → orders-service (ClusterIP) → orders Pods
   └── api.example.com → api-service (ClusterIP) → api Pods

Putting the Whole Request Path Together

Here's the full journey of one HTTP request hitting your cluster, tying every piece above together:

Request hits the cloud LoadBalancer, which forwards to the Ingress Controller Pods.
Ingress Controller reads the Ingress rules and decides which internal Service should handle this path/host.
It forwards the request to that Service's ClusterIP.
kube-proxy's iptables/IPVS rules on the Node intercept traffic to that ClusterIP and redirect it to one of the matching Pods, chosen via the Service's label selector.
Traffic lands inside the Pod's shared network namespace, reaching the actual container process — a Linux process running inside namespaces and cgroups, started by runc, pulled from an image by containerd, on instructions from the kubelet, which got that instruction from the API server, which persisted it from a Deployment you applied, which is stored durably in etcd.

That's the entire stack, front to back, in one sentence.

Closing Thoughts

Every Kubernetes concept exists to solve a concrete problem:

Problem	Kubernetes Answer
Isolate processes on Linux	namespaces + cgroups
Package an app reproducibly	container images
Actually run a container	container runtime (containerd + runc)
Group tightly-coupled containers	Pods
Run Pods somewhere	Nodes (kubelet, kube-proxy, runtime)
Decide cluster-wide state	Control plane (API server, etcd, scheduler, controllers)
Zero-downtime updates & scaling	Deployments + ReplicaSets
Stable networking to mortal Pods	Services + labels/selectors
Smart HTTP routing at the edge	Ingress + Ingress Controller

Once you see it this way, Kubernetes stops being "a huge list of YAML kinds to memorize" and becomes "a small number of real problems, each solved by one clean abstraction."

If this helped clarify things, I'll be following up with a deeper dive into Services and networking internals (ClusterIP vs NodePort vs LoadBalancer under the hood) — let me know in the comments what you'd like covered next.

Database Sharding Explained: How to Scale Your Database Like a Pro

M TOQEER ZIA — Tue, 07 Jul 2026 07:27:48 +0000

If you've ever worked on an application that grew from a handful of users to thousands—or millions—you've probably hit a point where your database starts groaning under the weight. Queries get slower. CPU spikes. Storage fills up. And no amount of indexing seems to fix it.

This is usually where database sharding enters the conversation.

In this article, we'll break down what sharding actually is, why it matters, the most common strategies used in production systems, and how to decide if (and when) your project actually needs it.

What Is Database Sharding?

Database sharding is a technique for splitting one large database into multiple smaller, independent databases called shards. Each shard holds a portion of the total data, and together, all the shards make up the complete dataset.

Think of it like a library system:

A single library holding 10 million books becomes slow to search and hard to manage.
So instead, the books get split across branches:
- Branch A → Books A–F
- Branch B → Books G–L
- Branch C → Books M–R
- Branch D → Books S–Z

Each branch only manages a fraction of the total collection, which makes browsing, searching, and organizing dramatically faster. Sharding applies this same idea to databases—except instead of book titles, we're splitting rows of data across servers.

Why Do We Need Sharding?

As an application scales, a single database server eventually runs into hard limits:

Slow query performance
High CPU and memory usage
Storage limitations
Too many concurrent users/connections

Sharding solves these problems by spreading data—and the load that comes with it—across multiple servers instead of forcing one machine to do all the work.

A Practical Example

Imagine you're running a social media platform with 100 million users.

Without sharding, everything lives on one server:

DB Server 1
------------
User 1
User 2
User 3
...
User 100,000,000

Every read, write, and query hits this same machine. Eventually, it becomes a bottleneck no matter how much you optimize.

With sharding, the data is distributed:

Shard 1 → Users 1 – 25M
Shard 2 → Users 25M – 50M
Shard 3 → Users 50M – 75M
Shard 4 → Users 75M – 100M

Now four servers share the load instead of one. Each shard only needs to handle a quarter of the traffic and storage.

Common Sharding Strategies

There's no one-size-fits-all approach to sharding. Here are the three strategies you'll encounter most often.

1. Range-Based Sharding

Data is split according to a defined range of values (like ID ranges).

Shard 1 → User IDs 1–1000
Shard 2 → User IDs 1001–2000
Shard 3 → User IDs 2001–3000

Query example:

SELECT * FROM users WHERE id = 1500;

The application already knows this record lives in Shard 2, so it routes the query there directly.

Pros:

Simple to understand and implement

Cons:

Uneven growth can cause "hot" shards—one shard may end up much larger or busier than the others

2. Hash-Based Sharding

A hash function decides which shard a piece of data belongs to.

shard = userId % 3

Example distribution:

1 % 3 = 1 → Shard 1
2 % 3 = 2 → Shard 2
3 % 3 = 0 → Shard 3
4 % 3 = 1 → Shard 1

Pros:

Distributes data evenly across shards, avoiding hotspots

Cons:

Adding or removing shards later is difficult, since it changes the hash math and can require re-shuffling large amounts of data (this is where consistent hashing usually comes in to soften the blow)

3. Geographic Sharding

Data is partitioned based on the user's region or location.

Shard Pakistan → Pakistani users
Shard USA      → American users
Shard Europe   → European users

This approach is common in global applications, since it also helps reduce latency (users are served from a shard closer to them) and can assist with data residency/compliance requirements.

How Does the Application Know Which Shard to Use?

This is handled through a shard key—a value used to determine where a specific piece of data lives.

const shard = userId % 4;

Once the shard is calculated, the application routes the query to the correct database:

if (shard === 0) {
  query(DB1);
} else if (shard === 1) {
  query(DB2);
}
// ...and so on

In real-world systems, this routing logic is often abstracted into a middleware layer or handled by the database driver/proxy itself, so individual services don't need to reimplement it everywhere.

Real-Life Example: E-Commerce Platform

Let's say you're building an e-commerce platform and want to shard your users table.

User ID
--------
1
2
3
4
...

Sharding strategy:

Shard 1 → User IDs ending in 0–4
Shard 2 → User IDs ending in 5–9

When User 1234 logs in:

1234 % 2 = 0

The request is routed to Shard 1.

Sharding vs. Replication: What's the Difference?

People often confuse sharding with replication, but they solve different problems.

Sharding	Replication
Splits data across servers	Copies the same data to multiple servers
Used for scaling writes	Used for scaling reads
Each shard holds different data	Every replica holds the same data
Increases storage capacity	Improves availability & redundancy

Sharding example:

DB1 → Users 1–1000
DB2 → Users 1001–2000

Replication example:

      Primary DB
          |
   +------+------+
   |             |
Replica 1     Replica 2

In many production systems, these two techniques are actually combined—each shard has its own set of replicas for both scalability and availability.

Interview-Ready Summary

If you're prepping for a system design interview, here's a concise way to explain it:

Database sharding is a horizontal scaling technique where data is partitioned across multiple database servers called shards. Each shard contains a subset of the data and handles part of the application's workload. Sharding improves scalability, performance, and storage capacity for large-scale systems. Common strategies include range-based, hash-based, and geographic sharding.

When Should You Actually Use Sharding?

Sharding is powerful, but it's not something you should reach for on day one. It adds real architectural complexity—cross-shard queries, distributed transactions, and rebalancing all become harder problems once you shard.

Consider sharding when:

Your database size is becoming too large for a single server
Write traffic is very high
Queries are slowing down despite indexing and optimization
Vertical scaling (more CPU/RAM) is no longer enough

Before reaching for sharding, try:

Query optimization
Better indexing
Caching layers (Redis, Memcached, etc.)
Read replicas

Sharding is typically introduced once an application reaches a large scale—think millions of users or datasets that no longer fit comfortably on one machine. Until then, simpler scaling techniques usually get you further with far less complexity.

Final Thoughts

Sharding is one of those concepts that sounds intimidating until you break it down—at its core, it's just "divide the data so no single server has to do all the work." The real challenge isn't understanding the concept, it's choosing the right shard key and strategy for your specific access patterns, and being ready for the operational complexity that comes with a distributed database layer.

If you're designing for scale, start simple, measure where your actual bottlenecks are, and only shard when you have clear evidence that you need to.

If you found this helpful, feel free to follow for more system design breakdowns!

JavaScript Under the Hood: Scope, Closures, Prototypes &

M TOQEER ZIA — Mon, 29 Jun 2026 18:02:21 +0000

A deep dive into the concepts that separate junior devs from engineers who actually understand the runtime.

Part 1: Scope and Closures

var vs let vs const — It's Not Just Syntax

Most devs learn early that let and const are "the modern way" and var is old. But the real difference runs deeper than style.

Hoisting

JavaScript hoists declarations to the top of their scope before execution. But how that hoisting behaves depends on which keyword you use.

console.log(name); // undefined (not an error!)
var name = "Toqeer";

console.log(age); // ReferenceError: Cannot access 'age' before initialization
let age = 27;

var is hoisted and initialized to undefined. So the engine sees this:

var name; // hoisted to top
console.log(name); // undefined
name = "Toqeer";

let and const are also hoisted — but they are not initialized. They exist in what's called the Temporal Dead Zone (TDZ) from the start of the block until the declaration is reached.

Temporal Dead Zone (TDZ)

{
  // TDZ for `score` starts here
  console.log(score); // ReferenceError
  let score = 100;    // TDZ ends here
}

The variable exists in the scope, the engine knows about it — but you cannot touch it yet. This is intentional. It prevents you from accidentally using a variable before it's ready.

const adds one more layer: once assigned, the binding cannot be reassigned. For objects and arrays, the reference is locked — but the contents can still mutate.

const config = { debug: false };
config.debug = true; // fine
config = {};         // TypeError: Assignment to constant variable

Scope: Function vs Block

var is function-scoped. It leaks out of blocks like if, for, while.

function checkStatus() {
  if (true) {
    var status = "active";
  }
  console.log(status); // "active" — leaked out of the if block
}

let and const are block-scoped. They stay inside {}.

function checkStatus() {
  if (true) {
    let status = "active";
  }
  console.log(status); // ReferenceError
}

This is why the classic loop bug exists:

for (var i = 0; i < 3; i++) {
  setTimeout(() => console.log(i), 100);
}
// Prints: 3, 3, 3

for (let i = 0; i < 3; i++) {
  setTimeout(() => console.log(i), 100);
}
// Prints: 0, 1, 2

With var, there's only one i shared across all iterations. With let, each iteration gets its own block-scoped i.

Closures: One of JavaScript's Superpowers

A closure is a function that remembers the variables from its outer scope — even after that outer scope has finished executing.

function makeCounter() {
  let count = 0;

  return function () {
    count++;
    return count;
  };
}

const counter = makeCounter();
console.log(counter()); // 1
console.log(counter()); // 2
console.log(counter()); // 3

makeCounter has long since returned. Its execution context is gone. But count lives on — because the inner function closed over it.

How Closures Form

A closure forms every time a function is created inside another function and references variables from the outer scope. The inner function holds a reference to the outer environment, not a copy.

Practical Uses

1. Data privacy / encapsulation:

function createBankAccount(initialBalance) {
  let balance = initialBalance;

  return {
    deposit(amount) { balance += amount; },
    withdraw(amount) { balance -= amount; },
    getBalance() { return balance; }
  };
}

const account = createBankAccount(1000);
account.deposit(500);
console.log(account.getBalance()); // 1500
// `balance` is not accessible from outside

2. Function factories:

function multiplier(factor) {
  return (number) => number * factor;
}

const double = multiplier(2);
const triple = multiplier(3);

console.log(double(5)); // 10
console.log(triple(5)); // 15

3. Memoization:

function memoize(fn) {
  const cache = {};
  return function (n) {
    if (cache[n] !== undefined) return cache[n];
    cache[n] = fn(n);
    return cache[n];
  };
}

const factorial = memoize(function f(n) {
  return n <= 1 ? 1 : n * f(n - 1);
});

IIFE — Immediately Invoked Function Expression

An IIFE is a function that runs the moment it's defined.

(function () {
  const secret = "hidden";
  console.log("Runs immediately");
})();

// `secret` is not accessible out here

The outer () wraps the function expression (makes it an expression, not a declaration), and the trailing () invokes it immediately.

Why use it?

Before ES modules and block scoping with let/const, IIFEs were the primary way to:

Create a private scope and avoid polluting the global namespace
Initialize libraries and plugins
Wrap module code

const app = (function () {
  let _privateState = 0;

  return {
    increment() { _privateState++; },
    getState() { return _privateState; }
  };
})();

app.increment();
console.log(app.getState()); // 1

Today, ES modules handle this naturally. But IIFEs still appear in bundled output and are worth understanding.

Part 2: Prototypes and Inheritance

The Prototype Chain

JavaScript is a prototype-based language. Every object has an internal link — [[Prototype]] — that points to another object. When you access a property, JavaScript first looks on the object itself, then walks up the chain.

const animal = {
  breathe() {
    return "breathing...";
  }
};

const dog = Object.create(animal);
dog.bark = function () { return "woof!"; };

console.log(dog.bark());    // "woof!" — found on dog
console.log(dog.breathe()); // "breathing..." — found on animal (via prototype)
console.log(dog.toString()); // found on Object.prototype

The chain: dog → animal → Object.prototype → null

When the end (null) is reached without finding the property, undefined is returned (or a TypeError for method calls).

Object.create and Object.getPrototypeOf

Object.create(proto) creates a new object whose prototype is proto:

const vehicleProto = {
  describe() {
    return `I am a ${this.type} with ${this.wheels} wheels`;
  }
};

const car = Object.create(vehicleProto);
car.type = "car";
car.wheels = 4;

console.log(car.describe()); // "I am a car with 4 wheels"

Object.getPrototypeOf(obj) lets you inspect the chain:

console.log(Object.getPrototypeOf(car) === vehicleProto); // true

Constructor Functions

Before ES6 classes, constructor functions were the way to create objects with shared behavior.

function Person(name, age) {
  this.name = name;
  this.age = age;
}

Person.prototype.greet = function () {
  return `Hi, I'm ${this.name}`;
};

const toqeer = new Person("Toqeer", 27);
console.log(toqeer.greet()); // "Hi, I'm Toqeer"

greet lives on Person.prototype, not on each instance — so all instances share it without duplication.

The `new` Keyword — What Actually Happens

When you call a function with new, four things happen under the hood:

1. A new empty object is created: {}
2. Its [[Prototype]] is linked to Constructor.prototype
3. `this` inside the function points to that new object
4. The function returns `this` (the new object) unless it explicitly returns another object

You can see this manually:

function myNew(Constructor, ...args) {
  const obj = Object.create(Constructor.prototype); // step 1 & 2
  const result = Constructor.apply(obj, args);      // step 3
  return result instanceof Object ? result : obj;   // step 4
}

const person = myNew(Person, "Toqeer", 27);
console.log(person.greet()); // "Hi, I'm Toqeer"

This is exactly what new does — now it's not magic.

ES6 Classes — Syntactic Sugar

Classes in JavaScript are not a new object model. They're a cleaner syntax over the same prototype mechanism.

class Animal {
  constructor(name) {
    this.name = name;
  }

  speak() {
    return `${this.name} makes a sound.`;
  }
}

class Dog extends Animal {
  speak() {
    return `${this.name} barks.`;
  }
}

const d = new Dog("Rex");
console.log(d.speak()); // "Rex barks."
console.log(d instanceof Animal); // true

Under the hood:

Animal is a constructor function
speak is added to Animal.prototype
extends sets up the prototype chain: Dog.prototype → Animal.prototype
super() calls the parent constructor

The output is identical to doing it manually with constructor functions and Object.create. Classes just make the intent clearer.

Key Differences from Constructor Functions

Feature	Constructor Function	Class
Hoisted	Yes (as `undefined`)	No (TDZ applies)
Strict mode	Optional	Always strict
Callable without `new`	Yes (bad idea)	TypeError
Inherited methods	Manual prototype chain	`extends` keyword
`super` call	Manual with `.call`	Built-in

Putting It All Together

Here's everything combined — a real-world-ish example:

class EventEmitter {
  #listeners = {};

  on(event, handler) {
    if (!this.#listeners[event]) this.#listeners[event] = [];
    this.#listeners[event].push(handler);
    return this; // for chaining
  }

  emit(event, ...args) {
    (this.#listeners[event] || []).forEach(fn => fn(...args));
  }
}

class Store extends EventEmitter {
  #state;

  constructor(initialState) {
    super();
    this.#state = initialState;
  }

  setState(newState) {
    this.#state = { ...this.#state, ...newState };
    this.emit("change", this.#state);
  }

  getState() {
    return this.#state;
  }
}

const store = new Store({ count: 0 });

store.on("change", (state) => {
  console.log("State changed:", state);
});

store.setState({ count: 1 }); // "State changed: { count: 1 }"

This uses:

Private class fields (#listeners, #state) for encapsulation — closure-like privacy at the class level
Prototype inheritance via extends
super() to call the parent constructor
Method chaining via return this

Quick Reference

Scoping Rules

var → function-scoped, hoisted and initialized to undefined
let / const → block-scoped, hoisted but in TDZ until declaration
const → block-scoped + immutable binding (contents can mutate)

Closures

Form when an inner function references outer variables
The inner function holds a reference to the outer environment, not a copy
Useful for: encapsulation, factories, memoization, event handlers

IIFE

Runs immediately, creates isolated scope
(function() { ... })()
Less common today due to ES modules, but still found in bundled code

Prototype Chain

Every object has a [[Prototype]] link
Property lookup walks the chain until null
Object.create(proto) sets the prototype explicitly

`new` Keyword Steps

Create empty object
Link prototype to Constructor.prototype
Execute constructor with this = new object
Return the new object (unless constructor returns an object)

Classes

Syntactic sugar over prototypes
extends wires prototype chain
super() calls parent constructor
Always strict, not hoistable

Understanding these fundamentals will make you a better debugger, a better code reviewer, and a more confident engineer — whether you're writing Node.js APIs, React apps, or contributing to open source.

Drop a comment if you want me to go deeper on any of these. Happy to cover the event loop, async/await internals, or WeakMaps and memory management next.

Fine-Tuning Large Language Models: A Practical Guide

M TOQEER ZIA — Tue, 16 Jun 2026 08:59:56 +0000

What Is Fine-Tuning?

A pre-trained model learns from billions of tokens of general text. It develops broad language understanding. Fine-tuning takes that base model and trains it further on a smaller, task-specific dataset.

The result: a model that performs well on your specific use case, whether that is medical diagnosis, legal summarization, customer support, or code generation.

Fine-tuning is not training from scratch. You preserve general knowledge and adapt behavior.

How a Neural Network Learns

Before fine-tuning makes sense, you need to understand what happens inside a neural network during training.

A neural network is a system of layers. Each layer contains nodes (neurons). Each connection between nodes has a weight. These weights determine what the network outputs given any input.

At the start, weights are random. Training adjusts them so the network produces correct outputs.

Here is the full training loop:

Training Data
      ↓
Forward Pass
      ↓
Prediction
      ↓
Loss Calculation
      ↓
Backpropagation
      ↓
Gradient Calculation
      ↓
Weight Update
      ↓
Repeat
      ↓
Trained Model

Phase 1: Forward Pass

The model receives an input (a sentence, an image, a token). Data flows forward through every layer. Each neuron applies a mathematical function and passes its result to the next layer.

At the end, the model produces a prediction. For a language model, this is a probability distribution over the next token.

No learning happens here. This phase only produces output.

Phase 2: Loss Calculation

The model's prediction is compared to the correct answer. A loss function measures how wrong the prediction is.

Common loss functions:

Cross-entropy loss: used for classification and language modeling
Mean squared error: used for regression tasks

A high loss means the model is far off. A loss near zero means the prediction was close to correct.

The loss is a single number. It summarizes the error for one batch of data.

Phase 3: Backpropagation

This is where learning begins.

Backpropagation works backward through the network. It calculates how much each weight contributed to the total loss. The math tool used here is the chain rule from calculus.

Think of it as blame assignment. Each weight receives a gradient: a number that says "changing this weight by X changes the loss by Y."

A large gradient means that weight had a big effect on the error. A small gradient means it had little effect.

Phase 4: Gradient Calculation

After backpropagation, every weight in the network has a gradient.

The gradient is a vector. It points in the direction of steepest increase in loss. To reduce the loss, you move in the opposite direction.

Phase 5: Weight Update (Optimization)

An optimizer uses the gradients to update every weight.

The most basic optimizer is Stochastic Gradient Descent (SGD):

new_weight = old_weight - (learning_rate × gradient)

The learning rate controls the step size. Too large: the model overshoots. Too small: training takes too long.

Modern optimizers like Adam adjust the learning rate automatically for each weight. Adam tracks the history of gradients and adapts accordingly. It converges faster than basic SGD in most cases.

The Full Loop

One pass through the data is one epoch. Training repeats this loop thousands of times across many epochs. With each pass, the weights improve. The loss decreases. The model gets better at the task.

Types of Fine-Tuning

There are three main approaches. They differ in what gets updated during training.

1. Full Fine-Tuning

All weights in the model are updated.

You take the pre-trained model and run the training loop on your dataset. Every parameter is fair game. The optimizer adjusts all of them.

Advantages:

Maximum flexibility
Best performance ceiling for large datasets

Disadvantages:

Requires significant GPU memory (storing the model, gradients, and optimizer states for billions of parameters)
Expensive and slow
Risk of catastrophic forgetting: the model overwrites general knowledge with task-specific patterns
A 7B parameter model at full precision requires roughly 28 GB of GPU memory for weights alone, plus optimizer states

Full fine-tuning makes sense when you have large datasets and substantial compute.

2. PEFT (Parameter-Efficient Fine-Tuning)

PEFT is a family of methods. The core idea: freeze most of the model and only train a small number of parameters.

You add new, trainable components to the frozen model. Only those new components are updated. The original weights stay fixed.

This reduces:

GPU memory required (no gradients for frozen weights)
Training time
Risk of catastrophic forgetting

PEFT methods include LoRA, prefix tuning, prompt tuning, and adapter layers. LoRA is the most widely used.

3. LoRA (Low-Rank Adaptation)

LoRA is the dominant PEFT method in 2024 and 2025.

The insight behind LoRA: weight updates during fine-tuning tend to have low intrinsic rank. You do not need to update a full weight matrix. You approximate the update with two much smaller matrices.

How it works:

For a weight matrix W of size (m × n), instead of updating W directly, LoRA adds two matrices:

Matrix A: size (m × r)
Matrix B: size (r × n)

Where r is the rank, typically 4, 8, 16, or 64. You only train A and B. The product AB approximates the full weight update.

During inference, the update is merged back: W_new = W + AB

Parameter count comparison:

A weight matrix of size 4096 × 4096 has 16,777,216 parameters.
LoRA with rank 8 adds two matrices: (4096 × 8) + (8 × 4096) = 65,536 parameters.
That is a 256x reduction in trainable parameters for that layer.

Advantages:

Trains on consumer GPUs (a 7B model with LoRA fits on 8-16 GB VRAM)
Multiple LoRA adapters for different tasks, swapped at inference time
The base model stays frozen and reusable

Disadvantages:

Slightly below full fine-tuning on performance in some benchmarks
Requires choosing rank (r) and which layers to target

4. QLoRA (Quantized LoRA)

QLoRA combines quantization with LoRA. It pushes LoRA further in terms of memory efficiency.

What quantization does:

Standard model weights use 16-bit or 32-bit floating point numbers. Quantization converts them to lower precision: 8-bit, 4-bit, or even 3-bit integers.

A 7B model in 16-bit uses roughly 14 GB of memory. The same model in 4-bit uses roughly 3.5 GB.

QLoRA workflow:

Load the base model in 4-bit precision (NF4 format, designed for neural network weights)
Freeze all quantized weights
Add LoRA adapters in 16-bit precision
Train only the LoRA adapters

The gradients flow through the quantized model, through the LoRA adapters. Only the adapters are updated.

Double quantization: QLoRA applies a second round of quantization to the quantization constants themselves, saving an additional 0.5 GB on a 7B model.

Advantages:

Fine-tune a 70B model on a single 48 GB GPU
Fine-tune a 7B model on a single 8 GB consumer GPU
Performance within 1-2% of full fine-tuning on most benchmarks

Disadvantages:

Slower than LoRA (dequantization overhead during forward/backward pass)
Quantization introduces approximation errors

Comparison Table

Method	Trainable Params	VRAM (7B model)	Performance	Use Case
Full Fine-Tuning	100%	80+ GB	Highest	Large datasets, max compute
PEFT (general)	0.1 - 1%	Varies	Good	Resource-constrained
LoRA	0.1 - 1%	16-24 GB	Near full	Most fine-tuning tasks
QLoRA	0.1 - 1%	6-10 GB	Near LoRA	Consumer GPU fine-tuning

Choosing the Right Method

Start with QLoRA if you are on a single GPU with less than 24 GB VRAM. It lets you work with models up to 13B or 30B parameters.

Use LoRA if you have 24-40 GB VRAM and want faster training than QLoRA.

Use full fine-tuning if you have a multi-GPU setup, a large high-quality dataset (100k+ samples), and need maximum task performance.

Key Hyperparameters in Fine-Tuning

Learning rate: Typically much lower for fine-tuning than pre-training. Common range: 1e-5 to 3e-4. Too high and you destroy pre-trained representations.

Rank (r) in LoRA: Higher rank captures more complex updates but uses more memory. Start with 8 or 16.

LoRA alpha: Scales the LoRA update. Common setting: alpha = 2 × rank.

Epochs: Fine-tuning usually needs far fewer epochs than pre-training. 1 to 5 epochs on domain-specific data is typical.

Batch size: Larger batches give smoother gradients. Use gradient accumulation if your GPU limits batch size.

What Makes Good Fine-Tuning Data

Data quality matters more than quantity.

1,000 high-quality examples often outperform 100,000 noisy ones
Format your data to match the task: instruction-response pairs for chat models, completions for base models
Balance your dataset across categories to avoid the model over-indexing on one topic
Remove duplicates; they waste training compute and bias the model

Summary

Neural networks learn by repeating a loop: forward pass, loss calculation, backpropagation, gradient calculation, weight update. Fine-tuning runs this loop on task-specific data starting from a pre-trained model.

Full fine-tuning updates everything. PEFT methods freeze most weights. LoRA approximates weight updates with low-rank matrices. QLoRA adds quantization to push memory requirements lower.

Your choice depends on your GPU, your dataset size, and your performance target.

What I Learned Building an Autonomous Deal-Hunting Agent System

M TOQEER ZIA — Mon, 15 Jun 2026 11:41:48 +0000

Over the past week I built a multi-agent AI system that autonomously scans the internet for bargains, estimates the true value of products using three different pricing techniques, and pushes a notification straight to my phone the moment it finds a deal worth acting on. Along the way I picked up a ton of practical, transferable lessons about agentic AI architecture, RAG, fine-tuning vs. prompting, tool calling, and shipping a real (if scrappy) product with a Gradio front end.

This post is my write-up of the whole journey, with the code that made it work.

The Big Picture: A Team of Specialist Agents

The system, nicknamed "The Price Is Right", is built around the idea that no single model is the best at everything. Instead of one giant prompt, the architecture splits the problem into focused agents that each do one job well, coordinated by a planning agent:

Scanner Agent – trawls deal RSS feeds and uses a cheap LLM to pick the 5 most promising, well-described deals.
Specialist Agent – a small open-source model (Llama 3.2) that I fine-tuned specifically to estimate product prices, deployed serverlessly on Modal.
Frontier Agent – a frontier model (GPT-5.1) doing price estimation with RAG, pulling similar products from a vector database for context.
Ensemble Agent – combines the Specialist, Frontier, and a classic neural network into a single weighted estimate.
Messaging Agent – sends a push notification to my phone via Pushover when a great deal is found.
Autonomous Planning Agent – the "brain" that ties everything together using function/tool calling, deciding what to scan, what to estimate, and when to notify.
Deal Agent Framework – the orchestration layer that runs the whole loop and persists memory between runs.
Gradio UI – a live dashboard showing deals as they're discovered, plus a 3D visualization of the underlying product vector store.

Here's how the five days of building this broke down.

Day 1: Deploying Models to the Cloud with Modal

The first lesson was about infrastructure: how do you run a model (especially a fine-tuned open-source LLM) without managing your own GPU server? The answer here was Modal, a serverless platform for running Python functions in the cloud — including on GPUs.

The "hello world" of Modal is refreshingly simple. You define an App, an Image (essentially a container spec with pip dependencies), and decorate a normal Python function with @app.function:

# hello.py
import modal
from modal import Image

# Setup

app = modal.App("hello")
image = Image.debian_slim().pip_install("requests")

# Hello!


@app.function(image=image)
def hello() -> str:
    import requests

    response = requests.get("https://ipinfo.io/json")
    data = response.json()
    city, region, country = data["city"], data["region"], data["country"]
    return f"Hello from {city}, {region}, {country}!!"


# New - added thanks to student Tue H.!


@app.function(image=image, region="eu")
def hello_europe() -> str:
    import requests

    response = requests.get("https://ipinfo.io/json")
    data = response.json()
    city, region, country = data["city"], data["region"], data["country"]
    return f"Hello from {city}, {region}, {country}!!"

What I loved here is the region="eu" parameter — with one keyword argument you can pin where in the world your function actually executes, which matters for latency, data residency, and sometimes cost.

Calling it locally vs. remotely is just as simple from a notebook:

from hello import app, hello, hello_europe

with app.run():
    reply = hello.local()   # runs on your machine

with app.run():
    reply = hello.remote()  # runs on Modal's cloud

Running a Real LLM on a GPU

The next step up is running an actual language model. This is where Modal's GPU support and Secrets come in — you don't want to hardcode your Hugging Face token, so you register it once in Modal's dashboard under a name (e.g. huggingface-secret) and reference it in code:

# llama.py
import modal
from modal import Image

# Setup

app = modal.App("llama")
image = Image.debian_slim().pip_install("torch", "transformers", "accelerate")
secrets = [modal.Secret.from_name("huggingface-secret")]
GPU = "T4"
MODEL_NAME = "meta-llama/Llama-3.2-3B"


@app.function(image=image, secrets=secrets, gpu=GPU, timeout=1800)
def generate(prompt: str) -> str:
    from transformers import AutoTokenizer, AutoModelForCausalLM, set_seed

    tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
    tokenizer.pad_token = tokenizer.eos_token
    tokenizer.padding_side = "right"

    model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto")

    set_seed(42)
    inputs = tokenizer.encode(prompt, return_tensors="pt").to("cuda")
    outputs = model.generate(inputs, max_new_tokens=5)
    return tokenizer.decode(outputs[0])

A few things clicked for me here:

gpu="T4" is all it takes to request a GPU-backed container. No CUDA driver wrangling, no Dockerfile.
timeout=1800 matters because the first call has to download and load model weights — that cold start can take minutes.
Everything inside the function body that needs the GPU (the transformers imports, tokenizer, model) is imported inside the function, so it only happens in the cloud container, not on my laptop.

From Ephemeral Apps to a Deployed Pricer Service

The really important conceptual jump on Day 1 was going from an ephemeral app (with app.run(): ..., which spins up and tears down for a single call) to a deployed app:

uv run modal deploy -m pricer_service

Once deployed, the service runs independently of my notebook, and I can call it from anywhere just by referencing it by name:

import modal
Pricer = modal.Cls.from_name("pricer-service", "Pricer")
pricer = Pricer()
reply = pricer.price.remote("Quadcast HyperX condenser mic, connects via usb-c to your computer for crystal clear audio")
print(reply)

This is essentially how you'd put a fine-tuned model "behind an API" for a production system — and it's the foundation for the Specialist Agent, which wraps this exact deployed pricer.

There's also a nice optimization here: by default a Modal container scales down to zero when idle, so the first call after inactivity can take ~30 seconds to wake up. If you're willing to spend a few extra credits, you can keep a container warm:

import modal
Pricer = modal.Cls.from_name("pricer-service", "Pricer")
pricer = Pricer()
pricer.update_autoscaler(scaledown_window=1200)  # stay warm for 20 minutes

Takeaway: Modal turns "deploy a fine-tuned model as a microservice" into a one-line decorator and a one-line CLI command. The mental model — write a normal Python function, decorate it, deploy it, call it like a remote object — is something I'll reuse for any future "specialist model as a service" project.

Day 2: RAG, the Frontier Agent, and an Ensemble of Pricers

Day 2 was about a different way to make a frontier model (GPT-5.1) better at a narrow task — Retrieval Augmented Generation (RAG) — and then about combining multiple pricing strategies into one.

Building the Vector Store

The first ingredient is a local, open-source sentence embedding model, which turns text into a 384-dimensional vector capturing its meaning:

from sentence_transformers import SentenceTransformer

encoder = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')

# Pass in a list of texts, get back a numpy array of vectors
vector = encoder.encode(["A proficient AI engineer who has almost reached the finale of AI Engineering Core Track!"])[0]
print(vector.shape)  # (384,)

These vectors get stored — along with the product description and metadata (category, price) — in a Chroma vector database, batched 1,000 items at a time across hundreds of thousands of products:

collection_name = "products"
existing_collection_names = [collection.name for collection in client.list_collections()]

if collection_name not in existing_collection_names:
    collection = client.create_collection(collection_name)
    for i in tqdm(range(0, len(train), 1000)):
        documents = [item.summary for item in train[i: i+1000]]
        vectors = encoder.encode(documents).astype(float).tolist()
        metadatas = [{"category": item.category, "price": item.price} for item in train[i: i+1000]]
        ids = [f"doc_{j}" for j in range(i, i+1000)]
        ids = ids[:len(documents)]
        collection.add(ids=ids, documents=documents, embeddings=vectors, metadatas=metadatas)

collection = client.get_or_create_collection(collection_name)

Visualizing the Vector Space

One of the most satisfying moments was reducing those 384-dimensional vectors down to 3D with t-SNE and seeing the products cluster by category — electronics in one corner, musical instruments in another:

from sklearn.manifold import TSNE
import plotly.graph_objects as go

tsne = TSNE(n_components=3, random_state=42)
reduced_vectors = tsne.fit_transform(vectors)

fig = go.Figure(data=[go.Scatter3d(
    x=reduced_vectors[:, 0],
    y=reduced_vectors[:, 1],
    z=reduced_vectors[:, 2],
    mode='markers',
    marker=dict(size=2, color=colors, opacity=0.7),
    text=[f"Category: {c}<br>Text: {d[:50]}..." for c, d in zip(categories, documents)],
    hoverinfo='text'
)])

fig.update_layout(
    title='3D Chroma Vector Store Visualization',
    scene=dict(xaxis_title='x', yaxis_title='y', zaxis_title='z'),
    width=1200, height=800,
    margin=dict(r=20, b=10, l=10, t=40)
)
fig.show()

It's one thing to be told "embeddings capture semantic similarity" — it's another to literally see the same product categories form tight clusters in 3D space.

Using Retrieval to Ground a Frontier Model

The actual RAG technique is simple once the vector store exists: for a new product, find its 5 nearest neighbours, and stuff their descriptions and prices into the prompt as context before asking GPT-5.1 to estimate the price:

def find_similars(item):
    vec = vector(item)
    results = collection.query(query_embeddings=vec.astype(float).tolist(), n_results=5)
    documents = results['documents'][0][:]
    prices = [m['price'] for m in results['metadatas'][0][:]]
    return documents, prices


def make_context(similars, prices):
    message = "For context, here are some other items that might be similar to the item you need to estimate.\n\n"
    for similar, price in zip(similars, prices):
        message += f"Potentially related product:\n{similar}\nPrice is ${price:.2f}\n\n"
    return message


def messages_for(item, similars, prices):
    message = f"Estimate the price of this product. Respond with the price, no explanation\n\n{item.summary}\n\n"
    message += make_context(similars, prices)
    return [{"role": "user", "content": message}]


def gpt_5__1_rag(item):
    documents, prices = find_similars(item)
    response = completion(model="gpt-5.1", messages=messages_for(item, documents, prices), reasoning_effort="none", seed=42)
    return response.choices[0].message.content

This became the heart of the Frontier Agent.

The Ensemble: Combining Three Very Different Models

The biggest "aha" of Day 2 was realizing that three completely different approaches to the same problem — estimate a product's price — could be blended into something better than any one of them alone:

RAG + GPT-5.1 (gpt_5__1_rag) — frontier model with retrieved context
The fine-tuned Specialist running on Modal (specialist) — small model, fine-tuned specifically on this task
A classic Deep Neural Network trained on the embeddings from Week 6 (deep_neural_network)

def get_price(reply):
    reply = reply.replace("$", "").replace(",", "")
    match = re.search(r"[-+]?\d*\.\d+|\d+", reply)
    return float(match.group()) if match else 0


def specialist(item):
    return pricer.price.remote(item.summary)


def ensemble(item):
    price1 = get_price(gpt_5__1_rag(item))
    price2 = specialist(item)
    price3 = deep_neural_network(item)
    return price1 * 0.8 + price2 * 0.1 + price3 * 0.1

The weighting (0.8 / 0.1 / 0.1) was chosen because, when evaluated against held-out test data, the RAG-based frontier model was the strongest individual predictor — but the other two still nudged the final estimate in a useful direction. This is essentially a tiny, hand-tuned mixture-of-experts, and it generalizes to most "estimate a number from text" problems: get a few independent estimators, then blend.

By the end of Day 2, all three of these had been wrapped into proper agent classes — FrontierAgent, NeuralNetworkAgent, and EnsembleAgent — each exposing a simple .price(description) method, ready to be called by higher-level orchestration.

Day 3: Scanning the Web and Pushing Notifications

Day 2 answered "given a deal, how much is it really worth?". Day 3 answered the question that has to come first: "where do the deals come from in the first place, and how do I find out about a good one without staring at a screen?"

The Scanner Agent

The Scanner Agent subscribes to deal RSS feeds, scrapes the raw listings, and then asks a cheap LLM (openai/gpt-oss-20b:free via OpenRouter) to pick the 5 best-described deals — specifically ones where the price is unambiguous, since deal sites love phrases like "$50 off" which describe the discount, not the price.

The prompt design here was a small lesson in itself — being explicit about edge cases massively improves reliability:

SYSTEM_PROMPT = """You identify and summarize the 5 most detailed deals from a list, by selecting deals that have the most detailed, high quality description and the most clear price.
Respond strictly in JSON with no explanation, using this format. You should provide the price as a number derived from the description. If the price of a deal isn't clear, do not include that deal in your response.
Most important is that you respond with the 5 deals that have the most detailed product description with price. It's not important to mention the terms of the deal; most important is a thorough description of the product.
Be careful with products that are described as "$XXX off" or "reduced by $XXX" - this isn't the actual price of the product. Only respond with products when you are highly confident about the price. 
"""

USER_PROMPT_PREFIX = """Respond with the most promising 5 deals from this list, selecting those which have the most detailed, high quality product description and a clear price that is greater than 0.
You should rephrase the description to be a summary of the product itself, not the terms of the deal.
Remember to respond with a short paragraph of text in the product_description field for each of the 5 items that you select.
Be careful with products that are described as "$XXX off" or "reduced by $XXX" - this isn't the actual price of the product. Only respond with products when you are highly confident about the price. 

Deals:

"""

USER_PROMPT_SUFFIX = "\n\nInclude exactly 5 deals, no more."

Combined with structured output (Pydantic models via .chat.completions.parse(... response_format=DealSelection ...)), this guarantees the agent returns exactly the shape of data the rest of the pipeline expects — no brittle JSON-parsing of free text required.

The Messaging Agent and Pushover

The last piece of Day 3 was closing the loop with the outside world: push notifications. Pushover makes this almost embarrassingly easy — register an app, get a user key and an API token, and send a notification with a single HTTP POST:

pushover_user = os.getenv('PUSHOVER_USER')
pushover_token = os.getenv('PUSHOVER_TOKEN')
pushover_url = "https://api.pushover.net/1/messages.json"

def push(message):
    print(f"Push: {message}")
    payload = {"user": pushover_user, "token": pushover_token, "message": message}
    requests.post(pushover_url, data=payload)

This got wrapped into a MessagingAgent with a .notify(description, deal_price, estimated_value, url) method — turning "we found a great deal" into "your phone buzzes."

Takeaway: Agentic systems feel magical, but a lot of the magic is just plumbing — RSS feeds in, structured LLM output, push notifications out. Getting the plumbing rock-solid (and the prompts very explicit about edge cases) is what makes the "intelligent" part trustworthy.

Day 4: The Autonomous Planning Agent — Teaching an LLM to Use Tools

This was, for me, the most conceptually important day. Up to this point, every agent was called explicitly by my code: "now run the scanner," "now run the ensemble," "now send a notification." Day 4 flips that around — the LLM itself decides what to do and in what order, by calling tools.

Step 1: Fake Tools, Real Concepts

Before wiring up the real agents, the notebook builds three fake functions just to understand the tool-calling loop:

def scan_the_internet_for_bargains() -> str:
    """ This tool scans the internet for great deals and gets a curated list of promising deals """
    print("Fake function to scan the internet - this returns a hardcoded set of deals")
    return test_results.model_dump_json()


def estimate_true_value(description: str) -> str:
    """
    This tool estimates the true value of a product based on a text description of it
    """
    print(f"Fake function to estimating true value of {description[:20]}... - this always returns $300")
    return f"Product {description} has an estimated true value of $300"


def notify_user_of_deal(description: str, deal_price: float, estimated_true_value: float, url: str) -> str:
    """
    This tool notifies the user of a great deal, given a description of it, the price of the deal, and the estimated true value
    """
    print(f"Fake function to notify user of {description} which costs {deal_price} and estimate is {estimated_true_value}")
    return "notification sent ok"

Each tool also needs a JSON Schema describing its name, description, and parameters — this is what actually gets sent to the LLM so it knows what's available and how to call it:

scan_function = {
    "name": "scan_the_internet_for_bargains",
    "description": "Returns top bargains scraped from the internet along with the price each item is being offered for",
    "parameters": {
        "type": "object",
        "properties": {},
        "required": [],
        "additionalProperties": False
    }
}

notify_function = {
    "name": "notify_user_of_deal",
    "description": "Send the user a push notification about the single most compelling deal; only call this one time",
    "parameters": {
        "type": "object",
        "properties": {
            "description": {"type": "string", "description": "The description of the item itself scraped from the internet"},
            "deal_price": {"type": "number", "description": "The price offered by this deal scraped from the internet"},
            "estimated_true_value": {"type": "number", "description": "The estimated actual value that this is worth"},
            "url": {"type": "string", "description": "The URL of this deal as scraped from the internet"}
        },
        "required": ["description", "deal_price", "estimated_true_value", "url"],
        "additionalProperties": False
    }
}

tools = [{"type": "function", "function": scan_function},
         {"type": "function", "function": estimate_function},
         {"type": "function", "function": notify_function}]

Step 2: The Agent Loop

The real magic is this loop. The LLM is given the tools and a goal; if it decides to call a tool, the code executes the real Python function and feeds the result back in — and this repeats until the model is satisfied:

def handle_tool_call(message):
    """
    Actually call the tools associated with this message
    """
    results = []
    for tool_call in message.tool_calls:
        tool_name = tool_call.function.name
        raw_args = json.loads(tool_call.function.arguments)
        tool = globals().get(tool_name)

        if tool:
            # Some models (especially smaller free ones) sometimes return
            # stray/invalid keys (like "") in the arguments JSON, even for
            # functions that take no parameters. Filter to only the keys
            # the function actually accepts.
            valid_params = set(inspect.signature(tool).parameters.keys())
            arguments = {k: v for k, v in raw_args.items() if k in valid_params}
            result = tool(**arguments)
        else:
            result = {}

        results.append({"role": "tool", "content": json.dumps(result), "tool_call_id": tool_call.id})
    return results


system_message = "You find great deals on bargain products using your tools, and notify the user of the best bargain."
user_message = """
First, use your tool to scan the internet for bargain deals. Then for each deal, use your tool to estimate its true value.
Then pick the single most compelling deal where the price is much lower than the estimated true value, and use your tool to notify the user.
Then just reply OK to indicate success.
"""
messages = [{"role": "system", "content": system_message}, {"role": "user", "content": user_message}]

done = False
while not done:
    response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)
    if response.choices[0].finish_reason == "tool_calls":
        message = response.choices[0].message
        results = handle_tool_call(message)
        messages.append(message)
        messages.extend(results)
    else:
        done = True
response.choices[0].message.content

A subtlety that's easy to miss but really matters in practice: smaller, free-tier models sometimes hallucinate extra arguments in their tool calls (like an empty-string key "" for a function that takes no parameters at all). The fix — filtering raw_args down to only the parameters the function's signature actually accepts via inspect.signature — is the kind of defensive coding that's invisible until you're debugging a mysterious TypeError at 11pm.

Step 3: Swap Fakes for Reality

Once the loop works with fake functions, the swap to the real AutonomousPlanningAgent is almost anticlimactic — same loop, same tool schemas, but scan_the_internet_for_bargains now really calls the ScannerAgent, estimate_true_value really calls the EnsembleAgent, and notify_user_of_deal really calls the MessagingAgent:

DB = "products_vectorstore"
client = chromadb.PersistentClient(path=DB)
collection = client.get_or_create_collection('products')

from agents.autonomous_planning_agent import AutonomousPlanningAgent
agent = AutonomousPlanningAgent(collection)
agent.plan()

Takeaway: Tool/function calling turns an LLM from "a thing that writes text" into "a thing that orchestrates other systems." The hard part isn't the API call — it's (a) writing tight descriptions so the model picks the right tool, and (b) writing tolerant glue code, because the model will occasionally send malformed arguments.

Day 5: Bringing It All Together — Framework, Memory, and a Live UI

The final day was about productionizing: wrapping everything in a reusable framework with persistent memory, colored logs, and a Gradio dashboard that updates in real time.

The Deal Agent Framework

DealAgentFramework is the top-level orchestrator. It owns the Chroma client, lazily creates the PlanningAgent, and — critically — persists discovered deals to memory.json so the system remembers what it's already found across restarts:

import os
import sys
import logging
import json
from typing import List
from dotenv import load_dotenv
import chromadb
from agents.planning_agent import PlanningAgent
from agents.deals import Opportunity
from sklearn.manifold import TSNE
import numpy as np

load_dotenv(override=True)

# Colors for logging
BG_BLUE = "\033[44m"
WHITE = "\033[37m"
RESET = "\033[0m"

# Colors for plot
CATEGORIES = [
    "Appliances",
    "Automotive",
    "Cell_Phones_and_Accessories",
    "Electronics",
    "Musical_Instruments",
    "Office_Products",
    "Tools_and_Home_Improvement",
    "Toys_and_Games",
]
COLORS = ["red", "blue", "brown", "orange", "yellow", "green", "purple", "cyan"]


def init_logging():
    root = logging.getLogger()
    root.setLevel(logging.INFO)

    handler = logging.StreamHandler(sys.stdout)
    handler.setLevel(logging.INFO)
    formatter = logging.Formatter(
        "[%(asctime)s] [Agents] [%(levelname)s] %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S %z",
    )
    handler.setFormatter(formatter)
    root.addHandler(handler)


class DealAgentFramework:
    DB = "products_vectorstore"
    MEMORY_FILENAME = "memory.json"

    def __init__(self):
        init_logging()
        client = chromadb.PersistentClient(path=self.DB)
        self.memory = self.read_memory()
        self.collection = client.get_or_create_collection("products")
        self.planner = None

    def init_agents_as_needed(self):
        if not self.planner:
            self.log("Initializing Agent Framework")
            self.planner = PlanningAgent(self.collection)
            self.log("Agent Framework is ready")

    def read_memory(self) -> List[Opportunity]:
        if os.path.exists(self.MEMORY_FILENAME):
            with open(self.MEMORY_FILENAME, "r") as file:
                data = json.load(file)
            opportunities = [Opportunity(**item) for item in data]
            return opportunities
        return []

    def write_memory(self) -> None:
        data = [opportunity.model_dump() for opportunity in self.memory]
        with open(self.MEMORY_FILENAME, "w") as file:
            json.dump(data, file, indent=2)

    @classmethod
    def reset_memory(cls) -> None:
        data = []
        if os.path.exists(cls.MEMORY_FILENAME):
            with open(cls.MEMORY_FILENAME, "r") as file:
                data = json.load(file)
        truncated = data[:2]
        with open(cls.MEMORY_FILENAME, "w") as file:
            json.dump(truncated, file, indent=2)

    def log(self, message: str):
        text = BG_BLUE + WHITE + "[Agent Framework] " + message + RESET
        logging.info(text)

    def run(self) -> List[Opportunity]:
        self.init_agents_as_needed()
        logging.info("Kicking off Planning Agent")
        result = self.planner.plan(memory=self.memory)
        logging.info(f"Planning Agent has completed and returned: {result}")
        if result:
            self.memory.append(result)
            self.write_memory()
        return self.memory

    @classmethod
    def get_plot_data(cls, max_datapoints=2000):
        client = chromadb.PersistentClient(path=cls.DB)
        collection = client.get_or_create_collection("products")
        result = collection.get(
            include=["embeddings", "documents", "metadatas"], limit=max_datapoints
        )
        vectors = np.array(result["embeddings"])
        documents = result["documents"]
        categories = [metadata["category"] for metadata in result["metadatas"]]
        colors = [COLORS[CATEGORIES.index(c)] for c in categories]
        tsne = TSNE(n_components=3, random_state=42, n_jobs=-1)
        reduced_vectors = tsne.fit_transform(vectors)
        return documents, reduced_vectors, colors


if __name__ == "__main__":
    DealAgentFramework().run()

A few patterns I want to remember from this file:

Lazy initialization (init_agents_as_needed) — spinning up the full agent stack (which includes loading models and connecting to vector stores) is expensive, so it only happens once, on first use.
Memory as a flat JSON file — no database needed for a project at this scale. memory.json is literally a list of Opportunity objects (a deal + an estimated value + a discount), serialized via Pydantic's model_dump().
reset_memory as a classmethod — a clean way to "rewind" the demo back to a known state (2 saved deals) without touching the running instance.
Colored terminal logging via raw ANSI escape codes (BG_BLUE, WHITE, RESET) — a small touch, but it makes the live log stream from multiple agents much easier to scan visually.

Reformatting ANSI Colors for the Browser

Speaking of colors — the terminal uses ANSI escape codes, but the Gradio UI renders HTML. log_utils.py is a tiny but clever bridge between the two: it maps each ANSI color combination to a CSS hex color and swaps the escape codes for <span style="color: ..."> tags:

# Foreground colors
RED = '\033[31m'
GREEN = '\033[32m'
YELLOW = '\033[33m'
BLUE = '\033[34m'
MAGENTA = '\033[35m'
CYAN = '\033[36m'
WHITE = '\033[37m'

# Background color
BG_BLACK = '\033[40m'
BG_BLUE = '\033[44m'

# Reset code to return to default color
RESET = '\033[0m'

mapper = {
    BG_BLACK+RED: "#dd0000",
    BG_BLACK+GREEN: "#00dd00",
    BG_BLACK+YELLOW: "#dddd00",
    BG_BLACK+BLUE: "#0000ee",
    BG_BLACK+MAGENTA: "#aa00dd",
    BG_BLACK+CYAN: "#00dddd",
    BG_BLACK+WHITE: "#87CEEB",
    BG_BLUE+WHITE: "#ff7800"
}


def reformat(message):
    for key, value in mapper.items():
        message = message.replace(key, f'<span style="color: {value}">')
    message = message.replace(RESET, '</span>')
    return message

Every agent in the system logs its activity with a different color (set in its own __init__), so when this gets rendered in the browser, you can visually tell at a glance which agent is talking — the planner, the scanner, the frontier agent, etc. — without reading a single word.

The Gradio UI: From Static Mockup to Live Dashboard

The UI was built up in layers, which I think is a great way to learn Gradio:

Layer 1 — just get something on screen:

with gr.Blocks(title="The Price is Right", fill_width=True) as ui:
    with gr.Row():
        gr.Markdown('<div style="text-align: center;font-size:24px">The Price is Right - Deal Hunting Agentic AI</div>')
    with gr.Row():
        gr.Markdown('<div style="text-align: center;font-size:14px">Autonomous agent framework that finds online deals, collaborating with a proprietary fine-tuned LLM deployed on Modal, and a RAG pipeline with a frontier model and Chroma.</div>')

ui.launch(inbrowser=True)

Layer 2 — add a live data table backed by application state:

with gr.Blocks(title="The Price is Right", fill_width=True) as ui:

    initial_deal = Deal(product_description="Example description", price=100.0, url="https://cnn.com")
    initial_opportunity = Opportunity(deal=initial_deal, estimate=200.0, discount=100.0)
    opportunities = gr.State([initial_opportunity])

    def get_table(opps):
        return [[opp.deal.product_description, opp.deal.price, opp.estimate, opp.discount, opp.deal.url] for opp in opps]

    with gr.Row():
        opportunities_dataframe = gr.Dataframe(
            headers=["Description", "Price", "Estimate", "Discount", "URL"],
            wrap=True,
            column_widths=[4, 1, 1, 1, 2],
            row_count=10,
            col_count=5,
            max_height=400,
        )

    ui.load(get_table, inputs=[opportunities], outputs=[opportunities_dataframe])

ui.launch(inbrowser=True)

A small but important Gradio version note from this layer: in Gradio v5, the height parameter for Dataframe was renamed to max_height — exactly the kind of breaking change that's easy to lose an hour to if you don't know to look for it.

Layer 3 — wire up real agents and make rows clickable:

agent_framework = DealAgentFramework()
agent_framework.init_agents_as_needed()

with gr.Blocks(title="The Price is Right", fill_width=True) as ui:
    ...
    def do_select(opportunities, selected_index: gr.SelectData):
        row = selected_index.index[0]
        opportunity = opportunities[row]
        agent_framework.planner.messenger.alert(opportunity)
    ...
    opportunities_dataframe.select(do_select, inputs=[opportunities], outputs=[])

ui.launch(inbrowser=True)

The Final Application: Streaming Logs + Background Agent Run + 3D Plot

The fully assembled price_is_right.py brings everything together: a background thread runs the agent framework's run() loop, a queue.Queue-based logging handler streams log lines into the UI in (near) real time, and a 3D Plotly visualization of the product vector store sits alongside the deal table:

import logging
import queue
import threading
import time
import gradio as gr
from deal_agent_framework import DealAgentFramework
from log_utils import reformat
import plotly.graph_objects as go
from dotenv import load_dotenv

load_dotenv(override=True)


class QueueHandler(logging.Handler):
    def __init__(self, log_queue):
        super().__init__()
        self.log_queue = log_queue

    def emit(self, record):
        self.log_queue.put(self.format(record))


def html_for(log_data):
    output = "<br>".join(log_data[-18:])
    return f"""
    <div id="scrollContent" style="height: 400px; overflow-y: auto; border: 1px solid #ccc; background-color: #222229; padding: 10px;">
    {output}
    </div>
    """


def setup_logging(log_queue):
    handler = QueueHandler(log_queue)
    formatter = logging.Formatter(
        "[%(asctime)s] %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S %z",
    )
    handler.setFormatter(formatter)
    logger = logging.getLogger()
    logger.addHandler(handler)
    logger.setLevel(logging.INFO)


class App:
    def __init__(self):
        self.agent_framework = None

    def get_agent_framework(self):
        if not self.agent_framework:
            self.agent_framework = DealAgentFramework()
        return self.agent_framework

    def run(self):
        with gr.Blocks(title="The Price is Right", fill_width=True) as ui:
            log_data = gr.State([])

            def table_for(opps):
                return [
                    [
                        opp.deal.product_description,
                        f"${opp.deal.price:.2f}",
                        f"${opp.estimate:.2f}",
                        f"${opp.discount:.2f}",
                        opp.deal.url,
                    ]
                    for opp in opps
                ]

            def update_output(log_data, log_queue, result_queue):
                initial_result = table_for(self.get_agent_framework().memory)
                final_result = None
                while True:
                    try:
                        message = log_queue.get_nowait()
                        log_data.append(reformat(message))
                        yield log_data, html_for(log_data), final_result or initial_result
                    except queue.Empty:
                        try:
                            final_result = result_queue.get_nowait()
                            yield log_data, html_for(log_data), final_result or initial_result
                        except queue.Empty:
                            if final_result is not None:
                                break
                            time.sleep(0.1)

            def get_plot():
                documents, vectors, colors = DealAgentFramework.get_plot_data(max_datapoints=800)
                fig = go.Figure(
                    data=[
                        go.Scatter3d(
                            x=vectors[:, 0],
                            y=vectors[:, 1],
                            z=vectors[:, 2],
                            mode="markers",
                            marker=dict(size=2, color=colors, opacity=0.7),
                        )
                    ]
                )
                fig.update_layout(
                    scene=dict(
                        xaxis_title="x", yaxis_title="y", zaxis_title="z",
                        aspectmode="manual",
                        aspectratio=dict(x=2.2, y=2.2, z=1),
                        camera=dict(eye=dict(x=1.6, y=1.6, z=0.8)),
                    ),
                    height=400,
                    margin=dict(r=5, b=1, l=5, t=2),
                )
                return fig

            def do_run():
                new_opportunities = self.get_agent_framework().run()
                return table_for(new_opportunities)

            def run_with_logging(initial_log_data):
                log_queue = queue.Queue()
                result_queue = queue.Queue()
                setup_logging(log_queue)

                def worker():
                    result_queue.put(do_run())

                thread = threading.Thread(target=worker)
                thread.start()

                for log_data, output, final_result in update_output(initial_log_data, log_queue, result_queue):
                    yield log_data, output, final_result

            def do_select(selected_index: gr.SelectData):
                opportunities = self.get_agent_framework().memory
                row = selected_index.index[0]
                opportunity = opportunities[row]
                self.get_agent_framework().planner.messenger.alert(opportunity)

            with gr.Row():
                opportunities_dataframe = gr.Dataframe(
                    headers=["Deals found so far", "Price", "Estimate", "Discount", "URL"],
                    wrap=True, column_widths=[6, 1, 1, 1, 3],
                    row_count=10, col_count=5, max_height=400,
                )
            with gr.Row():
                with gr.Column(scale=1):
                    logs = gr.HTML()
                with gr.Column(scale=1):
                    plot = gr.Plot(value=get_plot(), show_label=False)

            ui.load(run_with_logging, inputs=[log_data], outputs=[log_data, logs, opportunities_dataframe])

            timer = gr.Timer(value=300, active=True)
            timer.tick(run_with_logging, inputs=[log_data], outputs=[log_data, logs, opportunities_dataframe])

            opportunities_dataframe.select(do_select)

        ui.launch(share=False, inbrowser=True)


if __name__ == "__main__":
    App().run()

The two patterns I most want to carry forward from this file:

Background work + streaming UI via a generator. run_with_logging is a Python generator hooked up as a Gradio event handler. It kicks off a worker thread, then repeatedly yields updated state — so the UI refreshes live while a slow agentic process runs, instead of freezing for the whole duration.
gr.Timer for autonomous operation. A Timer set to 300 seconds means the whole "scan → estimate → notify" cycle re-runs automatically every 5 minutes — turning a notebook experiment into something that genuinely behaves like a background agent.

What I'd Tell Past-Me Before Starting This

A few cross-cutting lessons that apply far beyond this specific project:

Specialize, then orchestrate. Each agent (Scanner, Specialist, Frontier, Ensemble, Messaging, Planner) does one narrow thing well. The "intelligence" of the overall system comes from composition, not from any single giant prompt.
RAG is cheap leverage. Adding 5 similar examples with known prices to a prompt turned out to meaningfully improve a frontier model's estimates — for the cost of a vector lookup.
Ensembling beats picking a favorite. Rather than agonizing over which pricing approach is "best," a weighted blend of three approaches (0.8 / 0.1 / 0.1) outperformed any single one on the held-out test set.
Tool calling needs defensive code. The model will send slightly malformed arguments sometimes, especially smaller/free models. Filter arguments against the real function signature before calling it.
Memory is just a JSON file (for now). Don't reach for a database until you actually need one — persisting a List[Opportunity] to memory.json was enough to give the system continuity across restarts.
Generators + threads make agentic UIs feel alive. A background worker thread plus a generator-based Gradio handler is a lightweight way to show "the agent is thinking" in real time.
Small UX details (colored logs, a live 3D plot, clickable rows) make an agentic system feel trustworthy — you can see what it's doing and why, which matters enormously when the system is making autonomous decisions on your behalf.

Putting it all together, DealAgentFramework().run() now quietly: scans deal feeds, filters to the 5 best-described deals, estimates each one's true value via an ensemble of three models, picks the single best opportunity, saves it to memory, and — if it's a great deal — buzzes my phone. All while a live dashboard shows exactly what's happening and why.

How I Fine-Tuned Llama 3 to Think Like DeepSeek — A Practical Guide to LoRA & QLoRA

M TOQEER ZIA — Thu, 11 Jun 2026 06:32:38 +0000

I turned a base Llama 3.2 model into a step-by-step reasoning machine using a free Colab GPU. Here's exactly how it works.

So you've heard the buzz around fine-tuning LLMs, but every tutorial either drowns you in math or skips the "why" entirely. This article cuts through both. We'll cover the core concepts — LoRA, QLoRA, quantization — and then walk through a real coding demo that produced a genuinely surprising result: a fine-tuned Llama 3 model that reasons through problems step-by-step, just like DeepSeek.

Let's get into it.

The Problem With Base LLMs

A pre-trained model like Llama or GPT is incredibly general. That's its superpower — and its limitation.

Your company has private data the model has never seen. Your brand has a specific tone, vocabulary, and format for responses. A base LLM doesn't know any of that.

You have two main ways to fix this:

Option 1: RAG (Retrieval-Augmented Generation)

RAG doesn't retrain the model at all. Instead, it retrieves relevant chunks from an external knowledge source (a database, PDFs, docs) and injects them into the prompt at inference time.

Pros: Cheap, fast to set up, no GPU needed.

Cons: The model still responds in its own generic voice. It can sound robotic, off-brand, or miss the nuanced tone your use case needs.

Option 2: Fine-Tuning

Fine-tuning actually retrains the model on your specific dataset — teaching it new knowledge, tone, format, or reasoning style.

Pros: Precise outputs, correct brand voice, specific response formats.

Cons: Computationally expensive, requires a curated dataset, takes time.

In practice, the industry often combines both — fine-tune for tone and format, use RAG for dynamic knowledge retrieval.

What Is Fine-Tuning, Really?

Fine-tuning is a form of transfer learning. You take a model that already understands language, then continue training it on a smaller, task-specific dataset. The model adapts its weights to fit your new data without forgetting everything it already learned.

The challenge? A 7-billion parameter model has billions of weights to update. Full fine-tuning requires enormous GPU memory — often inaccessible to most developers.

That's where LoRA comes in.

LoRA: Low-Rank Adaptation

LoRA is a Parameter Efficient Fine-Tuning (PEFT) method. The key insight is elegant:

Instead of updating all the original model weights, freeze them — and train only a small set of new parameters layered on top.

How It Works

During full fine-tuning, you'd update a weight matrix W by computing a change ΔW (delta W). The problem is that ΔW is the same enormous size as W.

LoRA's trick: decompose ΔW into two much smaller matrices, A and B, where:

ΔW = A × B

If W is a 1024×256 matrix (262,144 parameters), and you set rank R = 16, then:

Matrix A is 1024×16
Matrix B is 16×256
Total trainable parameters: (1024×16) + (16×256) = 20,480 instead of 262,144

That's a ~93% reduction in parameters to train.

Key Hyperparameter: Rank (R)

The rank R controls the trade-off between efficiency and expressiveness.

Lower R (e.g., 4, 8) → fewer parameters, faster training, less expressive
Higher R (e.g., 16, 32) → more parameters, can capture more complex adaptations

Common starting values are R = 8 or R = 16.

QLoRA: When Your GPU Is Too Small

LoRA is efficient, but the base model still needs to fit in memory. A 7B parameter model at full float32 precision requires ~28 GB of GPU RAM. Most consumer GPUs and free Colab runtimes offer 12–16 GB.

QLoRA solves this by adding quantization before applying LoRA.

What Is Quantization?

Quantization converts high-precision numbers to lower-precision formats to save memory:

Format	Bytes per value	7B Model Memory
float32	4 bytes	~28 GB
int8	1 byte	~7 GB
NF4 (4-bit)	0.5 bytes	~3.5 GB

You're essentially compressing the model before training on it. The accuracy loss is surprisingly minimal for most tasks.

NF4: Normal Float 4

NF4 isn't just regular 4-bit quantization. Neural network weights follow a roughly normal (bell curve) distribution, and NF4 is specifically designed for this:

Instead of equal-width bins, NF4 uses percentile-based bins that pack more precision where the data is dense (near zero) and less where it's sparse (at the extremes).
This makes NF4 far more accurate than naive 4-bit quantization.

The Three Pillars of QLoRA

4-bit NF4 Quantization — Compress the base model to fit in less memory.
Double Quantization — Quantize the quantization constants themselves (the scale and zero-point values), squeezing out a few more MB.
Paged Optimizers — Uses NVIDIA's unified memory to page optimizer states between GPU and CPU RAM when the GPU runs out of space, preventing out-of-memory crashes during training.

Together, these let you fine-tune a 7B+ model on a single consumer GPU.

Hands-On: Fine-Tuning Llama 3.2 on Google Colab

Let's make this concrete. Here's how the actual demo was built.

Setup

Environment: Google Colab with a T4 GPU (free tier)

Library: Unsloth — a highly optimized fine-tuning library

Model: Llama 3.2 3B Instruct

Dataset: ServiceNow R1 Distill SFT — 172,000 rows of math/logic puzzles with problems, step-by-step reasoning chains, and solutions

pip install unsloth

Step 1: Load the Model with 4-bit Quantization

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Llama-3.2-3B-Instruct",
    max_seq_length=2048,
    load_in_4bit=True,  # QLoRA quantization
)

Step 2: Configure LoRA

model = FastLanguageModel.get_peft_model(
    model,
    r=16,                          # Rank
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
)

What are q_proj, k_proj, etc.?

These are the attention projection matrices inside the transformer. LoRA is applied specifically to these layers because they carry the most task-relevant learning signal.

Step 3: Format the Dataset

Each training example is structured as a prompt combining the problem, reasoning, and solution:

def format_prompt(example):
    return f"""
Problem: {example['problem']}

<thinking>
{example['thought']}
</thinking>

Solution: {example['solution']}
"""

The <thinking> tags teach the model to externalize its reasoning — this is how we get the DeepSeek-like behavior.

Step 4: Train with SFTTrainer

from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    args=TrainingArguments(
        num_train_epochs=3,
        per_device_train_batch_size=2,
        learning_rate=2e-4,
        output_dir="outputs",
    ),
)

trainer.train()

Training time: ~20 minutes on a T4 GPU

Final loss: ~0.48 ✅

The Result

After fine-tuning, the model was asked:

"How many R's are in the word 'strawberry'?"

Before fine-tuning (base model): Confidently gives the wrong answer.

After fine-tuning: The model works through it step-by-step:

<thinking>
Let me count each letter in "strawberry":
s-t-r-a-w-b-e-r-r-y
Position 3: r
Position 8: r
Position 9: r
That's 3 R's total.
</thinking>

The word "strawberry" contains 3 R's.

That reasoning pattern — breaking the problem down, checking each step — came entirely from the fine-tuning on the reasoning dataset. The base model didn't do this. The fine-tuned model does.

When Should You Use Each Approach?

Scenario	Best Approach
Add recent/private knowledge	RAG
Change response tone or format	Fine-Tuning
Teach step-by-step reasoning	Fine-Tuning
Limited GPU resources	QLoRA
Moderate GPU, need flexibility	LoRA
Production system at scale	RAG + Fine-Tuning combined

Key Takeaways

Fine-tuning adapts a pre-trained model to your specific task, tone, or data — but it's expensive without the right tools.
LoRA makes it efficient by training only a small set of low-rank adapter matrices, leaving original weights frozen.
QLoRA takes this further by quantizing the base model to 4-bit (NF4) first, making large model fine-tuning possible on consumer hardware.
With Unsloth + Google Colab, you can fine-tune a 3B parameter model in under 30 minutes for free.
The results can be genuinely impressive — a reasoning style the base model didn't have, emergent from the training data alone.

What's Next?

If you want to go deeper:

Try different ranks (R=4 vs R=32) and observe the loss difference
Experiment with which target modules to apply LoRA to
Combine your fine-tuned model with a RAG pipeline for maximum power
Push your model to Hugging Face Hub and share it

Fine-tuning used to require a research lab. Now it takes a free GPU and 30 minutes. There's never been a better time to build something genuinely your own.

Found this useful? Drop a reaction or share it with someone building with LLMs. Questions or corrections? I'm all ears in the comments.

Build a Meeting Minutes AI From Raw Audio

M TOQEER ZIA — Tue, 02 Jun 2026 04:23:16 +0000

A complete walkthrough of speech transcription, LLM inference, tokenization, and 4-bit quantization. Built with Whisper, Llama 3.2, and the HuggingFace ecosystem.

Skill level: Intermediate | Runtime: Google Colab T4 GPU | Models: Whisper-medium, Llama-3.2-3B

The Two-Step Pipeline
Tokenization
Quantization
The Chat Template
Neural Networks and Transformers
Tradeoffs
What You Can Build Next

The Two-Step Pipeline

You feed an audio file into a Python script. Minutes later, you get formatted meeting minutes, a summary, and action items. No manual transcription, no human editor.

The notebook splits the problem into two clean stages. Stage one converts audio to text. Stage two converts text to structured meeting minutes.

Both stages run locally on a free Colab T4 GPU. Stage one uses Whisper from OpenAI (or the API version). Stage two uses Meta's Llama 3.2 3B, loaded with 4-bit quantization so it fits in GPU memory.

What is ASR? Automatic Speech Recognition converts raw audio waveforms into text. Whisper treats audio as a sequence prediction problem: it predicts the next token given all previous audio frames.

Two Transcription Options

The notebook gives you two paths. The open-source path runs Whisper locally on the GPU. The API path sends the audio to OpenAI's servers.

from transformers import pipeline

# Open-source: runs locally on T4 GPU
pipe = pipeline(
    "automatic-speech-recognition",
    model="openai/whisper-medium.en",
    dtype=torch.float16,
    device='cuda',
    return_timestamps=True
)

result = pipe(audio_filename)
transcription = result["text"]

# API option: offloads to OpenAI servers
AUDIO_MODEL = "gpt-4o-mini-transcribe"
transcription = openai.audio.transcriptions.create(
    model=AUDIO_MODEL, file=audio_file, response_format="text"
)

	Open-source path	API path
Cost	Free	Per minute of audio
Data privacy	Stays local	Leaves your environment
Setup	Needs GPU	Works anywhere
Offline use	Yes	No

Tokenization: How Text Becomes Numbers

Language models do not read text. They read numbers. Tokenization is the bridge between the two.

A tokenizer splits your text into subword chunks called tokens, then maps each chunk to an integer ID. The word "quantization" might split into tokens like ["quant", "ization"], producing IDs like [42891, 2065]. The model works with these integer sequences from start to finish.

tokenizer = AutoTokenizer.from_pretrained(LLAMA)
tokenizer.pad_token = tokenizer.eos_token

# apply_chat_template formats your messages list
# into the exact token sequence Llama expects
inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt"
).to("cuda")

Why pad_token matters: When you process multiple sequences in a batch, they need equal length. Shorter sequences get padded with a special token. Setting pad_token = eos_token tells the tokenizer which ID to use. Without it, the tokenizer raises an error during batched inference.

The tokenizer also inserts special tokens like <|begin_of_text|> and role markers for system and user. These are instructions to the model, not content. apply_chat_template handles all of this based on the model's expected format.

Quantization: Shrinking the Model to Fit

A 3B parameter model in full 32-bit precision needs around 12 GB of GPU memory. A free T4 has 15 GB total. 4-bit quantization compresses each weight from 32 bits down to 4 bits, cutting memory use by roughly 8x.

quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,     # quantize the quantization constants too
    bnb_4bit_compute_dtype=torch.bfloat16,  # compute in bfloat16 for speed
    bnb_4bit_quant_type="nf4"           # NormalFloat4: better for normal distributions
)

model = AutoModelForCausalLM.from_pretrained(
    LLAMA,
    device_map="auto",
    quantization_config=quant_config
)

What NF4 means: NormalFloat4 is a 4-bit data type designed for neural network weights, which typically follow a normal distribution. It places more quantization levels near zero (where most weights cluster) and fewer at the extremes. This beats a naive 4-bit integer scheme in accuracy on nearly every benchmark.

Double quantization (bnb_4bit_use_double_quant=True) quantizes the quantization constants themselves too. It saves about 0.4 bits per parameter on top of base 4-bit compression. Small gain, no cost.

device_map="auto" tells HuggingFace which layers go on GPU versus CPU. For a 3B model on a T4, everything fits on GPU.

The Chat Template: Speaking the Model's Language

Instruction-tuned models like Llama 3.2 Instruct were fine-tuned on conversations in a specific format. Send text in the wrong format and the model either ignores your instructions or produces garbage. The chat template enforces the right format every time.

system_message = """
You produce minutes of meetings from transcripts, with summary,
key discussion points, takeaways and action items with owners,
in markdown format without code blocks.
"""

user_prompt = f"""
Below is an extract transcript of a Denver council meeting.
Please write minutes in markdown without code blocks, including:
- a summary with attendees, location and date
- discussion points
- takeaways
- action items with owners

Transcription:
{transcription}
"""

messages = [
    {"role": "system", "content": system_message},
    {"role": "user",   "content": user_prompt}
]

# apply_chat_template wraps messages in Llama's expected token format
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")

System vs user roles: The system message sets the model's persona and output constraints. The user message contains the actual task. Keeping them separate gives you fine-grained control: swap the transcript without touching output format instructions, or change the output format without touching the transcript.

After apply_chat_template, your clean Python dictionaries become a single integer tensor. That tensor goes directly into generate(). No string manipulation after this point — everything is numbers on the GPU.

Neural Networks and Transformer Quantization

A transformer model is a stack of layers. Each layer contains weight matrices stored as 2D arrays of floating point numbers. During a forward pass, the model multiplies your input token embeddings by these matrices over and over, applying attention at each step.

The weight matrices in feed-forward layers are what quantization compresses. At inference time, BitsAndBytes dequantizes each weight block just before the matrix multiplication, performs the multiplication in bfloat16, then moves on. The full 4-bit weights stay compressed in GPU memory at all times.

streamer = TextStreamer(tokenizer)

outputs = model.generate(
    inputs,
    max_new_tokens=2000,
    streamer=streamer  # streams tokens to stdout as generated
)

response = tokenizer.decode(outputs[0])

The TextStreamer prints tokens to the console as the model generates them. The model produces one token per forward pass. You see output build word by word because each word triggers a separate forward pass through all layers.

Tradeoffs to Know Before You Ship

Whisper medium vs large

The notebook uses whisper-medium.en. The .en suffix means English-only. It runs faster and uses less memory than the multilingual version. If your meetings include non-English speakers, swap to whisper-large-v3 and expect roughly 3x more GPU memory usage.

3B vs larger Llama models

Llama 3.2 3B handles summarization well. For long meetings with complex technical jargon, a 70B model produces more accurate action items. You cannot run 70B on a free T4, even with 4-bit quantization. You need either a paid Colab Pro instance or API inference.

Float16 vs bfloat16

Whisper runs in torch.float16. Llama's quantized compute runs in bfloat16. Both are 16-bit formats. Float16 has higher precision for small values. Bfloat16 has a wider dynamic range and is less prone to overflow on modern hardware.

The Colab CUDA error that trips everyone up: If you see CUDA is required but not available for bitsandbytes, your runtime was recycled by Google. Fix: Kernel menu, Disconnect and delete runtime. Reconnect to a fresh T4. Rerun from the top. Do not touch package versions.

What You Can Build Next

The notebook is a foundation. A few extensions that follow naturally:

Speaker diarization. Add pyannote-audio before the Whisper step to tag each segment with a speaker ID. Feed those labels into the prompt so Llama assigns action items to the right person.

Gradio streaming interface. Student Emad S. adapted the notebook to stream tokens into a Gradio UI using TextIteratorStreamer and Python background threads. The result is a browser-based app where you upload audio and watch minutes appear in real time.

Persistent storage. Write the output to a Google Doc via the Drive API. Every meeting auto-archives with a timestamp and a searchable transcript.

References

Resource	Link
MeetingBank dataset	huuuyeah/meetingbank on HuggingFace
Audio dataset	MeetingBank Audio
Denver extract MP3	Google Drive
Whisper model	`openai/whisper-medium.en`
Llama model	`meta-llama/Llama-3.2-3B-Instruct`
Gradio variation	Emad S. Colab

Source dataset: MeetingBank. Models: openai/whisper-medium.en and meta-llama/Llama-3.2-3B-Instruct. Quantization: bitsandbytes. Runtime: Google Colab T4 GPU. Framework: HuggingFace Transformers 4.57.6.

API Authentication: What It Is, Why It Matters, and Which Method to Use

M TOQEER ZIA — Fri, 22 May 2026 05:57:57 +0000

Every API you build is a door. Without authentication, that door has no lock.

Anyone with the URL can walk in, read your data, abuse your resources, or impersonate your users. This article breaks down how API authentication works, the four most common methods, and the one distinction that trips up most developers: authentication versus authorization.

What API Authentication Actually Does

API authentication verifies the identity of whoever is making a request.

Before your API returns any data, it asks one question: who are you? The client must prove its identity. If it cannot, the request fails.

Without this, your API is open to:

Data leaks from anonymous requests
Resource abuse from bots making unlimited calls
Unauthorized access to user data

Authentication also gives you control beyond just blocking strangers. Once you know who is calling your API, you can apply rate limits per client, audit exactly who accessed what and when, and assign different permission levels to different clients.

The Four Main API Authentication Methods

1. HTTP Basic Authentication

The client sends a username and password with every request. The credentials are Base64-encoded and placed in the HTTP header.

Base64 is encoding, not encryption. Anyone who intercepts the request reads the credentials in plain text.

Pros:

Dead simple to implement
Supported natively by every HTTP client

Cons:

Credentials travel with every single request
No hashing or encryption by default
Completely insecure without HTTPS
No way to revoke a single session without changing the password

Use it when: You need the fastest possible prototype on an internal tool behind HTTPS, and you will replace it later.

Real incident: In 2022, a misconfigured CI/CD pipeline exposed Basic Auth credentials in plain text logs. The credentials granted access to a private package registry. Thousands of packages were at risk before the team caught it. The fix required rotating every credential across every service.

2. API Key Authentication

The API provider issues a unique key to each client. The client sends this key with every request, either in the query string, request header, or a cookie.

GET /data HTTP/1.1
Host: api.example.com
X-API-Key: sk_live_4f8a92bc...

API keys are easy to generate and easy to rotate. They give you per-client tracking out of the box.

Pros:

Simple to implement and issue
Easy to monitor per-client usage
Easy to revoke a compromised key without touching other clients

Cons:

Functions like a password: whoever holds the key has full access
No expiry by default
Developers frequently hard-code keys in source code, which ends up in public repositories

Real incident: In 2023, a developer pushed an OpenAI API key to a public GitHub repository. Automated bots scraped it within seconds. The key accumulated thousands of dollars in charges before the developer noticed. GitHub now scans for exposed secrets, but the damage happens before the scan completes.

Use it when: You are building a server-to-server integration where you control both ends and your key never touches a browser or mobile client.

3. JWT (JSON Web Token) Authentication

When a user logs in, the server generates a JWT and returns it to the client. Every subsequent request includes this token. The server validates the token on each request without looking up a session database.

A JWT has three parts, separated by dots:

header.payload.signature

The payload contains the user's identity and claims:

{
  "sub": "user_8821",
  "email": "jane@example.com",
  "role": "admin",
  "exp": 1716560000
}

The signature is cryptographically generated. If anyone tampers with the payload, the signature breaks and the server rejects the token.

Pros:

Stateless: the server stores no session data
Scales horizontally with no shared session store
Token carries its own expiry
Works across domains and services

Cons:

Tokens cannot be invalidated before expiry without extra infrastructure (a blocklist)
If a token is stolen before it expires, the attacker has full access until expiry
Developers sometimes skip signature verification, which defeats the purpose entirely

Real incident: In 2022, researchers found that several APIs accepted JWTs with the algorithm set to "none," meaning no signature was required. Any attacker could forge a token claiming to be any user. The flaw existed because developers copied example code without reading the security notes.

Use it when: You need stateless authentication across multiple services, especially in a microservices architecture where a shared session store creates a bottleneck.

4. OAuth 2.0

OAuth 2.0 is the standard used when your API needs to act on behalf of a user without ever seeing their password.

When you click "Sign in with Google" on any app, you are using OAuth 2.0. Google authenticates you, then issues an access token to the app. The app uses that token to call Google's APIs on your behalf. Your Google password never touches the app.

User → App → Google (login) → Access Token → App uses token → Google API

Access tokens expire. They can be revoked. If an app is compromised, you revoke its token without changing your Google password.

OAuth 2.0 also introduced refresh tokens, which let apps get new access tokens silently without asking the user to log in again.

Pros:

Users never share credentials with third-party apps
Tokens expire and are revocable
Granular scopes: an app gets only the permissions it needs
The gold standard for public APIs

Cons:

More complex to implement than Basic Auth or API keys
Misconfigured redirect URIs are a common attack vector
Token storage on mobile and SPA clients requires care

Real incident: In 2021, a flaw in Facebook's OAuth implementation let attackers use malformed redirect URIs to steal access tokens. Hundreds of millions of accounts were potentially affected. The root cause: insufficient validation of where tokens were sent after authorization.

Use it when: You are building a public API, integrating with third-party services, or giving users the ability to connect their accounts to other apps.

Comparison: Which Method for Which Situation

Method	Security	Complexity	Best For
Basic Auth	Low	Very Low	Internal tools, quick prototypes
API Keys	Medium	Low	Server-to-server, developer APIs
JWT	High	Medium	Stateless services, microservices
OAuth 2.0	Very High	High	Public APIs, third-party integrations

Authentication vs. Authorization: The Distinction That Matters

These two words get used interchangeably. They are not the same.

Authentication answers: who are you?

Your API verifies the client's identity. A valid JWT, a matching API key, a Google OAuth token. If verification passes, the client is authenticated.

Authorization answers: what are you allowed to do?

A regular user on your platform is authenticated. That does not mean they are authorized to delete other users' accounts. Authorization determines what an authenticated identity is permitted to access or modify.

A concrete example:

You log into a hospital system with your nurse credentials. Authentication passed. You are inside. But when you try to access the prescriptions database, the system checks your role. Nurses read records. Only physicians write prescriptions. Authorization denied.

In code terms:

Request arrives → Authentication middleware (who are you?) → Passes
Request continues → Authorization middleware (can you do this?) → Passes or fails

Most security breaches that happen after login are authorization failures, not authentication failures. The attacker is legitimately logged in, then exploits missing or weak permission checks to access data they should not see.

The 2021 Peloton API breach is a clear example. The API was authenticated: requests required valid user tokens. But the authorization layer was broken. Any authenticated user could query any other user's private profile data by changing a user ID in the request. Millions of records were exposed to any logged-in user.

Pros and Cons of API Authentication Overall

Pros:

Prevents unauthorized access to sensitive data
Gives you per-client rate limiting and usage tracking
Creates a complete audit trail: who accessed what and when
Lets you revoke access instantly when credentials are compromised

Cons:

Adds latency to every request (token validation, key lookup)
Poor key management by clients creates vulnerabilities you cannot fully control
More complex auth flows increase surface area for implementation mistakes
Stateful session approaches do not scale without a shared data store

What to Take Away

Basic Auth is the simplest option and the least secure. Use HTTPS. Replace it early.
API Keys are practical for developer-facing APIs. Treat them like passwords. Never put them in client-side code.
JWT gives you stateless, scalable authentication. Always validate the signature. Set short expiry times.
OAuth 2.0 is the right choice for any API that acts on behalf of users or integrates with third parties.
Authentication tells you who the caller is. Authorization tells you what they can do. You need both.

The method you choose shapes how your API scales, how you handle breaches, and how much trust your clients place in your system. Get the foundation right before building on top of it.

Have questions about OAuth flows, JWT security, or RBAC authorization patterns? Drop them in the comments.

DEV Community: M TOQEER ZIA

Horizontal vs. Vertical Scaling: Which One Should You Actually Use?

Vertical Scaling: Make the Machine Bigger

Pros

Cons

Horizontal Scaling: Add More Machines

Pros

Cons

Side-by-Side Comparison

So Which One Should You Pick?

The Real Takeaway

Load Balancing Explained: The Complete Guide to Types and Algorithms

What Is Load Balancing?

Why Load Balancing Matters

How a Load Balancer Works

Types of Load Balancers

1. Hardware Load Balancers

2. Software Load Balancers

3. Cloud-Based / Managed Load Balancers

Load Balancing by OSI Layer

Layer 4 (Transport Layer) Load Balancing

Layer 7 (Application Layer) Load Balancing

Common Load Balancing Algorithms

1. Round Robin

2. Weighted Round Robin

3. Least Connections

4. Weighted Least Connections

5. IP Hash

6. Least Response Time

7. Random / Random with Two Choices

Global Server Load Balancing (GSLB)

Load Balancing vs. Reverse Proxy

Choosing the Right Load Balancer

Final Thoughts

The New HTTP QUERY Method: Why It Exists and What Problem It Actually Solves

HTTP Isn't a Network Protocol — It's an Agreement

Where GET Requests Start to Break Down

The Workaround Everyone Used (and Why It Was Broken)

Enter the QUERY Method

When Should You Actually Use It?

The Bigger Lesson

Building a Microservices App with Kubernetes: What I Learned the Hard Way

Introduction

Table of Contents

1. Introduction to the Project

What is this project?

Why microservices?

Why not a monolithic application?

What problems microservices solve

Architecture overview

2. Understanding Containers

What is Docker?

Why containers exist

Images vs containers

Why Kubernetes needs Docker

How containers communicate

Simple example

3. Kubernetes Basics

What is Kubernetes?

Why Kubernetes exists

What problems Kubernetes solves

Cluster

Node

Master Node or Control Plane

Worker Node

Pod

Container inside a Pod

Replica

Namespace

Service

4. Kubernetes Architecture

How the connection works

A larger picture

5. Deployments

Why Deployments exist

Creating Pods the right way

What each part means

Replica management

Rolling updates

Self healing

`skaffold dev`