DEV Community: Favour Lawrence

Why Your Microservices Should Talk Like Functions, Not URLs (A Practical gRPC Walkthrough in Go)

Favour Lawrence — Fri, 24 Apr 2026 12:14:41 +0000

In most microservice setups, service-to-service communication starts the same way:

GET /api/v1/users/{id}

It works. It’s familiar. It’s easy to debug.

But it forces service-to-service calls into a URL-driven model.

Internal services aren’t browsers. They don’t benefit from clean URLs or REST-style resource modeling. They don’t need JSON payloads designed around human readability. And they don’t need APIs designed around manual testing workflows.

What they need is a strict contract and a predictable call interface.

They need communication that behaves like calling a function:

typed requests and responses
enforced schemas
consistent error semantics
backward-compatible evolution
explicit timeouts and deadlines

With gRPC, your microservices don’t “hit endpoints”. They call methods. You define the interface once using Protocol Buffers, generate strongly typed clients, and treat cross-service communication like a normal function call, except it happens over the network.

In this walkthrough, we’ll build a gRPC service in Go from scratch, implement a client, and cover the production details that actually matter.

The Problem: Why Are Microservices Talking Like Web Browsers?

Say you have two internal services:

billing service
auth-service

billing service needs to charge a user. Before doing that, it needs to validate a few things with auth-service:

does the user exist?
is the user active?
what role does the user have?

A common approach is to expose a REST endpoint from auth-service and call it from billing-service:

resp, err := http.Get("http://auth-service:8080/api/v1/users/123")
if err != nil {
 log.Fatal(err)
}
defer resp.Body.Close()

var user struct {
 UserID string `json:"userId"`
 Active bool   `json:"active"`
 Role   string `json:"role"`
}

if err := json.NewDecoder(resp.Body).Decode(&user); err != nil {
 log.Fatal(err)
}

if !user.Active {
 log.Fatal("user is not active")
}

This works, and REST is a perfectly valid choice for internal communication.

But it comes with tradeoffs that tend to show up as systems grow.

This isn’t a browser fetching a page, it’s one backend service depending on another backend service. Yet REST forces that dependency to be expressed through URLs, HTTP verbs, and JSON payloads. Over time, those implementation details become the de-facto contract between services.

That introduces a few common pain points:

The contract is mostly implicit. The client learns the response shape through documentation and conventions, not enforced types.
Breaking changes are easy to introduce. A renamed JSON field or missing attribute can break consumers at runtime.
Error semantics rely on discipline. A 404 might mean “user not found”, but it can also mean “wrong route”, “bad version”, or “proxy misconfiguration”.
JSON adds overhead. It’s text-based, requires encoding/decoding, and failures often surface at runtime.
Boilerplate spreads everywhere. Every service ends up rewriting HTTP client logic, decoding, validation, and retries.

None of this makes REST “bad”. It just means that for internal service-to-service calls, where you want strict contracts and predictable behavior, REST often starts to feel like the wrong tool for the job.

And that’s usually when teams start looking at gRPC.

REST Inside Microservices Has a Silent Problem: Fake Contracts

The problem isn’t REST itself.

The problem is what REST often turns into inside a microservices environment:

endpoints become “agreements”
JSON becomes “schema”
Slack threads become “documentation”

Unless you enforce schemas and versioning aggressively, the contract between services is mostly social, not technical.

For example, if the auth-service team changes a response from:

{
  "active": true
}

to:

{
  "isActive": true
}

billing-service still compiles. Tests might even pass if they don’t cover that path.

But production breaks.

And that’s the worst kind of failure:

builds fine
deploys fine
fails at runtime

At that point, you’re not relying on a contract; you’re relying on hope.

What If Services Could Talk Like Functions Instead?

Instead of thinking:

“call this URL and parse whatever JSON comes back”

what if billing-service could just do this:

user, err := authClient.GetUser(ctx, &pb.GetUserRequest{UserId: "123"})

That’s not an endpoint. That’s a method call.

And the difference matters:

the request is typed
the response is typed
the contract is defined in one place
both sides generate code from the same definition

That’s the gRPC model.

You stop building internal APIs around URLs and start defining service interfaces the same way you’d define a package in Go: by its functions and the data structures they accept and return.

What gRPC Actually Is

gRPC is a service-to-service communication framework based on RPC (Remote Procedure Calls).

Instead of exposing resources through HTTP routes, a service exposes methods. Another service calls those methods using a generated client.

It’s still a network call. You still deal with latency, timeouts, retries, and failures.

The main difference is that gRPC enforces a defined interface using Protocol Buffers.

What Happens When You Call a gRPC Method?

When billing-service calls:

client.GetUser(ctx, req)

this is what happens:

the request struct is serialized using protobuf
the payload is sent over HTTP/2
the server deserializes the request
the server handler executes
the response is serialized and returned
the client deserializes the response into a typed struct

Both sides use generated code from the same .proto definition. That .proto file is the contract.

Step 1: Define the Contract (auth.proto)

📁 proto/auth.proto;

syntax = "proto3";

package auth;

option go_package = "github.com/example/microservices-grpc/proto/authpb;authpb";

service AuthService {
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
}

message GetUserRequest {
  string user_id = 1;
}

message GetUserResponse {
  string user_id = 1;
  bool active = 2;
  string role = 3;
}

This defines:

the service interface (AuthService)
available RPC methods (GetUser)
request and response message types

Step 2: Generate Go Code

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

Make sure the binaries are in your PATH:

export PATH="$PATH:$(go env GOPATH)/bin"

Now generate the code:

protoc \
  --go_out=. \
  --go-grpc_out=. \
  proto/auth.proto

This generates Go files under:

📁 proto/authpb/

Those files are machine-generated output, not your codebase.

If you ever need to change anything about the API:

edit the .proto file
regenerate the Go code again using protoc

Inside those generated files you get:

request/response structs
the server interface
the client stub

That client stub is what makes gRPC calls feel like function calls.

Step 3: Implement auth-service (Server)

📁 auth-service/main.go;

package main

import (
 "context"
 "log"
 "net"

 "github.com/example/microservices-grpc/proto/authpb"
 "google.golang.org/grpc"
)

type authServer struct {
 authpb.UnimplementedAuthServiceServer
}

func (s *authServer) GetUser(ctx context.Context, req *authpb.GetUserRequest) (*authpb.GetUserResponse, error) {
 log.Printf("GetUser called with user_id=%s", req.UserId)

 // fake DB lookup
 if req.UserId == "123" {
  return &authpb.GetUserResponse{
   UserId: "123",
   Active: true,
   Role:   "premium",
  }, nil
 }

 return &authpb.GetUserResponse{
  UserId: req.UserId,
  Active: false,
  Role:   "unknown",
 }, nil
}

func main() {
 lis, err := net.Listen("tcp", ":50051")
 if err != nil {
  log.Fatalf("listen failed: %v", err)
 }

 srv := grpc.NewServer()
 authpb.RegisterAuthServiceServer(srv, &authServer{})

 log.Println("auth-service listening on :50051")
 if err := srv.Serve(lis); err != nil {
  log.Fatalf("serve failed: %v", err)
 }
}

Step 4: Implement billing-service (Client)

📁 billing-service/main.go;

package main

import (
 "context"
 "log"
 "time"

 "github.com/example/microservices-grpc/proto/authpb"
 "google.golang.org/grpc"
 "google.golang.org/grpc/credentials/insecure"
)

func main() {
 conn, err := grpc.Dial(
  "localhost:50051",
  grpc.WithTransportCredentials(insecure.NewCredentials()),
 )
 if err != nil {
  log.Fatalf("dial failed: %v", err)
 }
 defer conn.Close()

 client := authpb.NewAuthServiceClient(conn)

 ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
 defer cancel()

 resp, err := client.GetUser(ctx, &authpb.GetUserRequest{UserId: "123"})
 if err != nil {
  log.Fatalf("GetUser failed: %v", err)
 }

 log.Printf("User=%s active=%v role=%s", resp.UserId, resp.Active, resp.Role)

 if !resp.Active {
  log.Fatal("user not active, abort billing")
 }

 log.Println("billing can proceed")
}

Note: grpc.WithInsecure() is deprecated. This uses the current supported approach.

Step 5: Run It

At the project root:

📁 go.mod;

module github.com/example/microservices-grpc

go 1.22

require google.golang.org/grpc v1.63.2

Run the services:

Terminal 1:

go run auth-service/main.go

Terminal 2:

go run billing-service/main.go

Expected output:

gRPC Call Flow (Diagram)

Server Streaming

Now let’s extend the example into something that shows where gRPC becomes strictly better than REST for event-style communication.

Say billing-service wants to subscribe to auth-related events like:

user logged in
password changed
account locked

In a REST world, you’d usually end up doing some form of polling:

GET /api/v1/events?since=...

And then you’d run it every few seconds like a caveman with a cron job.

Polling works, but it’s wasteful:

With gRPC, you don’t fake real-time communication.

You just stream.

Defining a Streaming RPC

Update the protobuf contract:

rpc WatchUserEvents(WatchUserEventsRequest) returns (stream UserEvent);

message WatchUserEventsRequest {
  string user_id = 1;
}

message UserEvent {
  string user_id = 1;
  string event_type = 2;
  int64 timestamp = 3;
}

That single keyword stream changes everything.

Instead of “request → response”, the server holds the connection open and pushes events as they occur.

Then regenerate the Go code:

protoc --go_out=. --go-grpc_out=. proto/auth.proto

Now both services share the same contract, and your compiler becomes the enforcer of compatibility.

Implementing Server Streaming in auth-service

Inside auth-service/main.go, implement the streaming method:

func (s *authServer) WatchUserEvents(
 req *authpb.WatchUserEventsRequest,
 stream authpb.AuthService_WatchUserEventsServer,
) error {
 log.Printf("WatchUserEvents started for user_id=%s", req.UserId)

 events := []string{"LOGIN", "PASSWORD_CHANGED", "ACCOUNT_LOCKED"}

 for _, e := range events {
  resp := &authpb.UserEvent{
   UserId:    req.UserId,
   EventType: e,
   Timestamp: time.Now().Unix(),
  }

  if err := stream.Send(resp); err != nil {
   return err
  }

  time.Sleep(2 * time.Second)
 }

 return nil
}

Don’t forget:

import "time"

This example is simplified (we’re just emitting fake events), but the shape is realistic.

In production, this loop would usually be backed by something like:

a Kafka consumer
Redis pub/sub
a database WAL stream
an internal event bus

The key idea stays the same: the server pushes messages as they happen.

Consuming the Stream in billing-service

On the client side, you call the RPC once and then continuously receive messages:

stream, err := client.WatchUserEvents(ctx, &authpb.WatchUserEventsRequest{
 UserId: "123",
})
if err != nil {
 log.Fatalf("WatchUserEvents failed: %v", err)
}

for {
 event, err := stream.Recv()
 if err != nil {
  log.Println("stream ended:", err)
  break
 }

 log.Printf("EVENT: %s at %d", event.EventType, event.Timestamp)
}

This is what “real-time service communication” actually looks like in clean engineering terms:

one connection
one contract
structured messages
backpressure handled by the transport
no polling loops

This is gRPC solving a real system problem in the most direct way possible.

Final Thoughts

At the end of the day, gRPC just solves a different problem.

If you’re building service-to-service communication, you quickly realize URLs and JSON start feeling like a workaround. You’re passing strings around, hoping everybody remembers the exact response shape, and most breakages only show up at runtime. With gRPC, the .proto file becomes the source of truth, your types are enforced, and calling another service feels like calling a real method, because the client stub is literally generated for that.

That said, gRPC isn’t always the smoothest experience everywhere. Debugging isn’t as simple as running curl and reading JSON. Most times you’ll use grpcurl, Postman, or enable reflection just to inspect and test things quickly. Also, browsers don’t speak gRPC natively, so if your consumers are frontend clients, you’ll probably keep REST at the edge or introduce gRPC-Web / a gateway.

And you still need discipline when evolving schemas. Protobuf makes it easier, but you can’t just reuse field numbers or delete fields carelessly without breaking older clients.

So the rule is pretty simple: if it’s internal microservices talking to each other, gRPC feels natural. If it’s a public API meant for browsers and humans, REST still makes sense.

Thanks for reading.

What DevOps Really Means (and Why It’s So Hard to Explain)

Favour Lawrence — Mon, 20 Apr 2026 09:49:28 +0000

You know how people still, even now, don’t really know what DevOps is? Honestly, even most DevOps engineers struggle to explain what they actually do. So, to make life easier, we just say “we’re software engineers”. I do it too it’s funny, really. We are software engineers, sure, but saying “DevOps engineer” opens the door to a conversation I’m never ready for.

I already stress enough trying to explain what I studied, “biomedical engineering” and now I’m in DevOps? Yeah, no thanks. I just say “software engineering.” Much simpler.

So one day, I called up a friend who’s also a DevOps engineer and asked him, “Hey, what exactly is DevOps engineering?” He paused and went, “Umm… it’s developer operations.” I said “okay,” waiting for more. Then he added, “You know, we automate things… CI/CD pipelines, containers, all that stuff.” I asked three other DevOps folks the same question got pretty much the same answer. These are senior people, by the way. But somehow, explaining what they do is still a challenge.

What do they say about engineers again? Hehehe.

“Engineers solve problems no one else understands, then fail to explain what they just did in plain English.”

So yeah, I started thinking, how do we actually break this down? This isn’t meant to be a long article, just a short one. I’m just here to tell you what we, the DevOps engineers, actually do. Here to explain what DevOps really is.

First off, this whole piece was inspired by an interesting definition I came across in Samuel’s article titled “Principles in Striking the Balance for DevOps.” He described DevOps as;

“a name given to a culture designed to increase an organization’s ability to deliver applications and services faster than traditional software development processes.”

Pretty solid definition, right? But then I paused and thought okay, what exactly was the traditional software development process? And are we saying that it’s no longer in use?

So what’s the big picture with DevOps then? Is it a replacement? A mindset? A fancy label for what engineers were already doing, but now with more YAML files? That’s what I wanted to unpack in plain, human language.

Back then, things were… let’s say, “well-structured” which is just a polite way of saying slow. Developers stayed in their corner writing code, tossing it over to the operations team and saying, “It works on my machine.” The ops folks would then spend nights trying to figure out why it doesn’t work on the actual server.

It was a clear divide; devs built, ops deployed, and whenever things broke (which they did, often), both sides pointed fingers until someone fixed it or gave up. It wasn’t that people were lazy or careless; the process itself just didn’t support speed or collaboration. Each team had its own tools, goals, and sometimes even vocabulary.

Deployments happened maybe once every few weeks or months. If something went wrong, rolling back meant pain, panic, and debugging marathons at 2 a.m. It worked, sort of but it wasn’t sustainable, especially as software grew more complex and user expectations skyrocketed.

That’s where DevOps came in not as a job title (though that’s what it’s become), but as a mindset. It’s about bridging that gap between developers and operations, breaking the silos, and automating the stuff that used to drain everyone’s sanity.

Now DevOps isn’t just a trendy term or a job title, it’s a way of working that brings developers and operations teams together. It is about collaboration and automation, with the goal of making software development, testing, and deployment more efficient and reliable.

Instead of handing code off between teams, DevOps promotes shared ownership through automated pipelines that everyone can depend on. Concepts like CI/CD, containers, infrastructure as code, and monitoring all support this approach. The idea is simple: move from “it works on my machine” to “it works everywhere.”

In practice, DevOps means automating repetitive tasks, testing early and frequently, deploying in small, manageable updates, and continuously monitoring systems in production. These practices lead to faster releases, fewer issues, and a more stable environment.

But beyond the tools and processes, DevOps is ultimately about culture. It’s built on trust, accountability, and collaboration. The mindset shifts from “developers versus operations” to “we build it, we run it.” When something goes wrong and it will teams work together to resolve it.

In the end, DevOps isn’t defined by a specific set of tools or tasks. It’s defined by how teams work: communicating openly, automating where it makes sense, and focusing on delivering reliable software.

So, if someone asks what DevOps is, the simplest answer might be: it’s about making sure the software we build actually works everywhere.

I’d love to hear your thoughts on DevOps, if you found this explanation useful, consider giving it a like and a follow for more content like this.

Introduction to OpenTelemetry: A DevOps Beginner's Guide

Favour Lawrence — Sat, 03 May 2025 22:56:52 +0000

As a DevOps engineer, understanding how your applications are performing in production is crucial to ensuring their reliability and stability. Whether you’re working with microservices, containers, or cloud-native environments, monitoring and observability play a vital role. This is where OpenTelemetry comes in.
In this article, we’ll go over what OpenTelemetry is, why it’s important for your DevOps practices, and introduce some key concepts to help you get started on the path to improved observability.

How OpenTelemetry Works

OpenTelemetry is a collection of APIs, libraries, agents, and instrumentation tools designed to help you collect, process, and export telemetry data things like traces, metrics, and logs from your applications. Essentially, it offers a standardized way to capture data, providing you with a clear view of your system’s performance and behavior.

This data is critical for understanding how your system is functioning, pinpointing issues, and optimizing overall performance. But how does OpenTelemetry gather all this useful information, and how can you use it to improve your system? Let’s break it down step-by-step.

1. Instrumentation: Making Your App Observable:

The first step is instrumentation. Basically, you’re wiring your application to be “observed.” This is where you decide how to generate signals (traces, metrics, and logs) from your code.

Now, depending on your setup, you can either go with any below;

Manually instrument: This means adding SDK calls in your code like creating spans around functions, or timing a DB query.
Automatically instrument: OpenTelemetry offers auto-instrumentation for many popular frameworks (like Express for Node.js, Spring Boot for Java, Flask for Python). This way, telemetry is collected without you writing much extra code.

Once your application is instrumented, you’ll have the data needed to analyze performance, identify issues, and understand system behavior in real time.

2. Signal Generation: Traces, Metrics, and Logs

Now your application is instrumented, it starts generating signals:

Traces track the path of a request as it moves through your system. They’re made up of spans, which capture operations like HTTP calls, database queries, or interactions with external services. Useful for pinpointing bottlenecks or latency issues.
Metrics are numerical values you can aggregate, graph, and alert on, like request rates, response times, CPU and memory usage, error counts, etc. Ideal for monitoring trends and setting SLOs.
Logs are timestamped text entries that capture specific events or states. They’re often used for debugging or to provide extra context alongside traces and metrics.

These signals are structured in a consistent format, usually using the OpenTelemetry Protocol (OTLP), so they can be processed and exported to your observability backend without friction.

3. Context Propagation: Keeping Traces Connected Across Services

In a distributed system, especially with microservices, tracking a single request across multiple services can get messy fast. That’s where context propagation comes in.

OpenTelemetry handles this by passing trace context things like trace IDs and span IDs along with each request. For HTTP, this happens via headers; for gRPC, it’s in the metadata. Each service picks up that context and attaches it to its own spans.

The result: when you view a trace later, you get a complete picture of the request’s path through the system, without any gaps. This is what makes distributed tracing actually usable.

4. The Collector: Centralized Ingestion and Processing

When your app starts producing telemetry data, you need a way to handle it. That’s where the OpenTelemetry Collector comes in.
Think of it as a central point that:

Ingests traces, metrics, and logs from instrumented services
Processes the data, things like filtering, redacting, or adding metadata
Exports it to your monitoring or observability backend

By separating data collection from where the data ends up, the Collector gives you a clean layer of control. You can standardize telemetry pipelines, apply consistent transformations, and avoid tight coupling with any single vendor.

5. Exporting: Routing Telemetry to the Right Tools

After the Collector processes your telemetry, it sends it off to the systems you use to monitor and troubleshoot.

Metrics → Prometheus, Cloud Monitoring, etc.
Traces → Jaeger, Zipkin, or other tracing backends
Logs → Log aggregation tools or cloud logging platforms

The best part? You don’t need to modify your application every time you want to switch or add a new backend. Just update the Collector’s config and you're good to go.

What Makes OpenTelemetry Different?

So maybe you’re thinking, “Why bother with OpenTelemetry? I already use Prometheus for metrics or Jaeger for tracing isn’t that enough?”

Fair question. But here’s the thing: OpenTelemetry isn’t here to replace those tools it’s here to bring them together.

- Unified Telemetry Framework: Instead of juggling separate tooling for traces, metrics, and logs, OpenTelemetry gives you a single, consistent way to collect all three. Same concepts, same configuration patterns, less cognitive load for your team.

- Vendor-Neutral by Design:
OpenTelemetry is vendor-neutral by design. You can pipe your data to Prometheus today, switch to Datadog next week, or go full open-source with Jaeger and Loki and your application code stays exactly the same. No proprietary agents. No rewriting instrumentation. You own the pipeline, not the vendor.

- Pluggable and Extensible Pipeline:
The Collector acts as a configurable processing layer. Need to drop certain spans, mask sensitive fields, or add metadata before sending data out? Just plug in the processors you need no custom tooling required.

- Community-Driven and CNCF-Backed
OpenTelemetry isn’t a startup’s side project. It’s part of the Cloud Native Computing Foundation (same folks behind Kubernetes), and it’s backed by a huge community of engineers and observability vendors. That means fast updates, strong documentation, and wide adoption.

In summary OpenTelemetry doesn’t replace Prometheus or Jaeger, it complements and unifies them. It gives you a consistent, flexible foundation for observability, no matter what your stack looks like.

Thanks for reading. Let me know in the comments if there’s a topic you’d like me to cover next. I know it’s been a while since I last posted. I’m still working on those practical, hands-on articles I promised, so keep an eye out. They’re on the way.

Squash, Rebase, Merge: Keeping Your CI/CD Pipelines Clean and Efficient 🚀

Favour Lawrence — Wed, 26 Mar 2025 05:05:28 +0000

In DevOps, efficiency is everything. A messy Git history slows down pipelines, causes unnecessary conflicts, and makes debugging harder.

When Git workflows are unmanaged, CI/CD pipelines can stall due to conflicting merge commits, and developers waste time digging through cluttered commit histories.

But with a clean Git workflow:
✅ Faster builds – No extra history slowing things down
✅ Fewer merge conflicts – Smoother collaboration, less frustration
✅ Clearer logs – Easier debugging and rollbacks

A structured Git history isn’t just about keeping things tidy, it directly improves pipeline speed, code quality, and developer productivity. Using squashing, rebasing, and merging correctly keeps your CI/CD pipeline fast, reliable, and hassle-free.

Squashing, Rebasing, and Merging – The Right Tool for the Job

Git is a powerful version control system, but how you manage your commits can either streamline or slow down your CI/CD pipeline. A cluttered history with unnecessary commits leads to:

❌ Slower builds due to excessive commit processing

❌ Merge conflicts that could have been avoided

❌ Hard-to-follow commit logs, making debugging difficult

To keep your workflow clean and efficient, you need to use the right Git strategy at the right time. Squashing, rebasing, and merging each serve a unique purpose. Here’s how they work and when to use them.

When to Squash: Keeping PRs Clean

What is Squashing?

Squashing combines multiple commits into a single commit. This is useful when a pull request (PR) contains many small, incremental commits that don’t need to be preserved individually. Instead of polluting the Git history with minor fixes and adjustments, you merge everything into one meaningful commit before merging into the main branch.

Why Use Squash?

🔹 Keeps commit history clean – Instead of “Fixed bug,” “Fixed typo,” and “Final final fix,” you get a single commit with a meaningful message.

🔹 Reduces clutter – A clean commit history makes it easier to review and debug.

🔹 Best for feature branches – Squash commits before merging into main or develop to keep the history readable.

Example Scenario

You’re working on a new login feature. During development, you make multiple commits:

commit 1: Added login form  
commit 2: Fixed validation error  
commit 3: Adjusted button alignment  
commit 4: Updated error messages  
commit 5: Finalized login feature

Instead of merging all five commits into main, squash them into one:

commit 1: Implemented login feature with validation and UI fixes

How to Squash Commits

Interactive Rebase (Local Changes)

git rebase -i HEAD~n  # 'n' is the number of commits to squash

This opens an interactive editor where you replace "pick" with "squash" for the commits you want to combine.

Squash on GitHub (PR Merging)

When merging a PR, select "Squash & Merge" to combine all commits into one before merging.

When to Rebase: Keeping History Linear

What is Rebasing?

Rebasing rewrites history by moving your branch’s commits on top of the latest main branch, as if your feature branch was built on the most up-to-date code. It prevents unnecessary merge commits and maintains a linear commit history.

Why Use Rebase?

🔹 Keeps history clean – Avoids merge commits like Merge branch 'main' into feature-xyz.

🔹 Ensures a smooth merge – Applying your changes on top of the latest main helps prevent conflicts later.

🔹 Best for feature branches – Before merging a feature, rebase it onto main to ensure it integrates smoothly.

Example Scenario

You started working on a new checkout feature based on an older version of main. Meanwhile, other developers pushed updates to main. Now, your branch is behind:

* (main) commit A - Updated payment logic  
* (main) commit B - Fixed checkout bug  
|
|--- (feature-checkout) commit C - Added checkout form  
|--- (feature-checkout) commit D - Implemented discount logic

Instead of merging main into your branch (which would create an unnecessary merge commit), rebase it:

git fetch origin
git rebase origin/main  # Moves your commits on top of the latest main

After rebasing, your history is clean and linear:

* (main) commit A - Updated payment logic  
* (main) commit B - Fixed checkout bug  
* (feature-checkout) commit C - Added checkout form  
* (feature-checkout) commit D - Implemented discount logic

Handling Conflicts During Rebase

If there are conflicts, Git will stop and ask you to resolve them. Once fixed, continue the rebase:

git add .
git rebase --continue

When to Merge: Preserving Full History

What is Merging?

Merging combines one branch into another, preserving all commits and commit messages exactly as they were made. This keeps a detailed commit history but may introduce merge commits that clutter the log.

Why Use Merge?

🔹 Preserves full commit history – Useful for tracking all incremental changes.

🔹 Best for team collaborations – When multiple developers contribute to a feature branch, merging keeps individual contributions visible.

🔹 Used in Gitflow workflows – Commonly used for merging develop → main.

Example Scenario

Your team worked on a new analytics dashboard in a shared branch. The commit history looks like this:

commit X: Setup analytics API  
commit Y: Implemented dashboard UI  
commit Z: Added data visualization

Since multiple developers contributed, merging into main without squashing retains authorship and commit details.

How to Merge Properly

Basic Merge (Fast-Forward)

git merge feature-branch  # Merges feature branch into current branch

If your branch is ahead of main, Git will fast-forward (move the branch pointer) without creating a merge commit.

Preserving a Merge Commit
To keep a merge commit for tracking:

git merge --no-ff feature-branch  # Creates a merge commit even if a fast-forward is possible

Which One Should You Use?

✔️ Squash → When cleaning up a messy PR with too many small commits.

✔️ Rebase → When syncing with main before merging to avoid extra merge commits.

✔️ Merge → When you want to preserve full commit history, especially in team collaborations.

Git Strategies for DevOps Teams

Choosing the right Git strategy is essential for maintaining an efficient CI/CD workflow. A poorly structured process can cause bottlenecks, increase merge conflicts, and slow down deployments. On the other hand, a well-defined strategy keeps the development pipeline smooth, ensuring faster releases and better collaboration.

There are two primary approaches teams use: Feature Branch Workflow and Trunk-Based Development (TBD). Each has its place, and automation plays a key role in enforcing these workflows.

Feature Branch Workflow

This is the traditional approach where developers create separate branches for each feature, bug fix, or enhancement. The branch remains isolated from the main branch until the work is complete and ready for review. Merging happens through pull requests (PRs), allowing for code reviews before the changes are integrated.

This workflow is often used in structured development models like Gitflow, where teams work on long-running feature branches before merging into a staging or main branch. It provides stability and makes it easier to review code, but if branches are kept open for too long, they can diverge significantly from the main branch, leading to complex merge conflicts.

To prevent this, developers should rebase frequently, ensuring their feature branch stays in sync with the latest changes from the main branch. Automated tests and linters should be triggered on every PR to catch potential issues early. Before merging, it’s good practice to squash commits into a single, meaningful commit to keep the Git history clean.

Trunk-Based Development (TBD)

Unlike the feature branch workflow, Trunk-Based Development encourages short-lived branches or even direct commits to main. Instead of working in isolation for extended periods, developers integrate changes continuously—sometimes multiple times a day. This approach ensures that the main branch is always in a deployable state and minimizes the complexity of long-lived branches.

Frequent integration reduces merge conflicts and makes debugging easier since each change set is small and easier to track. However, this strategy requires strict CI/CD enforcement to prevent unstable code from being deployed. Automated testing and static analysis tools must be in place to verify every commit before it reaches production.

Since features may be merged incrementally, feature flags are often used to hide unfinished work while allowing continuous integration. This allows developers to merge work early without exposing incomplete features to end users.

Rolling back changes in Trunk-Based Development should be handled with git revert instead of git reset to maintain a clear history of what was changed and why.

Choosing the Right Strategy

The best approach depends on the team’s structure and release process. Feature Branch Workflow works well for teams that need structured releases, code reviews, and stability. Trunk-Based Development is better suited for high-velocity DevOps teams where frequent deployments and quick iteration cycles are necessary.

Some teams adopt a hybrid model—using feature branches for significant changes while following Trunk-Based Development for smaller, incremental updates. Regardless of the approach, automation is key to maintaining a clean workflow.

Enforcing Git Workflows with CI/CD Automation

Manual enforcement of Git best practices is not scalable. Teams must automate workflow rules to ensure consistency and reduce human errors.

Branch protection rules should prevent direct commits to main in feature branch workflows.
Pre-commit hooks can enforce commit message formats and prevent invalid commits.
PR approvals and CI/CD checks should be mandatory before merging changes.

Automating Git Workflow Enforcement with GitHub Actions

GitHub Actions can be used to enforce rules such as requiring squashed commits and ensuring branches are rebased before merging.

jobs:
  enforce-workflow:
    runs-on: ubuntu-latest
    steps:
      - name: Ensure commits are squashed
        run: git log --format="%s" origin/main..HEAD | wc -l | grep -q '^1$' || exit 1

      - name: Prevent merging without rebase
        run: git merge-base --is-ancestor main HEAD || exit 1

This automation ensures that multiple commits are squashed before merging and that branches are properly rebased.

Beyond enforcing Git hygiene, integrating automated code quality checks with tools like SonarQube, ESLint, or Prettier ensures that coding standards are followed. Requiring all tests to pass before merging prevents broken changes from entering production.

A well-structured Git workflow is essential for maintaining clean CI/CD pipelines, reducing conflicts, and ensuring efficient collaboration. Whether your team follows a Feature Branch Workflow for stability or Trunk-Based Development for faster iterations, enforcing best practices through automation is key to keeping development smooth and predictable.

By using squashing, rebasing, and merging correctly, along with CI/CD automation, teams can improve code quality, streamline deployments, and eliminate unnecessary complexity in their repositories.

Thanks for reading! If you found this helpful, follow me for more DevOps concepts and best practices. If there’s a topic you’d like me to break down next, let me know! 🚀

ELK Stack Explained: How Elasticsearch, Logstash & Kibana Work Together for Real-Time Data Insights.

Favour Lawrence — Mon, 24 Mar 2025 11:38:17 +0000

Introduction to ELK Stack

Logs are everywhere in every app, server, and system generates them. But when something goes wrong, digging through endless log files to find the issue can be so overwhelming. Here ELK turns raw log data into clear, searchable, and visual insights.

What is the ELK Stack?
The ELK Stack is an open-source log management and data analytics tool made up of:

Elasticsearch – A search engine that stores and retrieves log data quickly.

Logstash – A tool that collects, processes, and forwards logs.

Kibana – A dashboard for visualizing and analyzing log data.

Together, these tools make it easy to collect, search, and analyze logs in real time, helping teams troubleshoot issues, monitor systems, and make data-driven decisions.

Why is ELK Important?
Modern applications generate tons of log data, and manually searching through it isn’t practical. ELK helps by:
✔️ Finding issues fast – Instantly search massive log files.
✔️ Handling large data – Works across multiple servers and systems.
✔️ Turning data into insights – Creates real-time dashboards for monitoring and decision-making.

Elasticsearch vs. Traditional RDBMS: A Developer’s Perspective

If you're used to working with relational databases like MySQL or PostgreSQL, switching to Elasticsearch might feel like stepping into a whole new world. But Elasticsearch is just another way to store, retrieve, and search data—the difference is in how it’s structured and optimized.

Instead of tables and rows, Elasticsearch works with documents and indices. Let’s break it down using concepts you already know.

Thinking in Tables vs. Thinking in Documents

RDBMS Concept	Elasticsearch Equivalent
Database	Index
Table	Type (deprecated in newer versions)
Row	Document
Column	Field
Schema	Mapping

In relational databases, data is neatly organized into tables with predefined schemas—every row must follow a fixed structure.

Elasticsearch, on the other hand, is schema-less (to an extent). Instead of rows, it stores JSON documents inside an index (similar to a table). Each document can have a flexible structure, making it great for semi-structured or dynamic data.

How Data is Stored: Rows vs. JSON Documents

🔹 RDBMS Example (Users Table in MySQL)

Here’s how you’d define a simple users table in MySQL:

CREATE TABLE users (
    id INT PRIMARY KEY,
    name VARCHAR(100),
    email VARCHAR(100),
    age INT
);

🔹 Elasticsearch Equivalent (JSON Document in an Index)

In Elasticsearch, each user record is stored as a JSON document inside an index:

json
{
    "id": 1,
    "name": "Jane Doe",
    "email": "jane.doe@example.com",
    "age": 30
}

Instead of inserting data as rows, Elasticsearch stores each entry as a self-contained JSON document. This structure allows for fast searching and flexible querying without requiring rigid table schemas.

Querying: SQL vs. Elasticsearch Query DSL

One of the biggest differences between RDBMS and Elasticsearch is how you search for data.

Traditional databases use SQL, while Elasticsearch has its own Query DSL (Domain-Specific Language), which is JSON-based.

🔹 Finding all users aged 30 in MySQL (SQL Query)

SELECT * FROM users WHERE age = 30;

🔹 Finding all users aged 30 in Elasticsearch (Query DSL)

json
{
    "query": {
        "match": {
            "age": 30
        }
    }
}

Elasticsearch

Fast Search & Distributed Indexing

At the core of the ELK Stack is Elasticsearch, a highly scalable, distributed search engine that enables rapid data access retrieval. Unlike traditional databases that are optimized for structured data and transactions, Elasticsearch is designed for:

Full-text search – Finds relevant results instantly, even in massive datasets.
Real-time indexing – New data becomes searchable almost immediately.
Scalability – Distributes data across multiple nodes to handle petabytes of information.

It’s built on Apache Lucene, a powerful search library, and uses an inverted index, a structure specifically optimized for search queries.

How Elasticsearch Works

1️⃣ Indices & Documents – The Building Blocks

Elasticsearch doesn’t use tables and rows like a relational database. Instead, it stores data as JSON documents inside an index.

🔹 Think of an index like a database, and each document inside it as a record. Unlike relational databases, these documents can have different structures—offering flexibility for handling dynamic or semi-structured data.

2️⃣ Shards & Replicas – How Elasticsearch Scales

Handling massive amounts of data requires scalability, and that’s where sharding and replication come in.

Shards: Elasticsearch splits an index into smaller pieces (shards) to distribute data across multiple nodes.
Replicas: Each shard can have replicas—copies stored across different nodes to improve redundancy and performance.

This architecture makes Elasticsearch both fault-tolerant and lightning fast, even when dealing with billions of records.

Why Elasticsearch is Powerful for Log Analysis

Imagine you’re managing a cloud-based web application that logs thousands of events every second. A typical log entry might look like this:

{
    "timestamp": "2025-03-24T12:34:56Z",
    "level": "ERROR",
    "message": "Database connection failed",
    "service": "authentication",
    "user_id": 1234
}

Once indexed in Elasticsearch, you can instantly search for all error messages related to the authentication service:

{
    "query": {
        "match": {
            "service": "authentication"
        }
    }
}

This makes Elasticsearch incredibly powerful for log analysis, large-scale search applications, and real-time data insights.

Logstash – The Data Pipeline

Collecting, Transforming, and Shipping Data

While Elasticsearch is great for searching and analyzing data, it doesn’t collect or process data on its own. That’s where Logstash comes in.

Logstash acts as a data pipeline that:

1️⃣ Collects data from multiple sources (logs, databases, cloud services).

2️⃣ Transforms it into a structured format (parsing, filtering, masking sensitive data).

3️⃣ Sends it to Elasticsearch (or other destinations like Kafka).

How Logstash Works

Logstash follows a simple ETL (Extract, Transform, Load) workflow.

1️⃣ Input – Collecting Data

Logstash gathers logs from multiple sources:

✅ Files – System logs, application logs, web server logs.

✅ Databases – MySQL, PostgreSQL, MongoDB.

✅ Cloud Services – AWS CloudWatch, Google Cloud Logs.

Example: Collecting Logs from a File

input {
  file {
    path => "/var/log/syslog"
    start_position => "beginning"
  }
}

2️⃣ Filter – Transforming & Enriching Data

Before sending data to Elasticsearch, Logstash can clean, modify, and enrich logs.

✅ Parse JSON logs for better searchability.

✅ Mask sensitive data like passwords.

✅ Geo-location enrichment (find a user’s country based on IP).

Example: Masking Passwords in Logs

filter {
  json {
    source => "message"
  }
  mutate {
    gsub => ["password", ".*", "[REDACTED]"]
  }
}

3️⃣ Output – Sending Data to Elasticsearch

After processing, Logstash ships logs to Elasticsearch.

Example: Indexing logs in Elasticsearch

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "logs-%{+YYYY.MM.dd}"
  }
}

This automates log ingestion and ensures that logs are structured, searchable, and ready for analysis in Kibana.

Kibana – Bringing Data to Life

Dashboards, Analytics, and Insights

Now that our logs are in Elasticsearch, how do we make sense of all this data? Kibana makes it visual and interactive.

Kibana is a dashboard and analytics tool that allows you to:

✅ Monitor logs and metrics in real-time.

✅ Run searches and filter data with ease.

✅ Set up alerts for anomalies or critical issues.

Key Features of Kibana

1️⃣ Dashboards & Visualizations

Kibana lets you build custom dashboards using bar charts, line graphs, pie charts, and heatmaps.

See server performance trends over time.
Track error rates in real time.
Visualize traffic spikes on your website.

2️⃣ Discover & Search

Kibana’s search interface helps drill down into logs.

For example, you can filter logs to show only:

✔️ ERROR messages from a specific service

✔️ API requests made by a certain user

✔️ Security alerts from a particular IP range

3️⃣ Spotting Patterns & Trends with Kibana
Kibana makes it easy to spot patterns in your data over time. With simple tools like Timelion and Lens, you can:

See sudden jumps in website visitors and understand why.

Connect system crashes to specific events to troubleshoot faster.

Identify trends in user activity to improve your services.

Conclusion

The ELK Stack—Elasticsearch, Logstash, and Kibana, turns raw logs into searchable, visual insights for better monitoring and decision-making. Elasticsearch handles fast searches, Logstash collects and processes data, and Kibana brings it to life with dashboards. Together, they power real-time analytics for DevOps, security, and business intelligence.

Next, we’ll deploy ELK on AWS, covering setup, scaling, and optimization. Stay tuned for the hands-on guide! 🚀

🚀 InfluxDB Architecture: A Beginner’s Guide for DevOps Engineers

Favour Lawrence — Sun, 02 Mar 2025 23:24:59 +0000

Shoutout to @madhurima_rawat for requesting a deep dive into InfluxDB architecture after reading my Prometheus and Grafana breakdown! 🚀

If you're a DevOps engineer trying to understand how InfluxDB stacks up against Prometheus, you’re in the right place. We’ll keep it simple, comparing their architectures side by side, so you know when to use which tool.

Also, if there's a DevOps concept you'd love me to explain next, drop a comment. I just might write about it next! 😉

Now, let’s dive in! 🔥

The Core of InfluxDB Architecture

InfluxDB is a high-performance time-series database built to handle massive amounts of timestamped data efficiently. It is widely used for monitoring, real-time analytics, and IoT applications.

InfluxDB is designed around four key components:

Storage Engine (TSM & TSI)
Data Ingestion & Retention Policies
Query Engine (InfluxQL & Flux)
High Availability & Scaling Let’s break these down one by one.

Storage Engine: TSM & TSI

Time-Structured Merge Tree (TSM) – The Heart of InfluxDB Storage

InfluxDB uses a TSM (Time-Structured Merge Tree) engine, which is optimized for:
✅ Efficient writes – It writes new data to an in-memory cache and periodically flushes it to disk in compact, immutable files.
✅ High compression – TSM stores data in compressed segments, reducing storage costs.
✅ Fast reads – TSM is optimized for quick lookups, even in large datasets.
🔹 How is this different from Prometheus?
Prometheus chunks data into 2-hour blocks and doesn’t have built-in long-term retention. InfluxDB’s TSM engine allows for more efficient long-term storage and querying.

Time-Series Index (TSI) – Handling Millions of Tags

InfluxDB also introduces TSI (Time-Series Indexing), which is crucial when dealing with millions of time-series labels.

✅ Fast queries on large datasets – Unlike databases that slow down with too many unique tags, TSI ensures smooth performance.
✅ Disk-based indexing – TSI allows InfluxDB to scale efficiently without consuming too much RAM.

🔹 Why does this matter?
One of the common pitfalls in Prometheus is high cardinality issues, when you have too many labels, queries become slow. InfluxDB handles high cardinality better thanks to TSI.

Data Ingestion & Retention Policies

Push-Based Data Collection

InfluxDB primarily relies on a push-based model for ingesting data. This means data sources send metrics to InfluxDB rather than InfluxDB pulling them.
✅ Telegraf – InfluxDB’s official data collection agent, supporting 300+ integrations.
✅ Direct HTTP API writes – Developers can push metrics to InfluxDB using simple REST API calls.

🔹If you recall from my prometheus article
Prometheus scrapes metrics (pull-based). Here InfluxDB expects data to be pushed.

Retention Policies (RP) & Continuous Queries (CQ)

RP Allows you to automatically delete old data after a set period. Great for managing storage costs.
CQ Helps precompute and aggregate data in real-time, reducing query load.
🔹Unlike Prometheus, where you need external tools (like Thanos) for retention, InfluxDB manages data lifecycle natively.

High Availability & Scaling

Scaling InfluxDB
InfluxDB supports horizontal scaling via:
✅ Clustering (Enterprise Edition) – Distributes data across multiple nodes.
✅ InfluxDB Cloud – Fully managed, scalable version.

🔹 How is this different from Prometheus?
Prometheus doesn’t natively support clustering—you need Thanos or Cortex for that. InfluxDB offers built-in clustering.

When Should You Use InfluxDB?

✅ Best for long-term storage & analytics – If you need to keep metrics for months/years.
✅ Great for IoT, sensors, and business analytics – Ideal for financial, industrial, and IoT applications.
✅ Advanced query capabilities – If you need complex joins, transformations, and external API integrations.

Conclusion

InfluxDB is a powerful time-series database designed for high-performance storage, flexible querying, and efficient scaling. Compared to Prometheus, it’s better suited for long-term retention, high-cardinality data, and deep analytics.

🚀 Want to see InfluxDB in action? Let me know in the comments if you’d like a hands-on tutorial or use case examples! 🔥

Thanks for reading! Don’t forget to follow, and feel free to leave a comment with the next DevOps concept you’d like me to dive into. Let’s keep the learning going!

Grafana Architecture Explained: How the Backend and Data Flow Work.

Favour Lawrence — Thu, 20 Feb 2025 14:37:16 +0000

Grafana is a powerful open-source tool that helps turn raw data into clear, interactive dashboards making it a go-to for DevOps teams. But what’s really happening behind the scenes? In this article, we’ll break down how Grafana processes and visualizes data, keeping things simple, practical, and to the point. Whether you’re new to DevOps or just curious about how it all works under the hood, this guide will give you a solid starting point.

Grafana at a Glance

Grafana is built on two main parts: the frontend and the backend.

Frontend: This is the part you see and interact with the dashboards, graphs, and visualizations. Built with modern web technologies, it ensures a smooth and responsive experience, making it easy to explore and analyze your data.
Backend: This is where the heavy lifting happens. The backend processes data, runs queries, and connects to various data sources like Prometheus and InfluxDB. In short, it gathers and prepares the data that the frontend turns into useful insights.

🔍 The Backend

Grafana’s backend is where most of the activities happen, handling data requests, processing queries, and keeping everything running smoothly. Let’s break it down into two key parts:

⚙️ The Grafana Server & API Layer

The Grafana server is the engine running behind the scenes. It acts as the bridge between your dashboards and your data sources, ensuring seamless communication. Here’s what it does:

🌍 Manages Requests: When you interact with Grafana, the server processes your actions, whether it’s loading a dashboard, changing a time range, or modifying settings.
🔌 Connects to Data Sources: Through its RESTful APIs, the server fetches data from sources like Prometheus, InfluxDB, or MySQL.
🔄 Enables Automation: Beyond the web interface, the API lets you integrate Grafana into scripts and automation workflows, making it a flexible tool for DevOps teams.

📊 How Data Queries & Processing Work

Every time you load a dashboard, Grafana works behind the scenes to fetch and process data. Here’s a step-by-step breakdown:

1️⃣ Request Initiation: The frontend (your dashboard) sends a query request to the backend via the API.

2️⃣ Data Retrieval: The backend translates this request and reaches out to the right data source.

3️⃣ Processing: Once the data is retrieved, the server processes it applying filters, aggregations, or calculations as needed.

4️⃣ Response & Rendering: The processed data is sent back to the frontend, where it’s transformed into the visualizations you see.

This smooth backend operation is what makes Grafana such a powerful tool for real-time monitoring and analysis.

🔗 Connecting to Data Sources

Grafana is like a universal translator for data, it seamlessly connects to a wide range of sources, from time series databases like Prometheus and InfluxDB to search engines like Elasticsearch. Whether you're monitoring server metrics, analyzing logs, or tracking application performance, Grafana knows how to fetch and display the data you need.

🛠 Setting Up a Data Source

Connecting a data source in Grafana is a straightforward process:

1️⃣ Pick Your Source: In Grafana’s intuitive UI, you select the database or service you want to connect to.

2️⃣ Choose the Right Plugin: Grafana has built-in plugins that "speak" the native query language of each data source, ensuring seamless communication.

3️⃣ Configure & Authenticate: You provide connection details like the database URL, credentials, and any necessary authentication tokens.

4️⃣ Test & Save: Grafana lets you test the connection before saving, so you can ensure everything is working smoothly.

Once set up, Grafana sends queries directly to your data source in real time, pulling in the latest metrics for visualization.

🔄 Understanding Data Flow

Every time you interact with a Grafana dashboard, there's a well orchestrated sequence happening in the background. Let’s break it down step by step:

🚀 1. User Action → Sending a Query

It all starts when you interact with a dashboard, maybe you select a different time range, refresh a panel, or zoom into a specific data point. This triggers a request that gets sent to Grafana’s backend.

🔍 2. Query Processing → Talking to the Data Source

Grafana’s backend translates your request into a query that the selected data source understands. If you're using Prometheus, for example, Grafana converts your request into a PromQL query. If it’s Elasticsearch, it turns into a structured search request.

📦 3. Data Retrieval & Processing → Cleaning & Formatting

The data source processes the request and sends back raw data. But before it reaches your dashboard, Grafana’s backend cleans it up, applies filters, aggregates values, and formats it properly, making sure you get exactly what you need.

📊 4. Visualization → Data Comes to Life

Finally, the processed data is sent to the frontend, where Grafana transforms it into interactive graphs, charts, and tables. This real-time flow ensures that what you're seeing is always current, accurate, and easy to interpret.

Thanks for reading! If you found this helpful, follow for more DevOps concepts explained in a clear and simple way. Got a topic you'd like me to cover next? Let me know!

Kubernetes Services vs. Ingress: What You Need to Know.

Favour Lawrence — Sat, 08 Feb 2025 22:51:29 +0000

Ever tried accessing a containerized application running inside Kubernetes and realized it wasn’t as simple as running a server on your local machine? Unlike traditional setups where an app binds to a port and is instantly reachable, Kubernetes operates in a world of dynamic, ever-changing pods. If a pod dies and gets recreated, it might get a new IP, breaking direct access.

So, how do applications running inside a Kubernetes cluster communicate reliably? And how do we expose these applications to the outside world? This is where Kubernetes Services and Ingress come in.

Services ensure that even if pods come and go, your application remains accessible via a stable endpoint.
Ingress provides a smarter way to manage external access, acting as a traffic controller to route requests to the right service.

In this article, we’ll break down Kubernetes Services and Ingress, explaining when and why you need them with practical examples. Let's dive in!

🔹What is a Kubernetes Service?

In Kubernetes, a Service is an abstraction that provides a stable network endpoint to access a group of pods. Since pods are dynamic (they can be created, deleted, or rescheduled), their IPs keep changing. A Service ensures that applications can communicate reliably without worrying about changing pod IPs.

🔸 Why Do We Need Services?

🚀 Pods are ephemeral—they can restart or move to another node, getting a new IP.

🚀 Directly accessing a pod’s IP is unreliable since it might change at any moment.

🚀 A Service creates a fixed Cluster IP that stays the same, ensuring stable communication between pods and external users.

🔹 Types of Kubernetes Services

Kubernetes offers different types of Services based on how you want your application to be accessible. Let’s break them down:

🔸 ClusterIP (Default & Internal-Only)

✅ The default service type in Kubernetes.

✅ Creates an internal-only IP, meaning it’s only accessible inside the cluster.

✅ Perfect for internal communication between microservices.
Example: A backend API serving a frontend within the cluster.

🔸 NodePort (Basic External Access)

✅ Exposes the service on every node using a high-numbered port (30000–32767).

✅ You can access it via NodeIP:NodePort.

✅ Easy to set up but not ideal for production—managing ports can get messy!

🔸 LoadBalancer (Cloud-Managed External Access)

✅ Works in cloud environments like AWS, GCP, or Azure.

✅ Automatically provisions a cloud load balancer to handle traffic.

✅ The best option for production-grade external access.

🔸 Headless Service (For Direct Pod Access)

✅ Used when you don’t need a stable Cluster IP.

✅ Instead of routing traffic, it helps apps discover individual pods via DNS.

✅ Useful for databases, stateful applications, and custom service discovery.

🔹 Practical Example: ClusterIP Service YAML

Here’s a simple YAML configuration for a ClusterIP Service that exposes an Nginx pod:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx-service
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80      # The Service's Port
      targetPort: 80 # The Pod's Port

✅ selector: Matches pods with the label app: nginx, so the Service knows where to send traffic.

✅ port: The port where the Service is exposed inside the cluster.

✅ targetPort: The actual port inside the pod where traffic should go.

Understanding Ingress

Kubernetes gives us multiple ways to expose applications, but Ingress is the smartest option. Instead of creating separate external access points for each service, Ingress acts as a single entryway, efficiently routing traffic to the right service inside your cluster.

🔹 What is Ingress in Kubernetes?

Ingress is a traffic manager for your cluster. It controls external access to services using rules based on domains, paths, and protocols like HTTP/HTTPS. Think of it as a router that directs requests to the correct backend service.

🔹 Why Use Ingress Instead of NodePort or LoadBalancer?

While NodePort and LoadBalancer work, they have limitations:

❌ NodePort exposes each service on a random high-numbered port—not ideal for production.

❌ LoadBalancer works better but creates a new cloud load balancer per service, which can get expensive and complex.

✅ Ingress solves both problems by allowing multiple services to share a single entry point, reducing cost and simplifying management.

🔹 How Ingress Routes External Traffic

1️⃣ A user sends an HTTP/HTTPS request to your cluster.

2️⃣ The Ingress resource checks its rules to decide which service should handle the request.

3️⃣ Traffic is forwarded to the correct pod inside the cluster.

🔹 The Role of an Ingress Controller

An Ingress Controller is needed to process Ingress rules. Popular choices include:

✔ NGINX Ingress Controller (most common)

✔ Traefik, HAProxy, AWS ALB, Istio, etc.

🔹 Setting Up Ingress: Simple Example

Here’s a basic Ingress configuration that routes traffic based on a hostname:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
spec:
  rules:
  - host: myapp.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80

✅ host: myapp.local → Requests to myapp.local are routed.

✅ path: / → All requests are sent to the backend service.

✅ backend.service.name: my-service → Traffic goes to my-service.

✅ port.number: 80 → The port where the service listens.

🔹 How to Apply and Test Ingress in Minikube

1️⃣ Enable the NGINX Ingress Controller in Minikube:

minikube addons enable ingress

2️⃣ Apply the Ingress YAML file:

kubectl apply -f my-ingress.yaml

3️⃣ Modify /etc/hosts to point to Minikube’s IP (Linux/macOS):

echo "$(minikube ip) myapp.local" | sudo tee -a /etc/hosts

4️⃣ Test it in a browser or with curl:

curl http://myapp.local

🚀 Service vs. Ingress: When to Use Each

Choosing between a Service (NodePort/LoadBalancer) and Ingress depends on how you want to expose your applications. Here’s the breakdown:

Use a Service when you need direct access to a single service, either internally or externally. However, each exposed service requires its own endpoint, which can get expensive and inefficient if you have many services.
Use Ingress when you want to manage multiple services under one entry point. It routes traffic based on domain names or paths, reducing complexity and cost.
A Service is simple but lacks advanced traffic control. Ingress, on the other hand, supports routing, TLS termination, and virtual hosts, making it ideal for large-scale apps.

Thanks for reading! If you found this helpful, please like and follow for more DevOps content. Feel free to comment with any questions or topics you'd like to see next!

Understanding Kubernetes Volumes: Persistent Volume and Persistent Volume Claim

Favour Lawrence — Fri, 07 Feb 2025 10:51:43 +0000

Consider a case where data gets added or updated on PostgreSQL, when pod restart all changes will be gone.

Why?

Kubernetes doesn’t automatically persist data when a pod restarts. By default, when a pod dies, its storage disappears with it. This is because Kubernetes treats pods as ephemeral, they come and go, and their associated storage doesn’t stick around unless you explicitly configure it to.

For databases, logs, and any application that needs to retain state, this is a huge problem. That’s where Persistent Volumes (PV) and Persistent Volume Claims (PVC) come in. They allow Kubernetes to handle storage separately from pods, ensuring your data doesn’t vanish every time a pod is replaced.

If you’re coming from Docker, you might wonder how this compares to Docker volumes,
"Doesn’t Docker have volumes for persistent storage?"

Yes, it does. But Kubernetes handles storage in a more decentralized, scalable way. Unlike Docker, where volumes are tied to a single host, Kubernetes volumes are designed to be cluster-wide and can be provisioned dynamically.

In this guide, we’ll break down:
✅ What Persistent Volumes (PV) are and how they’re managed by administrators.
✅ How developers request storage using Persistent Volume Claims (PVC).

By the end, you’ll not only understand how Kubernetes storage works but also be able to set up persistent storage for your own applications. Let’s get started 🚀

How Does Kubernetes Handle Storage?

Kubernetes uses Volumes to provide persistent storage, but here’s the thing:

➡️ A Kubernetes Volume is just an abstraction—it doesn’t store data itself. It needs to be backed by actual physical storage.

So, where does this storage come from?

Types of Storage in Kubernetes

1️⃣ Local Storage (Node-Specific)

Tied to a single node.
If a pod moves to another node, the data doesn’t follow.
Examples:
- emptyDir (temporary storage that lasts as long as the pod exists).
- hostPath (uses a directory on the host machine).

2️⃣ Remote Storage (Cluster-Wide)

Decoupled from any single node, so pods can move freely without losing data.
Provided by external storage systems.
Examples:
- Cloud storage: AWS EBS, Google Persistent Disks, Azure Disk.
- Network storage: NFS, Ceph, GlusterFS.

Persistent Volumes (PV) and Persistent Volume Claims (PVC)

To make storage management easier, Kubernetes introduces:

✔ Persistent Volumes (PV): The actual storage backend.

✔ Persistent Volume Claims (PVC): A way for pods to request storage dynamically.

Think of it like a hotel:

A PV is a hotel room.
A PVC is a reservation.
When a pod needs storage, it "books" a room (PV) through a PVC.

📌 Kubernetes Volumes are not actual storage—they just connect your pod to a real storage system.

Persistent Volume (PV) – The Admin’s Role

We now know that Kubernetes doesn’t provide storage by itself, so who sets it up?

A PV is a pre-configured storage unit set up by the cluster administrator. Once it's available, developers can claim it using a Persistent Volume Claim (PVC) (which we’ll cover next). But first, let’s see how admins actually set up storage in Kubernetes.

How Do Admins Provision Storage?

Admins set up storage by:

1️⃣ Setting up the storage backend (Local disk, NFS, AWS EBS, etc.).

2️⃣ Defining a Persistent Volume (PV) that connects to this storage.

3️⃣ Making the PV available for developers to claim via PVCs.

Example: Creating a Persistent Volume (PV)

Here’s a simple YAML configuration for a local storage PV:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  hostPath:
    path: "/mnt/data"

Breaking It Down

🔹 capacity.storage: 5Gi → Provides 5GB of storage.

🔹 accessModes: ReadWriteOnce → Can be mounted by only one node at a time.

🔹 persistentVolumeReclaimPolicy: Retain → Storage remains even after the pod using it is deleted.

🔹 storageClassName: local-storage → Specifies which storage class to use.

🔹 hostPath: /mnt/data → Uses a local directory as storage.

Of course, admins can also configure networked storage like NFS, AWS EBS, or Google Persistent Disk instead of using a local directory.

📌 PVs exist independently of pods, ensuring data persists even if a pod is deleted or rescheduled.

Persistent Volume Claim (PVC) – The Developer’s Role

The admin has set up a Persistent Volume (PV)—now how do developers actually use it?

That’s where Persistent Volume Claims (PVCs) come in.

A PVC is a storage request from a developer. It’s like saying:

🗣️ “Hey Kubernetes, I need 5GB of storage with read/write access. Find me a PV that matches!”

If a suitable PV is available, Kubernetes automatically binds the PVC to it, making storage available for the pod.

How Developers Request Storage Using PVC

1️⃣ Create a PVC specifying storage requirements.

2️⃣ Kubernetes finds a matching PV and binds the PVC to it.

3️⃣ Mount the PVC inside a pod to store data persistently.

Example: Creating a PVC

Here’s a simple YAML configuration for a PVC requesting 5GB of storage:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: local-storage

Breaking It Down

🔹 accessModes: ReadWriteOnce → Storage can be mounted by only one node at a time.

🔹 resources.requests.storage: 5Gi → Requests 5GB of storage.

🔹 storageClassName: local-storage → Uses a PV with the matching storage class.

How the PVC Gets Bound to a PV

✅ If a PV with the right storage size, access mode, and storage class exists, Kubernetes automatically binds the PVC to it.

✅ Once bound, the PVC can be used inside a pod for persistent data storage.

Mounting the PVC in a Pod

Now that we have a PVC, let’s use it inside a pod:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
    - name: my-container
      image: postgres
      volumeMounts:
        - mountPath: "/var/lib/postgresql/data"
          name: my-storage
  volumes:
    - name: my-storage
      persistentVolumeClaim:
        claimName: my-pvc

📌 Now, even if the pod restarts, the PostgreSQL database will still have its data

Thanks for reading! Be sure to follow for more DevOps content, and feel free to comment with the DevOps concepts you'd like to see covered next.

Very Insightful

Favour Lawrence — Wed, 05 Feb 2025 23:32:25 +0000

5 Terraform Best Practices I Wish I Knew When I Started

Bobby ・ Jan 31

#terraform #devops #cloud #beginners

Prometheus Architecture: Understanding the Workflow 🚀

Favour Lawrence — Mon, 03 Feb 2025 22:18:10 +0000

Have you ever used Prometheus for monitoring systems? It’s great at collecting and storing metrics, but have you ever stopped to wonder how it actually works under the hood? What makes its architecture so efficient, and why is it the go to choice for cloud-native monitoring?
Unlike traditional monitoring tools that passively wait for data, Prometheus actively scrapes metrics from defined targets, stores them efficiently in a time-series database.
We’ll explore how it collects metrics, how its components interact, and why its design makes it a favorite among developers and SREs. Let’s get started! 🚀

Prometheus Architecture: Breaking It Down

If you want to truly understand how Prometheus works, you need to go beyond just “it collects metrics” and dive into its architecture. At its core, Prometheus is built on three essential pillars:

✅ Time-Series Database (TSDB) – Where all metrics are efficiently stored.

✅ Data Retrieval Engine – Responsible for actively pulling (scraping) metrics.

✅ Query & API Layer (Web Server) – The interface where you analyze and visualize data.

Each of these components plays a critical role in making Prometheus fast, scalable, and cloud-native. Now, let’s break them down in detail.

1. Prometheus Server – Command center

The Prometheus Server is the central hub that coordinates everything, ensuring your metrics are collected, stored, and made accessible. Here’s what it does:

🔹 Pulls metrics from configured targets (applications, databases, and exporters).

🔹 Stores the collected data in a time-series format.

🔹 Provides a powerful query interface to analyze and visualize the data.

2. Time-Series Database (TSDB) – Storing Metrics Efficiently

Once Prometheus scrapes metrics, it needs a way to store them efficiently. That’s where the Time-Series Database (TSDB) comes in. This isn’t your average database; it’s specifically designed for handling time-series data. Here’s what happens behind the scenes:

📌 Metrics are stored as time-series data each metric is recorded with a timestamp and value.

📌 Compression techniques uses advanced compression techniques to store data efficiently without slowing down performance.

📌 A label-based system metrics are tagged with labels (e.g., http_requests_total{status="200"}), making it easy to filter and query data with precision.

3. Data Retrieval Engine – How Prometheus Collects Metrics

Prometheus doesn’t sit around waiting for data—it actively goes out and pulls it from defined targets. This is known as the pull-based model.

How It Works:

Prometheus periodically scrapes /metrics endpoints from configured targets. These can be:

✔️ Applications exposing Prometheus-compatible metrics

✔️ Databases and external services

✔️ Exporters that convert non-Prometheus metrics into a readable format

This approach ensures Prometheus collects data efficiently while remaining highly adaptable.

4. Query & API Layer (Web Server) – Making Data Useful

Storing metrics is one thing, but being able to query, analyze, and visualize them is where the real power comes in. This is where the Query & API Layer play their role.

Key Responsibilities:

🔎 Handles PromQL (Prometheus Query Language) for in-depth metric analysis.

🔎 Runs an HTTP API server, allowing external tools (like Grafana) to pull data.

🔎 Provides built-in graphing for quick insights.

How It All Comes Together

1️⃣ Prometheus scrapes metrics from various targets.

2️⃣ It stores data efficiently in TSDB.

3️⃣ The query engine allows users to analyze trends and set up alerts.

4️⃣ Other tools (like Grafana) fetch data via Prometheus' API for visualization.

Pull Mechanism

Here’s how it works: Prometheus is set up with a list of targets, think applications, databases, or exporters that provide metrics through a /metricsendpoint. At regular intervals, Prometheus sends an HTTP request to these endpoints, grabs the metrics, adds a timestamp to each one, and then stores everything in its Time-Series Database (TSDB).
It’s like Prometheus is constantly checking in on these targets, gathering fresh data, and keeping everything organized for easy analysis later on.

Why Prometheus?

No External Storage Needed: Unlike some monitoring systems that rely on external storage, Prometheus keeps things simple by storing data locally—cutting down on complexity and external dependencies.
Resilient Pull-Based Monitoring: By actively scraping metrics instead of waiting for them, Prometheus is more resilient to network issues, ensuring data is consistently collected even when connections are not stable.
Handles Short-Lived Jobs: For tasks that don’t run long enough to be scraped, Prometheus offers the Pushgateway. This lets ephemeral jobs push their metrics before exiting, ensuring no data is lost.

There are plenty of reasons why Prometheus is used worldwide—its architecture truly sets it apart. I hope this article helped you get a clear understanding of how it all works.

Thanks for reading! Don’t forget to follow, and feel free to leave a comment with the next DevOps concept you’d like me to dive into. Let’s keep the learning going!

Namespaces in Kubernetes Explained: 🔍 Understanding Isolation and Sharing

Favour Lawrence — Sat, 01 Feb 2025 21:46:39 +0000

Kubernetes Namespace?

If you've worked with Kubernetes long enough, you’ve probably seen how quickly things can spiral out of control. One team deploys a new service, another updates their staging environment, and suddenly, production is down because someone accidentally messed with the wrong resources. Sound familiar?

That’s where Kubernetes namespaces come in. Instead of stuffing everything into one disorganized cluster or deploying individual clusters for each project, namespaces help maintain order, enforce security boundaries, and enhance resource management efficiency.

In this article, we’ll explore:

What Kubernetes namespaces are and how they work.
Why they’re essential for managing multi-team, multi-application clusters.
Resources that can and can't be shared across namespaces.

By the end, you’ll have a solid grasp of how to use namespaces to keep your cluster structured and scalable. Let’s dive in. 🚀

What Are Kubernetes Namespaces, Really?

A namespace in Kubernetes is basically a way to split your cluster into separate environments. At their core, namespaces are like virtual partitions in your cluster. They let you split your resources; pods, services, deployments, and more into separate, logical groups.

The Default Namespace

When you first start with Kubernetes, everything you deploy lands in the default namespace. It’s quick, easy, and works fine for small project.

Kubernetes comes with a few built-in namespaces:

- default – The catch-all for resources if you don’t specify a namespace.
- kube-system – Reserved for critical system components (like the Kubernetes API server).
- kube-public – Mostly unused, but contains publicly accessible resources.
- kube-node-lease – Helps track node health and optimize performance.

For example , you could create your own namespaces. Here’s how:

kubectl create namespace dev  
kubectl create namespace staging  
kubectl create namespace production

This way, dev doesn’t interfere with staging, and staging doesn’t break production.

When to Use Namespaces (and When Not To)

✅ Use namespaces if:

You have multiple teams sharing the same cluster.
You need clear separation between environments (dev, staging, prod).
You want to enforce security policies and resource limits per group.

❌ Don’t bother with namespaces if:

Your cluster is small and managed by a single team.
You need hard isolation—separate clusters might be the better option.
You’re dealing with global resources like cluster-wide CRDs.

At the end of the day, namespaces help keep your Kubernetes setup clean and organized.

Working with Namespaces: Let’s Get Hands-On

Now that you understand the concept of namespaces, let’s dive into how you can actually work with them in your Kubernetes cluster. This section will cover the essential commands you need to list, create, and manage namespaces, as well as how to deploy resources to specific namespaces.

Listing Existing Namespaces

To see what namespaces you’ve got in your cluster, run this:

kubectl get namespaces

This command will give you a list of namespaces, including the default ones like default, kube-system, and any you’ve created yourself. It's a quick way to check what namespaces are active and available.

Creating a New Namespace

Creating a new namespace is super straightforward. Just run the kubectl create namespace command followed by the name you want for your new namespace:

kubectl create namespace my-namespace

And that’s it! Your new namespace is ready to go. Now you can deploy your resources into it, keeping everything organized.

Deploying Resources to a Namespace

If you want to deploy resources like pods, services, or deployments into a specific namespace, you can use the -n flag with kubectl apply. For example, to apply a YAML configuration to the my-namespace namespace:

kubectl apply -f app.yaml -n my-namespace

This will create the resources defined in app.yaml within the my-namespace namespace. Just make sure your YAML file doesn’t already have a namespace field in the metadata unless you want to override the command-line flag.

Switching Between Namespaces

Sometimes you’ll need to switch between namespaces while working with kubectl. To make this easier, you can set the default namespace for your session with this command:

kubectl config set-context --current --namespace=my-namespace

After running this, all your subsequent kubectl commands will default to my-namespace, so you won’t have to keep adding the -n flag. To switch back to the default namespace, just run:

kubectl config set-context --current --namespace=default

Resources That Can and Can’t Be Shared Across Namespaces

In Kubernetes, namespaces help isolate different resources within a cluster, but not all resources are confined to a single namespace. Some can be shared across namespaces, while others stay restricted. Knowing which resources can be shared (and which can't) is crucial when it comes to managing your cluster effectively. Let's break it down.

Resources That Can’t Be Shared Across Namespaces

Some resources are strictly bound to a specific namespace. These are isolated within their own namespace to ensure everything remains organized and secure.

Pods

Pods are created within a namespace, and they can’t be directly accessed by pods in other namespaces. They’re local to the namespace they reside in, which means if you want to let pods in other namespaces communicate, you'll need to set up things like network policies.

Why is this important? Pods are isolated by default, so if you need cross-namespace communication, you'll have to jump through a few hoops, like using services or network policies.

Services

In Kubernetes, services are tied to specific namespaces. They expose resources like pods only within their own namespace. If you need a service accessible across multiple namespaces, you'll have to either create an external service or configure DNS settings between the namespaces.

Why is this important? For inter-namespace communication, managing service discovery becomes key. You’ll have to reference the service using a fully qualified domain name, like my-service.my-namespace.svc.cluster.local.

ConfigMaps & Secrets

Both ConfigMaps and Secrets are tied to namespaces. While you can reference them within the same namespace or copy them to another namespace, you can't share them directly across namespaces.

Note: It’s important to scope things like app configuration and sensitive data to the correct namespace. While you can replicate or reference them elsewhere, they can’t just be shared freely across namespaces.

Deployments and StatefulSets

Just like pods, Deployments and StatefulSets live within a namespace. These resources manage pods within that namespace, so they won’t span multiple namespaces.

Why this matters: This helps keep things isolated and manageable, especially when you're scaling applications or managing their lifecycle.

Resources That Can Be Shared Across Namespaces

Not everything in Kubernetes is namespace-bound. There are a few resources that can span multiple namespaces, which helps Kubernetes maintain global management while respecting the boundaries that namespaces provide.

Nodes

Nodes exist at the cluster level, not tied to any specific namespace. Every pod, regardless of which namespace it belongs to, can run on any available node in the cluster.
Nodes make it possible to efficiently manage resources across the whole cluster without worrying about namespace boundaries.

Cluster-wide Resources (e.g., CRDs, ClusterRoles)

Resources like Custom Resource Definitions (CRDs) and ClusterRoles are not limited to namespaces. These resources are designed to work across the entire cluster, whether you’re defining custom resources or setting cluster-wide access policies.
CRDs let you create custom objects that can be accessed from anywhere, while ClusterRoles manage permissions at the cluster level, allowing users and services to access resources across namespaces.

Network Policies

While network policies are generally scoped to namespaces, they can still define how pods in different namespaces communicate with each other. By setting up cross-namespace rules, you can control traffic between pods in different namespaces.

Why this is important: Network policies allow you to maintain control over which namespaces can talk to each other, even if they’re isolated.

Persistent Volumes (PVs)

Persistent Volumes are another cluster-level resource that isn’t tied to a specific namespace. However, Persistent Volume Claims (PVCs) are namespace-bound, and while PVCs can request storage from a PV, the PV itself can be accessed by resources in multiple namespaces (depending on access mode).

Why this matters: While you can manage storage across namespaces with PVs, the data and requests are still scoped to namespaces through PVCs.

Ingress Resources

Ingress resources allow external access to services and can be configured to route traffic to services across different namespaces. By setting up the right rules, a single Ingress controller can manage traffic to services across multiple namespaces.

Note: You can centralize traffic management with one Ingress, even when your services span multiple namespaces.

Thanks for reading! Stay tuned for more deep dives into Kubernetes concepts!

Feel free to dm let's talk devOps on X