DEV Community: DevOps Man

Mastering CMake: A Practical Guide for DevOps Engineers and Developers

DevOps Man — Sat, 10 May 2025 03:49:30 +0000

CMake is the backbone of modern C++ build systems. Whether you're compiling a simple executable or orchestrating large-scale multi-module projects with external libraries, mastering CMake can make your builds cleaner, faster, and more maintainable.

This guide captures everything you need to know to write robust CMakeLists.txt files with real-world examples and explanations.

Basic CMake Commands
Project Structure
Commonly Used Variables
Best Practices
Compiler Warnings
Configuring files
Unit Testing with Catch2
Compile Features and Definitions
Sanitizers
IPO & LTO
Generator Expressions
External Libraries (Git Submodules & FetchContent)
Useful CMake CLI Flags

Basic CMake Commands

cmake_minimum_required(VERSION 3.10)
This command specifies the minimum CMake version required for your project. It should be the first line in your top-level CMakeLists.txt.

project(MyApp VERSION 1.0 LANGUAGES CXX)
Defines the project name, version, and language.

add_executable(MyApp main.cpp helper.cpp)
This tells CMake to compile main.cpp and helper.cpp into an executable named MyApp. You can list as many source files as needed. The target name (MyApp) is used in other commands, like target_link_libraries(MyApp PRIVATE SomeLib)

add_library(MyLib STATIC mylib.cpp)
Defines a library named MyLib.

STATIC: Creates a static library (.lib or .a). Linked at compile time.
SHARED: Creates a shared (dynamic) library (.dll or .so). Loaded at runtime.
MODULE: Creates a library that is not linked but loaded at runtime (e.g., for plugins).
INTERFACE: No output file is built. Used for header-only libraries.

target_include_directories(MyLib PUBLIC include)
Specifies include directories. Types:

PUBLIC: Used by both the library and users
PRIVATE: Used only by the library
INTERFACE: Used only by users

add_subdirectory(utils)
Includes utils/CMakeLists.txt in the build.

target_link_libraries(MyApp PRIVATE MyLib)
Links MyLib with MyApp.

set(MY_VAR "value")
Defines a variable.

option(BUILD_TESTS "Build test binaries" ON)
Defines a boolean option with description.

if(USE_MY_FEATURE)
    message(STATUS "Building with my feature")
else()
    message(STATUS "Building without my feature")
endif()

Conditional logic in CMake.

Project Structure

project-root/
├── CMakeLists.txt
├── main.cpp
├── include/
│   └── mylib.hpp
├── src/
│   └── mylib.cpp
├── tests/
│   ├── CMakeLists.txt
│   └── test_main.cpp

Commonly Used Variables

CMAKE_SOURCE_DIR: Top-level source directory
CMAKE_PROJECT_NAME: Name set in project()
CMAKE_BINARY_DIR: Top-level build directory
CMAKE_CURRENT_SOURCE_DIR: Source dir of the current CMakeLists.txt
CMAKE_CURRENT_LIST_DIR: Directory containing the currently processed CMake file
CMAKE_MODULE_PATH: List of additional module directories

Best Practices

✅ Always list files manually instead of using file(GLOB ...)
✅ Modularize using add_subdirectory()
✅ Use target_* commands (not global ones)
✅ Separate headers and sources by folder
❌ Don’t pollute global scope with variables

Compiler Warnings

target_compile_options() adds compiler flags/options at compile time to a specific target (executable or library).
Use portable warnings with:

target_compile_options(MyApp PRIVATE
  $<$<CXX_COMPILER_ID:GNU,Clang>:-Wall -Wextra -Wpedantic>
  $<$<CXX_COMPILER_ID:MSVC>:/W4>
)

🚀 Common Compiler Flags (for GCC/Clang)

Flag	Meaning
`-Wall`	Enable most common warnings
`-Wextra`	Enable additional warnings
`-Werror`	Treat warnings as errors
`-O2`, `-O3`	Optimization levels (O2 = good speed, O3 = aggressive)
`-g`	Generate debug symbols for gdb or lldb
`-std=c++17`	Use C++17 standard (replace with `c++11`, `c++20`, etc.)
`-DDEBUG`	Define macro `DEBUG`, useful for conditional code blocks
`-fPIC`	Position Independent Code (for shared libraries)

Configuring Files with configure_file

CMake’s configure_file() command lets you copy and process a template file at configure time, replacing variable placeholders. For example, if you have a header template file.h.in:

// file.h.in
#define VERSION @PROJECT_VERSION@

and in your CMakeLists.txt you do:

project(MyApp VERSION 1.2.3)
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/file.h.in
               ${CMAKE_CURRENT_BINARY_DIR}/file.h
               @ONLY)

CMake will produce file.h in the build directory with @PROJECT_VERSION@ replaced by the actual version string (e.g. #define VERSION 1.2.3). In general, configure_file() copies a “.in” file to the build tree and substitutes any ${VAR} with the current CMake variable values
cliutils.gitlab.io
. This is very useful for generating headers (or even .cmake files) that convey build-time settings like version, feature toggles, or compile options. Remember to add the build directory to your include path (e.g. via target_include_directories) so that code can #include "file.h" from the generated location.

Unit Testing with Catch2

Setup
git submodule add https://github.com/catchorg/Catch2.git external/Catch2

CMakeLists.txt

add_subdirectory(external/Catch2)
add_executable(tests test_main.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)
include(CTest)
include(Catch)
catch_discover_tests(tests)

Compile Features and Definitions

Enforce Features
target_compile_features(MyApp PRIVATE cxx_std_17)

Add Macros
target_compile_definitions(MyApp PRIVATE VERSION=1.0)

Sanitizers

target_compile_options(MyApp PRIVATE -fsanitize=address)
target_link_options(MyApp PRIVATE -fsanitize=address)

Apply similar for undefined sanitizer.

IPO & LTO

Enable link-time optimization:
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE)

Or per target:
set_property(TARGET MyApp PROPERTY INTERPROCEDURAL_OPTIMIZATION TRUE)

Generator Expressions

Inline conditional logic:
$<$<CONFIG:Debug>:-DDEBUG>

External Libraries

Git Submodules
Add submodule: git submodule add <repo> external/LibName
In root CMake:

add_subdirectory(external/LibName)
target_link_libraries(MyApp PRIVATE LibName)

FetchContent

include(FetchContent)
FetchContent_Declare(
  fmt
  GIT_REPOSITORY https://github.com/fmtlib/fmt.git
  GIT_TAG master
)
FetchContent_MakeAvailable(fmt)
target_link_libraries(MyApp PRIVATE fmt::fmt)

Useful CMake CLI Flags

cmake -S . -B build/ -G "Ninja"
-S: Source directory
-B: Build directory
-G: Generator

cmake --build build/ --target MyApp
Builds only MyApp

cmake --build . --target extLib
Build external target

Dependency Graph

Use Graphviz to visualize target dependencies:

cmake --graphviz=graph.dot .
dot -Tpng graph.dot -o graph.png

Conclusion

Mastering CMake is about understanding its modular philosophy. Start with small clean builds, add subdirectories and targets gradually, and leverage the power of target_ commands for scalability and control.

Useful links:

🔧 GitLab CI/CD — The Complete Guide

DevOps Man — Tue, 06 May 2025 15:01:26 +0000

🚀 Why GitLab CI/CD Over Jenkins and GitHub Actions?

Feature	GitLab CI/CD	Jenkins	GitHub Actions
Integrated UI	✅ Built-in	❌ Plugin-based	✅ Built-in
Auto DevOps	✅ Native support	❌ Manual	❌
Kubernetes Deployment	✅ First-class	✅ With plugins	✅ Beta level
Security (Secrets, SAST, DAST)	✅ Built-in	❌ Manual config	⚠️ External
Runners Management	✅ Docker/Machine/K8s	✅ Nodes	✅ Hosted/self-hosted

Why choose GitLab CI/CD?

Everything in one place (version control + CI/CD)
Great for GitOps and K8s-based workflows
Easy syntax, strong security model
No plugin hell like Jenkins

📁 Pipeline Basics

'# Comments'
Use # to write inline documentation.

# This job builds the project
build:
  script:
    - echo "Compiling..."

stages
Defines the execution order of jobs.

stages:
  - build
  - test
  - deploy

stage
Assigns a job to a defined stage.

test:
  stage: test
  script: echo "Running tests"

script
Commands that the job will execute.

build:
  stage: build
  script:
    - make build

🔀 Multi-Jobs in a Stage

build_app:
  stage: build
  script: make app

build_api:
  stage: build
  script: make api

⏳ timeout
Job-level timeout.

test:
  script: run_tests
  timeout: 30 minutes

📌 needs – Optimizing Pipeline Dependency

deploy:
  stage: deploy
  script: deploy.sh
  needs: [build]

Use when job depends on output from another but not the full stage.

🧹 before_script and after_script

before_script:
  - echo "Setup"

after_script:
  - echo "Cleanup"

Used globally or at job level for pre/post commands.

📦 artifacts – Store Build Outputs

build:
  script: make build
  artifacts:
    paths:
      - dist/
    expire_in: 1 week

🧬 variables – 5 Levels

Project-level: via GitLab UI → Settings → CI/CD → Variables
Used across all jobs.
Global level:

variables:
  ENV: production

Job-level:

test:
  variables:
    ENV: test
  script: echo $ENV

Built-in:

$CI_PIPELINE_ID and $CI_COMMIT_BRANCH..............

From file:

variables:
  CONFIG: config.yml

🛂 only and except (Deprecated → Use rules)

build:
  script: build.sh
  only:
    - main

📜 rules, if, and when

test:
  script: test.sh
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: always

🏃 Run File Inside Script

script:
  - chmod +x ./deploy.sh
  - ./deploy.sh

🔁 workflow – Pipeline Start Conditions

workflow:
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: always
    - when: never

🐳 image – Docker Environment

image: python:3.9

script:
  - pip install -r requirements.txt

🔢 Matrix/Loops (via parallel:matrix)

job:
  stage: test
  parallel:
    matrix:
      - VARIANT: [A, B, C]
  script:
    - echo $VARIANT

🌐 Multi-Project Pipelines

trigger-downstream:
  stage: trigger
  trigger:
    project: other-group/other-project
    branch: main

📦 Downloading/Packaging in a Pipeline

build:
  script:
    - wget https://example.com/app.tar.gz
    - tar -xzf app.tar.gz

⏰ Pipeline Scheduling
Go to GitLab UI → CI/CD → Schedules

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: always

🏃 GitLab Runner Types

Runner Type	Description	Use Case
Shared Runners	Maintained by GitLab	Quick use, limited config
Group Runners	Shared by a group	Org-level control
Project Runners	Bound to a project	Isolation, security
Docker Executor	Containers	Microservices, clean env
Shell Executor	Host machine	Full access needed
Kubernetes Executor	Pods on K8s	Cloud-native CI/CD

🗃️ Introduction to Database Design: Concepts Every Beginner Should Know

DevOps Man — Fri, 18 Apr 2025 07:19:54 +0000

If you're diving into the world of databases and wondering how to structure data efficiently, you're in the right place. In this article, we’ll walk through foundational concepts in database design — from understanding what data is, to topics like normalization and joins. Whether you're building apps, handling data for projects, or prepping for tech interviews, these definitions will be your solid starting point.

📌 What is Data?

Data refers to raw facts, figures, or symbols that have not yet been processed or given context. For example, "23", "Alice", or "2025-04-18" are all data. When processed and organized, data becomes information.

📌 What is a Database?

A Database is an organized collection of data that can be easily accessed, managed, and updated. Think of it as a digital filing system for storing information efficiently.

📌 What is a Relational Database?

A Relational Database organizes data into tables (also known as relations) that are related to each other through keys. This model uses SQL for querying and is based on mathematical set theory and relational algebra.

📌 What is an Entity?

An Entity is a real-world object or concept about which data is stored in a database. For example: a Student, Book, or Order can be considered entities.
An Entity is also called Tuple, Row and Record.

📌 What are Attributes?

Attributes are the properties or characteristics of an entity. For a Student entity, attributes could be Name, Roll Number, Age, and Class.
The attributes are also called column or field.

📌 What is a Table?

A Table is a collection of rows and columns in a relational database. Each table stores data about a specific entity type.
The table is also called file.

📌 What are Entity Types and Attribute Types?

Entity Types: A category of entities sharing the same attributes (e.g., Student, Teacher).
Attribute Types: Attributes can be:

Simple: Cannot be divided (e.g., Age)
Composite: Can be split (e.g., Full Name)
Derived: Derived from other data (e.g., Age from DOB)
Multivalued: Can hold multiple values (e.g., Phone Numbers)

📌 What is DBMS and RDBMS?

DBMS (Database Management System): Software used to store and manage data (e.g., MongoDB, Redis).
RDBMS (Relational DBMS): A DBMS that stores data in tables and supports relationships between them (e.g., MySQL, PostgreSQL, Oracle).

📌 What is SQL?

SQL (Structured Query Language) is the standard language for interacting with relational databases. It allows you to Create, Read, Update, and Delete data — often abbreviated as CRUD.

📌 What are SQL Languages?

SQL is divided into sub-languages:

DDL (Data Definition Language): CREATE, ALTER, DROP
DML (Data Manipulation Language): SELECT, INSERT, UPDATE, DELETE
DCL (Data Control Language): GRANT, REVOKE
TCL (Transaction Control Language): COMMIT, ROLLBACK, SAVEPOINT

📌 Why Database Design?

Good design ensures:

Efficiency in queries
Data integrity
Scalability
Avoidance of redundancy and anomalies

📌 What are Schemas and Types?

A Schema is the blueprint of a database — it defines how data is organized, including tables, columns, data types, and relationships.

Types of schemas:

Logical Schema: High-level design
Physical Schema: Storage-level design
View Schema: Customized user perspectives

📌 Naming Convention

Use clear, consistent names in lowercase with underscores:

Tables: student_data
Columns: first_name, created_at Avoid spaces, reserved keywords, and overly complex names.

📌 What are Data Types?

Each column has a data type which defines the kind of data it holds. Examples:

INT for integers
VARCHAR(n) for text
DATE, BOOLEAN, FLOAT, etc.

📌 What is Data Integrity and its Types?

Data Integrity ensures the accuracy and consistency of data. Types include:

Entity Integrity: Primary keys must be unique and not null
Referential Integrity: Foreign keys must refer to valid rows
Domain Integrity: Data must be valid for its field (e.g., age must be a number)

📌 What are Atomic Values?

Atomic values mean that each field should contain a single, indivisible value — not a list or multiple items. This is a core rule of First Normal Form (1NF).

📌 What is Normalization?

Normalization is the process of organizing data to reduce redundancy. It involves breaking down tables into smaller ones and establishing relationships.

Common forms:

1NF (First Normal Form): Eliminate repeating groups
2NF: Remove partial dependency
3NF: Remove transitive dependency

🔗 Relationships in a Relational Database

Relationships define how tables (entities) in a database are connected to each other. Understanding relationships is key to designing normalized and efficient databases. There are three main types of relationships:

📘 One-to-One (1:1) Relationship
In a One-to-One relationship, a row in Table A is related to only one row in Table B, and vice versa.
✅ Example Design:

Table: User

user_id (Primary Key)
username

Table: UserProfile

profile_id (Primary Key)
user_id (Foreign Key referencing User.user_id)
bio
avatar_url

Each user has exactly one profile, and each profile belongs to one user.

📘 One-to-Many (1:N) Relationship

In a One-to-Many relationship, a row in Table A can relate to multiple rows in Table B, but each row in Table B relates to only one row in Table A.

✅ Example Design:
Table: Customer

customer_id (Primary Key)

name

Table: Order

order_id (Primary Key)
customer_id (Foreign Key referencing Customer.customer_id)
order_date
total_amount

One customer can place many orders, but each order belongs to one customer.

📘 Many-to-Many (M:N) Relationship
In a Many-to-Many relationship, multiple rows in Table A relate to multiple rows in Table B. This is implemented using a junction (join) table.

✅ Example Design:
Table: Student

student_id (Primary Key)
name

Table: Course

course_id (Primary Key)
course_name

Table: StudentCourse (Join Table)

student_id (Foreign Key referencing Student.student_id)
course_id (Foreign Key referencing Course.course_id)

A student can enroll in many courses, and a course can have many students.

🔑 What is a Primary Key?
A Primary Key is a column (or set of columns) that uniquely identifies each record in a table.

It must be unique
It cannot be null
A table can have only one primary key

🔗 What is a Foreign Key?
A Foreign Key is a column that creates a relationship between two tables. It refers to the primary key in another table.

It ensures referential integrity
It can accept nulls (unless otherwise restricted)

will be continue

Mastering Git: The Ultimate Developer's Guide to Version Control

DevOps Man — Thu, 17 Apr 2025 15:20:28 +0000

What is Git? What is Version Control?

Git is a distributed version control system (VCS) designed to track changes in source code during software development. It allows multiple developers to work on the same project simultaneously without interfering with each other's work.

Version control, in general, is the practice of tracking and managing changes to software code over time. It allows developers to revert to previous versions, collaborate effectively, and maintain the integrity of their codebase.

Types of Version Control and Difference Between Them

Centralized Version Control (CVCS): A system where the version history is stored on a central server, and developers access this repository to make changes. Examples include Subversion (SVN) and Perforce. The main issue is that if the central server goes down, no one can access the version history or make changes.

Distributed Version Control (DVCS): Systems like Git, where each developer has a complete copy of the repository, including its history, on their local machine. This allows for greater flexibility and resilience. Git is an example of a DVCS, and it doesn't rely on a central server for basic operations, allowing offline work and better collaboration.

Git Config – Global, Local, and System Config
Git provides three types of configuration levels to control settings:

Global Config: This configuration applies to all repositories of the current user. It's stored in the user's home directory
(~/.gitconfig or ~/.config/git/config).
Local Config: This configuration applies only to a single repository. It is stored in the .git directory of the repository
(.git/config).
System Config: This configuration applies to all users on the system and is stored in the system's Git config file, usually located at /etc/gitconfig.

Git Config Command to View
To view the configuration values set at different levels, you can use the following commands:

Global Config:

git config --global --list

Local Config:

git config --local --list

System Config:

git config --system --list

To view all configuration levels, you can use:

git config --list

Git Workflow Overview (Working Directory, Staging Area, Repository)
Git workflow involves three main areas:

Working Directory: The directory where files are modified. This is where you make changes to your files.
Staging Area (Index): This is where files are prepared before being committed. It allows you to decide which changes to include in the next commit.
Repository (Git Directory): The .git directory where the full history and configuration of the project are stored. This is where the commits are made and tracked.

Creating a Repository

git init: Initializes a new empty Git repository in the current directory.

git init

git clone: Creates a local copy of an existing remote repository. It downloads all the files, commits, branches, and history.

git clone <repository_url>

Checking Status
To check the current state of your repository, including which files have been modified or are staged, use:

git status

This command will show you the differences between your working directory, the staging area, and the repository.

Adding and Committing

git add: Adds changes in the working directory to the staging area, preparing them to be committed.

git add <file_name>
git add .  # Adds all changes

git commit: Records the changes in the staging area to the repository with a commit message.

git commit -m "Commit message"

Viewing History

git log: Displays the commit history with commit hashes, authors, dates, and messages.

git log

git show: Displays detailed information about a specific commit.

git show <commit_hash>

git diff: Shows the differences between the working directory and the staging area or between commits.

git diff

Reverse of Git Workflow (Working Directory to Staging Area and Staging Area to Repository)

From Repository to Staging Area: You can undo a commit and move the changes back to the staging area using:

git reset --soft HEAD^

This will reset the repository to the previous commit but keep the changes in the staging area.

From Staging Area to Working Directory: If you want to unstage changes (move them back to the working directory without losing them):

git reset <file_name>

This will unstage a specific file but leave it in the working directory.

Branching and Merging

Git branches allow developers to work on new features or bug fixes in isolation from the main production code. Branching is a powerful feature that enables parallel development, experimentation, and clean merging of code.

Merging is the process of integrating changes from one branch into another. Usually, changes from a feature branch are merged into the main branch once the feature is complete.

Creating and Switching Branches

Create a new branch using:

git branch <branch_name>

Switch to another branch using:

git checkout <branch_name>

Or using the modern alternative (since Git 2.23+):

git switch <branch_name>

Create and switch in one step:

git checkout -b <branch_name>
or
git switch -c <branch_name>

Merging Branches
To merge changes from one branch into the current branch:

git merge <branch_name>

This command applies the changes from the specified branch into the branch you're currently on.

Fast-forward vs. 3-way Merge

Fast-forward Merge: If there have been no changes on the base branch since the feature branch was created, Git will simply move the pointer forward to include the new commits.

*---*---*   (main)
         \
          *---*---* (feature)

After fast-forward merge:

*---*---*---*---* (main)

3-Way Merge: If both branches have diverged, Git uses a 3-way merge to combine changes. It compares the common ancestor with both branches and creates a new commit to reconcile them.

Merge Conflicts and Resolving Them
A merge conflict occurs when Git is unable to automatically merge changes due to overlapping edits on the same lines in the same file.

When this happens:

Git will mark the conflict in the file.
Manually edit the file to resolve the conflict.
After resolving the conflict, do add and commit again.

git stash – Save and Reapply Work
The git stash command allows you to temporarily save changes that you don’t want to commit immediately. It's helpful when you need to switch branches or pull updates without committing unfinished work.

Stash current changes:

git stash

List all stashes:

git stash list

Apply the most recent stash (and keep it in the stash list):

git stash apply

Apply and remove the stash from the list:

git stash pop

Apply a specific stash:

git stash apply stash@{2}

Drop a stash:

git stash drop stash@{0}

Clear all stashes:

git stash clear

Adding Remotes (git remote add)

A remote is a reference to a repository hosted elsewhere (typically on GitHub, GitLab, Bitbucket, etc.). You can connect your local repository to a remote to collaborate and share code.

Add a remote repository:

git remote add origin <remote_repository_url>

- List all remotes:

git remote -v

Rename a remote:

git remote rename origin upstream

Remove a remote:

git remote remove origin

Pushing Changes (git push)

After committing locally, you need to push the changes to a remote repository so others can access them.

Push to the default remote (origin) and current branch:

git push

Push a specific branch to a remote:

git push origin <branch_name>

Push all local branches to origin:

git push --all origin

Push with upstream tracking (first-time push):

git push -u origin <branch_name>

Pulling Changes (git pull)

The git pull command fetches and integrates changes from a remote repository into your current branch. It's a combination of git fetch and git merge.

Pull latest changes from origin:

git pull origin <branch_name>

If your current branch tracks a remote branch, simply:

git pull

Fetching Changes (git fetch)

The git fetch command downloads commits, files, and refs from a remote repository without merging them into your working branch.

Fetch from origin:

git fetch origin

After fetching, you can compare or merge changes manually:

git merge origin/<branch_name>

git fetch is a safe way to see what others have pushed without touching your current work.

Tracking Branches

A tracking branch is a local branch that is connected to a remote branch. This relationship makes commands like git push and git pull more convenient.

Create a tracking branch manually:

git checkout -b <local_branch> origin/<remote_branch>

Set upstream while pushing:

git push -u origin <branch_name>

Check which remote branch your local branch is tracking:

git branch -vv

Change the upstream branch:

git branch --set-upstream-to=origin/<remote_branch> <local_branch>

Git Tags (git tag, git tag -a)

Tags are used to mark specific points in history as important — typically to label releases (v1.0.0, v2.0.1, etc.).

List all tags:

git tag

Create a lightweight tag:

git tag <tag_name>

Create an annotated tag:

git tag -a <tag_name> -m "Tag message"

Tag a specific commit:

git tag -a <tag_name> <commit_hash> -m "Message"

Push tags to remote:

git push origin <tag_name>

Push all tags:

git push origin --tags

Delete a tag locally:

git tag -d <tag_name>

Delete a remote tag:

git push origin --delete <tag_name>

Reverting Changes (git revert, git reset)

git revert: Safely undoes a commit by creating a new commit that reverses the changes. It preserves history.

git revert <commit_hash>

git reset: Moves the HEAD and optionally resets the index (staging area) and working directory. Can erase commits if used carelessly. Soft (keep changes staged):

git reset --soft <commit_hash>

Mixed (unstage changes):

git reset --mixed <commit_hash>

Hard (discard everything):

git reset --hard <commit_hash>

Difference Between reset, revert, checkout

Command	Effect on Commits	Effect on Working Directory	Safe?	Usage
`reset`	Removes commits	Optional (hard resets files)	❌ Risky	Clean up local history before pushing
`revert`	Adds new commit	Leaves files as-is	✅ Very Safe	Undo changes in a shared/public branch
`checkout`	Switches branches or restores files	Overwrites working directory files	⚠️ Safe (with caution)	Restore files or move between branches

Rewriting History

Modify the most recent commit (message or contents):

git commit --amend

This is useful when you forgot to add a file or want to edit the commit message.

Rebase allows you to replay commits from one branch onto another, enabling a clean linear history.

git rebase <base_branch>

Apply a specific commit from one branch onto another:

git cherry-pick <commit_hash>

This is useful to copy fixes or features without merging the entire branch.

Cleaning Up (git clean)

Removes untracked files and directories from your working directory.

Dry run (see what will be deleted):

git clean -n

Delete untracked files:

git clean -f

Delete untracked directories too:

git clean -fd

⚠️ Use this command with care — it permanently deletes files not tracked by Git.

Detached HEAD

A detached HEAD state means you're not on any branch — instead, you're pointing directly to a commit.

This happens when you:

git checkout <commit_hash>

You can view, explore, or test changes, but any commits made now won’t belong to any branch unless you create one:

Create a branch from detached HEAD:

git checkout -b new-branch

This is useful for reviewing or reverting specific historical commits without affecting branches.

git log with Options

The git log command shows the commit history of the repository. It’s highly customizable using flags.

Basic log:

git log

Compact one-line per commit:

git log --oneline

Show graph of branches and merges:

git log --graph

Decorate with branch and tag names:

git log --decorate

Combine all (graph, oneline, decorate):

git log --oneline --graph --decorate

Limit the number of commits:

git log -n 5

Show commits by a specific author:

git log --author="Author Name"

Filter commits by date:

git log --since="2 weeks ago"
git log --after="2024-01-01" --before="2024-12-31"

View changes made in each commit:

git log -p

git blame – Who Changed What, When

git blame shows who last modified each line of a file. It’s useful for tracking down when a line of code was added or changed.

Basic usage:

git blame <file>

Ignore whitespace changes:

git blame -w <file>

Blame a file as of a specific commit:

git blame <commit_hash> -- <file>

git bisect – Find the Bug

git bisect helps you find the commit that introduced a bug by using binary search between a known good and bad commit.
Steps:

Start bisect:

git bisect start

Mark the bad (buggy) commit:

git bisect bad

Mark the good (working) commit:

git bisect good <commit_hash>

Git checks out a commit halfway between. You test it, and mark it as:

git bisect good
git bisect bad

Continue until Git pinpoints the exact commit that introduced the bug.
Exit bisect mode:

git bisect reset

git reflog – Recover Lost Commits

The reflog records all movements of your HEAD — even commits not visible in git log. If you've lost a branch, a commit, or reset hard by mistake, git reflog can rescue you.

Show all recent HEAD changes:

git reflog

Restore a lost commit: Find its hash in the reflog and reset or checkout:

git checkout <commit_hash>

Recover a deleted branch: If you deleted a branch and want to restore it:

git checkout -b <branch_name> <commit_hash_from_reflog>

Undo a hard reset:

git reset --hard <commit_hash_from_reflog>

Submodules and Subtrees

Both let you include one Git repository inside another.

🧩 Git Submodules

Track external repositories as subfolders.
You need to init/update them manually.

git submodule add <repo_url> <path>
git submodule init
git submodule update

🌳 Git Subtrees

Merge another repository directly into a subdirectory of your project.
Easier to manage compared to submodules.

git subtree add --prefix=<dir> <repo_url> <branch> --squash

To update later:

git subtree pull --prefix=<dir> <repo_url> <branch> --squash

More Git magic coming soon — stay tuned for advanced workflows, GitOps, and real-world use cases!

Mastering Linux File Permissions: The Ultimate Guide.

DevOps Man — Tue, 15 Apr 2025 08:59:28 +0000

Linux file permissions can feel cryptic at first, but once you break them down, they’re not only simple — they’re powerful. Whether you're building systems, writing scripts, or deploying code, understanding how to control file access is a must-have skill. This guide will walk you through everything you need to know.

Why File Permissions Matter

Linux is a multi-user system. Proper permissions:

Keep your system secure
Prevent accidental or malicious changes
Allow collaboration without compromising safety

Understanding rwx and User Classes
Permissions are shown as:

-rwxr-xr--

Breakdown:

First char: - (file) or d (directory)
Next 3: rwx for User (owner)
Next 3: r-x for Group
Last 3: r-- for Others

User Classes

u = User (owner)
g = Group
o = Others
a = All (u+g+o)

Permission Types

r = Read
w = Write
x = Execute
-= No permission

Viewing File Permissions

Using ls

ls -l /etc/passwd
-rw-r--r-- 1 root root 2871 Aug 22 14:43 /etc/passwd

Using stat

stat /etc/shadow
Access: (0640/-rw-r-----)  Uid: (0/root)   Gid: (42/shadow)

Modifying Permissions

Symbolic (Relative) Mode

chmod u+x file.txt        # Add execute for user
chmod g-w file.txt        # Remove write from group
chmod o-r file.txt        # Remove read from others
chmod a+r file.txt        # Add read for all

Octal (Absolute) Mode

User	Group	Other	Octal	Command	Meaning
rwx	rwx	rwx	777	`chmod 777 file.txt`	Full permissions to everyone
rwx	r-x	r-x	755	`chmod 755 file.txt`	Owner: all, Others: read/exec
rw-	r--	r--	644	`chmod 644 file.txt`	Owner: read/write, rest: read

Setting Permissions from Another File

chmod --reference=file1 file2

Recursive Permissions

chmod -R u+rw,o-rwx mydir/

Special Permissions

SUID (Set User ID)
Executes file with owner’s privileges.

chmod u+s file
chmod 4755 file

Example:

ls -l /usr/bin/umount
-rwsr-xr-x 1 root root 39144 /usr/bin/umount

SGID (Set Group ID)
Runs with group’s privileges, or maintains group ownership in directories.

chmod g+s dir
chmod 2750 dir

Sticky Bit
Only file owner can delete their files in shared directories.

chmod +t dir
chmod 1777 dir

Example:

ls -ld /tmp
drwxrwxrwt 10 root root 4096 /tmp

UMASK: Default Permissions

View Current UMASK

umask

Set New UMASK

umask 0022

How it Works
UMASK subtracts permissions from 666 (files) or 777 (dirs).

Ownership Commands

Change Owner

chown new_user file

Change Group

chgrp new_group file

Change Both

chown user:group file

Recursive Ownership Change

chown -R user:group dir

Bonus: File Attributes (Advanced Layer)

View Attributes

lsattr file

Change Attributes

sudo chattr +i file   # Make file immutable
sudo chattr -i file   # Make it editable again

Cheat Sheet Summary

chmod → change permissions
chown / chgrp → change ownership
umask → set default permissions
ls -l / stat → view permissions
+x, -w, a+r → symbolic changes
644, 755, 777 → octal changes
SUID, SGID, Sticky Bit → special bits

Conclusion

Linux permissions are a superpower once you understand the logic. Master these commands, practice regularly, and you’ll never get caught off-guard by a “Permission denied” again.

💥 Master Jenkins Pipelines in One Go – With a Universal Jenkinsfile Template

DevOps Man — Tue, 15 Apr 2025 07:51:13 +0000

Whether you're a DevOps engineer, backend developer, or automation enthusiast, mastering Jenkins pipelines is a must. In this guide, we’ll break down everything you need to know to write powerful, flexible Jenkins pipelines using Groovy — all in one place. You'll also get a universal Jenkinsfile that you can use across projects and tweak to your needs.

🚀 What is a Jenkinsfile?

A Jenkinsfile is a text file that contains the definition of a Jenkins pipeline and is checked into your source control (GitHub, GitLab, Bitbucket, etc). Instead of configuring jobs in the Jenkins UI, you define everything as code – which makes it version-controlled, repeatable, and collaborative.

🧠 Declarative vs Scripted Pipeline: What’s the Difference?

Feature	Declarative	Scripted
Syntax	Structured (block-based)	Freestyle (Groovy code)
Readability	Easier to read and write	More flexible, less readable
Groovy Knowledge	Minimal required	Moderate to advanced Groovy needed
Recommended For	Most users/projects	Complex, dynamic logic

📚 Groovy Concepts You Need to Know for Jenkins Pipelines

Jenkins pipelines are written in Groovy, but you only need a subset of it:

✅ Variables

def name = "Jenkins"

✅ String Interpolation

echo "Hello ${name}"

✅ Lists & Maps

def servers = ["web1", "web2"]
def config = [env: "prod", region: "us"]

✅ Conditionals & Loops

if (env == "prod") {
    echo "Production deploy"
}

servers.each { s -> echo "Deploying to ${s}" }

✅ Functions

def greet(name) {
    echo "Hi ${name}"
}

✅ Closures
Used behind the scenes with blocks like steps {} and stage {}.

🧱 Jenkins Declarative Pipeline Concepts (Explained)

pipeline { }
Root block that wraps everything.

agent
Defines where the pipeline runs (any, docker, specific label).

agent any

stages { } & stage { }
Defines the steps in your CI/CD lifecycle.

stages {
    stage('Build') {
        steps {
            echo "Building..."
        }
    }
}

steps { }
Actual shell commands or Jenkins steps.

steps {
    sh 'npm install'
    echo "Done"
}

environment
Set environment variables.

environment {
    NODE_ENV = "production"
    VERSION = "${env.BUILD_ID}"
}

when
Conditional execution of stages.

when {
    branch 'main'
}

post
Defines actions after success/failure/always.

post {
    success { echo "✔️ Success" }
    failure { echo "❌ Failed" }
    always  { cleanWs() }
}

parameters
Accept runtime input to your pipeline.

parameters {
    string(name: 'DEPLOY_ENV', defaultValue: 'staging', description: 'Choose env')
    booleanParam(name: 'RUN_TESTS', defaultValue: true, description: 'Run tests?')
}

credentials
Access stored secrets securely.

withCredentials([usernamePassword(credentialsId: 'git-creds', usernameVariable: 'USER', passwordVariable: 'PASS')]) {
    sh 'git push https://${USER}:${PASS}@repo.git'
}

tools
Preinstalled tools like JDK, Maven, etc.

tools {
    maven 'Maven 3.8.1'
}

script
For advanced Groovy logic inside declarative pipelines.

script {
    def result = "build_${env.BUILD_NUMBER}"
    echo result
}

🌐 Universal Jenkinsfile Template (Copy-Paste & Customize)

pipeline {
    agent any

    options {
        timeout(time: 15, unit: 'MINUTES')
        buildDiscarder(logRotator(numToKeepStr: '10'))
        skipDefaultCheckout()
    }

    parameters {
        string(name: 'DEPLOY_ENV', defaultValue: 'staging', description: 'Target environment')
        booleanParam(name: 'RUN_TESTS', defaultValue: true, description: 'Run tests before build')
    }

    environment {
        APP_NAME = "universal-app"
        BUILD_TAG = "${env.BUILD_NUMBER}-${new Date().format('yyyyMMdd-HHmm')}"
    }

    stages {
        stage('Init') {
            steps {
                echo "🚀 Starting pipeline for ${APP_NAME}"
                checkout scm
            }
        }

        stage('Install Dependencies') {
            steps {
                script {
                    if (fileExists('package.json')) {
                        echo "Node project detected"
                        sh 'npm install'
                    } else if (fileExists('requirements.txt')) {
                        echo "Python project detected"
                        sh 'pip install -r requirements.txt'
                    } else {
                        echo "Unknown project type"
                    }
                }
            }
        }

        stage('Test') {
            when {
                expression { return params.RUN_TESTS }
            }
            steps {
                script {
                    try {
                        sh 'npm test || echo "Tests skipped/failing gracefully"'
                    } catch (e) {
                        echo "Test failure ignored for now"
                    }
                }
            }
        }

        stage('Build') {
            steps {
                script {
                    echo "🛠️ Building ${APP_NAME} - ${BUILD_TAG}"
                    sh 'mkdir -p build && echo "Build successful" > build/status.txt'
                }
            }
        }

        stage('Deploy') {
            when {
                anyOf {
                    branch 'main'
                    expression { params.DEPLOY_ENV == 'production' }
                }
            }
            steps {
                echo "🚀 Deploying to ${params.DEPLOY_ENV}"
                sh './deploy.sh ${params.DEPLOY_ENV} || echo "Deploy script not found"'
            }
        }

        stage('Parallel Quality Checks') {
            parallel {
                stage('Lint') {
                    steps {
                        echo "🔍 Linting code..."
                        sh 'sleep 3'
                    }
                }
                stage('Security Scan') {
                    steps {
                        echo "🔐 Running security scan..."
                        sh 'sleep 3'
                    }
                }
            }
        }
    }

    post {
        success {
            echo "✅ Pipeline completed successfully"
        }
        failure {
            echo "❌ Pipeline failed"
        }
        always {
            echo "🧹 Cleaning up workspace"
            cleanWs()
        }
    }
}

🎯 Final Thoughts
With this article, you now:

Understand how Jenkins pipelines work
Know the key Groovy concepts used inside Jenkins
Can write and reuse a universal Jenkinsfile
Are ready to build, test, deploy, and automate like a DevOps pro 🚀

🧠 Ultimate Guide to Linux Networking Commands

DevOps Man — Mon, 14 Apr 2025 17:11:54 +0000

Master the Linux networking essentials with proper command usage, flag explanations, and common troubleshooting tips. Perfect for SysAdmins, DevOps engineers, and Linux learners.

🖧 Getting Network Interface Information

ifconfig – Interface Configuration (Deprecated but still useful)

ifconfig                   # Show info about active interfaces
ifconfig -a               # Show info about all interfaces (active/inactive)
ifconfig enp0s3           # Show info for specific interface

-a: Show all interfaces
Displays IPv4, MAC address, TX/RX packets, etc.

ip – Modern replacement for ifconfig

ip address show                  # Show all addresses
ip addr show dev enp0s3          # Show specific device
ip -4 address                    # IPv4 only
ip -6 address                    # IPv6 only
ip link show                     # Show MAC and Layer 2 info
ip link show dev enp0s3

address or addr: Shows IP info
link: Layer 2 (MAC level) info
-4: IPv4 only
-6: IPv6 only

🧭 Displaying and Setting Routing Information
route – Legacy command

route                    # Show routing table
route -n                 # Show numerical IPs only

-n: Don't resolve IP to hostnames

ip route – Modern replacement

ip route show                   # Show routing table
ip route add default via 192.168.0.1
ip route del default

🧰 Network Interface Control

# Bringing interfaces up/down
ifconfig enp0s3 down
ifconfig enp0s3 up

ip link set enp0s3 down
ip link set enp0s3 up

# Assigning IP addresses
ifconfig enp0s3 192.168.0.222/24 up
ip addr add 192.168.0.112/24 dev enp0s3
ip addr del 192.168.0.111/24 dev enp0s3

# Secondary IP using sub-interface
ifconfig enp0s3:1 10.0.0.1/24

# Gateway changes
route del default gw 192.168.0.1
route add default gw 192.168.0.2

ip route del default
ip route add default via 192.168.0.1

# MAC address change
ip link set dev enp0s3 address 08:00:27:51:05:a3

⚙️ Netplan – Static Network Config (Ubuntu)

# /etc/netplan/01-netcfg.yaml
network:
  version: 2
  renderer: networkd
  ethernets:
    enp0s3:
      dhcp4: false
      addresses:
        - 192.168.0.20/24
      gateway4: "192.168.0.1"
      nameservers:
        addresses:
          - "8.8.8.8"
          - "8.8.4.4"

sudo systemctl stop NetworkManager
sudo systemctl disable NetworkManager
sudo netplan apply

🛠️ Network Troubleshooting
Using ping

ping 8.8.8.8                  # Ping Google DNS
ping -c 5 192.168.0.1         # Send 5 packets
ping -i 0.5 google.com        # Interval of 0.5s

-c: Count
-i: Interval

🛠️ Troubleshooting Network Issues in Linux
When you're facing network issues, here's a step-by-step checklist to diagnose and fix them.
1️⃣ No Internet?

✅ Check Internet connectivity

ping 8.8.8.8

🛠️ If ping fails, check your gateway configuration.

2️⃣ DNS Not Resolving?

✅ Test DNS resolution

ping google.com

🔍 If this fails but 8.8.8.8 works, check DNS settings:

resolvectl status

3️⃣ Interface Down?

🔎 Check interface status

ip link show

🧰 Bring interface up (example: enp0s3)

ip link set enp0s3 up

🔐 SSH – Secure Shell

ssh -p 22 user@host                # Connect to host
ssh -l user -p 22 host
ssh -v -p 22 user@host             # Verbose mode

-p, --port
-l, --login-name
-v, --verbose

SSH Daemon Control:

sudo systemctl status sshd
sudo systemctl restart sshd
sudo systemctl enable sshd

📤 SCP – Secure Copy

scp a.txt user@host:~               # Upload file
scp -P 2222 file.txt user@host:~/   # With custom port
scp -r dir/ user@host:~/            # Recursive directory copy
scp -P 2222 user@host:~/a.txt .     # Download file

-P: Port
-r: Recursive

🔁 RSYNC – File Synchronization

rsync -av /src/ /dest/                       # Archive & verbose
rsync -av --delete /src/ /dest/              # Mirror
rsync -av -e ssh /src/ user@host:/dest/      # Over SSH
rsync -av --exclude '*.mp4' /src/ /dest/     # Exclude files

-a, --archive
-v, --verbose
--delete: Mirror deletions
-e ssh: Use SSH

🌐 WGET – Web Download Tool

wget https://file.com/sample.iso
wget -c https://file.com/sample.iso             # Resume
wget -P ~/Downloads/ https://file.com/sample    # Set dir
wget --limit-rate=100k -P dir/ URL              # Limit rate
wget -i urls.txt                                # Batch download
wget -b URL && tail -f wget-log                 # Background
wget --mirror --convert-links http://site.com   # Full site

-c, --continue
-P, --directory-prefix
--limit-rate=
-b, --background
-i, --input-file

📊 NETSTAT, SS, and LSOF

netstat -tupan          # TCP/UDP active ports
ss -tupan               # Faster alternative
lsof                    # List open files
lsof -u user            # By user
lsof -c sshd            # By command
lsof -iTCP -sTCP:LISTEN # TCP listening ports

-t, --tcp
-u, --udp
-a, --all
-p, --program

🔍 NMAP – Network Scanner

nmap -sS 192.168.0.1               # SYN scan
nmap -sT 192.168.0.1               # TCP connect
nmap -p- 192.168.0.1               # All ports
nmap -sV -p 22,80 192.168.0.1      # Version scan
nmap -O 192.168.0.1                # OS detection
nmap -A 192.168.0.1                # Full scan
nmap -iL hosts.txt -oN report.txt  # Input file + save

-sS: SYN scan
-sT: TCP connect
-sV: Version detection
-O: OS detection
-A: Aggressive scan
-iL: Input from file
-oN: Normal output file

🧠 Mastering Linux Processes: Viewing, Managing, and Killing Like a Pro 🧨

DevOps Man — Sat, 12 Apr 2025 11:56:06 +0000

Understanding and controlling processes is a key skill for every Linux user—especially for DevOps, system administrators, or anyone building backend systems. This guide breaks down everything you need to know about processes, their types, real-time monitoring, and how to kill them—with commands, outputs, flags, and scenarios.

🔍 What is a Process in Linux?
A process is an instance of a program that's being executed. Every command you run, service started, or script executed becomes a process.

🧬 Process Properties

PID: Process ID
PPID: Parent Process ID
UID/GID: User/Group ID of the owner
TTY: Terminal associated with the process
CMD: Command used to start the process
STATE: (e.g., R - Running, S - Sleeping, Z - Zombie)

⚙️ Process Types

Type	Description
Foreground	Runs in the terminal
Background	Runs behind the scenes (appended with `&`)
Daemon	System-level background service
Zombie	Terminated, but waiting for parent to read its exit status
Orphan	Parent exited before the process; adopted by `init`

🧵 Task vs Process

A task refers to a kernel-level thread or unit of scheduling.
A process may contain multiple tasks (threads).

🔍 Is It a Shell Built-in or Executable?

type cd     # => cd is a shell built-in
type rm     # => rm is /usr/bin/rm

This helps in understanding performance and behavior. Built-ins run faster and don’t spawn a new process.

👁️ Process Viewing Commands

ps – Snapshot of Processes

ps                 # Processes in current shell
ps -ef             # All processes (full format)
ps aux             # BSD style output
ps aux | less      # Paged view
ps aux --sort=%mem | less  # Sorted by memory
ps -ef --forest    # Tree view
ps -f -u username  # User-specific processes

pgrep – Search for Processes by Name

pgrep -l sshd     # Shows PID and name

pstree – ASCII Tree of Running Processes

pstree            # Tree structure
pstree -c         # Don’t merge branches with same name

🆚 ps vs pgrep vs pstree

Command	Use Case
`ps`	When you want full detail, CPU/mem usage
`pgrep`	When you need just PIDs (scripting, killing)
`pstree`	When visualizing parent-child process hierarchy

📈 Real-Time Process Monitoring: top and htop

top – Classic Real-Time Monitor

top

🔤 Key Shortcuts:

h: Help
1: CPU per core
m: Memory view
u: Filter by user
x/y: Highlight sort/running processes
< / >: Change sorting column
q: Quit

📥 Batch Mode Logging:

top -d 1 -n 3 -b > top_log.txt

Flag	Meaning
-d	Delay between refreshes
-n	Number of iterations
-b	Batch mode (non-interactive)

htop – Improved, Colorful, Interactive

sudo apt install htop
htop

🧭 Navigation:

Use arrow keys
F2 to configure columns
F5 for tree view
F9 to kill processes
F6 to sort

🆚 top vs htop

Feature	top	htop
Interface	Text-based	Colorful UI
Navigation	Keyboard	Interactive
Tree view	Manual	Built-in
Ease of use	Moderate	High

Use htop for interactive analysis, and top -b for logs in scripts or automation.

💀 Killing Processes

kill – Send Signal by PID

kill -l              # Lists all signals
kill 1234            # Default SIGTERM (15)
kill -9 1234         # SIGKILL (force kill)
kill -HUP 1234       # Reload (used in daemons)
kill -1 1234         # Same as above

💡 Common Signals

Signal	Number	Purpose
SIGHUP	1	Reload configuration
SIGINT	2	Interrupt (Ctrl+C)
SIGTERM	15	Graceful termination
SIGKILL	9	Force kill (cannot be ignored)

pkill – Kill by Process Name

pkill firefox
pkill -HUP nginx

killall – Kill All by Name

killall sleep

pidof + kill

kill -9 $(pidof nginx)

🆚 kill vs pkill vs killall

Command	Works On	Pros	When to Use
kill	PID	Precise, script-friendly	When you know the PID
pkill	Name	Pattern-based killing	Dynamic process names
killall	Full name	Kills all with exact match	Multiple instances of command

🎭 Background Job Management

sleep 100 &     # Background process
jobs            # List jobs
Ctrl + Z        # Pause foreground job
fg %1           # Resume in foreground
bg %1           # Resume in background

👻 nohup – Immune to Hangup (SIGHUP)

nohup long_command &

Even if the terminal closes, the command keeps running. Output is saved to nohup.out.

🧪 Real World Use Cases

Scenario	Command
Reload SSH server config	`kill -HUP $(pidof sshd)`
Gracefully stop a Python app	`pkill python`
Log processes every second	`top -b -d 1 -n 10 > usage.log`
Kill all background jobs	`killall sleep` or `jobs -p`
Prevent process kill on logout	`nohup python server.py &`

✅ Summary

Command	Purpose
ps	One-time process snapshot
pgrep	Find PID by name
pstree	View process hierarchy
top/htop	Real-time monitoring
kill	Terminate process by PID
pkill	Terminate process by name
killall	Kill all by command name
nohup	Run processes immune to SIGHUP

🦸‍♂️🛠️ The Avengers of Bash: grep, awk, sed, cut & tr Assembled! ⚡🐧💻

DevOps Man — Sat, 12 Apr 2025 11:11:18 +0000

🛠️ Bash Power Tools: grep, awk, sed, cut, and tr Explained with Comparison.
When working in Bash, you’ll come across several powerful text-processing tools. Each of them — grep, awk, sed, cut, and tr — has a unique role. Here's a deep dive into what each one does, when to use them, and how they compare.

📊 Text Processing Tools Comparison Table

Tool	Primary Use	Works On	Supports Patterns	Field/Column Support	Modifies Text	Best Use Case
`grep`	Search/filter lines with a pattern	Full lines	✅ Regex	❌ No	❌ No	Search for matching lines (logs, files)
`awk`	Field-wise processing + logic	Line-by-line	✅ Regex + conditions	✅ Yes	✅ Yes	Column-wise reporting, filtering, summaries
`sed`	Find & replace, stream editing	Line-by-line	✅ Regex	✅ Limited (via regex)	✅ Yes	Replace/edit text in streams/files
`cut`	Extract specific fields/characters	Line-by-line	❌ No	✅ Yes (`-d`, `-f`)	❌ No	Quickly extract columns by delimiter
`tr`	Translate/delete characters	Char-by-char	❌ No	❌ No	✅ Yes (chars)	Replace or remove specific characters

💡 When to Use What

Scenario	Use Tool	Why?
Find lines containing `"ERROR"` in logs	`grep`	Simple pattern match
Replace `"localhost"` with `"127.0.0.1"`	`sed`	Fast line/text replacement
Convert lowercase to uppercase in a file	`tr`	Char-level transformation
Extract usernames from `/etc/passwd`	`cut`	Fixed delimiter (:) and fields
Print names and emails from CSV, filter age > 25	`awk`	Column-based logic and filtering
Delete 2nd line from a file	`sed '2d'`	Line manipulation
Remove digits from a string	`tr -d '0-9'`	Clean unwanted characters

🧠 Quick Summary

Tool	Simple	Powerful	Script-Friendly	Supports Logic
`grep`	✅	⚠️ Only for matching	✅	❌
`cut`	✅	⚠️ Fixed format only	✅	❌
`tr`	✅	⚠️ Char-level only	✅	❌
`sed`	✅	✅	✅	⚠️ Basic logic
`awk`	⚠️	✅✅	✅	✅✅

✂️ cut (Extract fields)

echo "Alice,25,Developer,NY" | cut -d ',' -f 1
echo "Bob,30,Manager,LA" | cut -d ',' -f 2,3

🧠 awk (Fields + Logic)

echo "Eve,28,Developer,TX" | awk -F ',' '{ print $1, $3 }'
echo "Daniel,35,CEO,CA" | awk -F ',' '$2 > 30 { print $1 " is senior" }'

🔤 tr (Character transformations)

echo "developer" | tr 'a-z' 'A-Z'
echo "Remove123Digits" | tr -d '0-9'
echo "one:two:three" | tr ':' '-'

🧹 sed (Stream edit)

echo "Welcome dev!" | sed 's/dev/developer/'
echo "Line to delete" | sed '1d'
echo "TX" | sed 's/^/[Location] /'

🔍 grep – The Pattern Hunter

grep is used to search for lines that match a pattern. It's perfect for filtering logs, outputs, or data streams.

✅ Basic Examples with echo:

echo "Alice,25,Developer,NY" | grep "Developer"
# Output: Alice,25,Developer,NY

echo "Bob,30,Manager,LA" | grep "Developer"
# Output: (nothing, because it doesn't match)

echo "Charlie,22,intern,TX" | grep -i "Intern"
# Output: Charlie,22,intern,TX
# -i makes it case-insensitive

echo "Daniel,35,CEO,CA" | grep -v "Developer"
# Output: Daniel,35,CEO,CA
# -v inverts the match (shows non-matching lines)

🧠 Explanation:
grep "pattern" – Matches lines containing pattern.

-i – Case-insensitive match.
-v – Show only lines that don’t match the pattern.
-E – Use extended regular expressions (for advanced patterns).
-o – Show only the matching part of the line.

📌 Conclusion

Each of these Bash superheroes has a specialty. Combine them like Unix Avengers and you can process any kind of text or log stream like a pro!

👤 Linux User & Group Management

DevOps Man — Fri, 11 Apr 2025 10:27:10 +0000

Want to really understand how Linux handles users and groups? This hands-on guide not only gives you commands — it breaks them down flag by flag, so you know exactly what you're doing.

Grab your terminal and follow along. Ready? Let’s get started. 🚀

📁 Important Files — Know What Powers User Management

/etc/passwd   # User info: username:x:UID:GID:comment:home:shell
/etc/shadow   # Has (hashed) passwords
/etc/group    # Lists all groups

💡 Try:

cat /etc/passwd | head -n 3
sudo cat /etc/shadow | head -n 3
cat /etc/group | head -n 3

🛠️ Create a User with useradd

useradd [OPTIONS] username

🔑 Common Flags:

-m → Create a home directory (e.g., /home/username)
-d /custom/path → Set a custom home directory
-s /bin/bash → Set user’s login shell
-c "comment" → Add a comment (e.g., full name or role)
-G group1,group2 → Add to secondary groups (must already exist)
-g group → Set primary group (must already exist)

📌 Practical Example:

sudo useradd -m -d /home/john -c "C++ Developer" -s /bin/bash -G sudo,adm john

Explanation:

-m: Create /home/john
-d /home/john: Use this as the home directory
-c "C++ Developer": Store this comment
-s /bin/bash: Set Bash as login shell
-G sudo,adm: Add to secondary groups

➡️ Now set a password:

sudo passwd john

🔄 Modify a User with usermod

usermod [OPTIONS] username

Useful Flags:

-aG group1,group2 → Append user to new secondary groups
-s /new/shell → Change default shell
-d /new/home → Change home dir (-m to move files)
-c → Change comment

🔧 Example:

sudo usermod -aG docker,developers john

Explanation:

-a: Append (don’t remove existing groups!)
-G: Add to docker and developers groups

❌ Delete a User with userdel

sudo userdel [OPTIONS] username

Flags:

-r → Remove user’s home directory and mail spool Example:

sudo userdel -r john

Result: Deletes john + /home/john

👥 Group Management with groupadd, groupdel

➕ Add a Group:

sudo groupadd devops

❌ Delete a Group:

sudo groupdel devops

🔍 View All Groups:

cat /etc/group

🔍 Check Group Membership

groups           # For current user
groups username  # For another user
id username      # UID, GID, group list

🧑‍💼 Make a User Admin (Sudo Access)

On Ubuntu:
sudo usermod -aG sudo john

On CentOS:
sudo usermod -aG wheel john

🕵️ Monitor Logged-In Users

who -H       # Show who is logged in
w            # Who is logged in + what they’re doing
uptime       # Show load, users, system uptime
id           # Show UID, GID, and groups
whoami       # Show your effective user ID

📜 Login History

last             # Recent logins
last -u username # Logins by a specific user

✍️ Final Words

Linux user & group management is must-know stuff for sysadmins, DevOps engineers, and anyone using Linux daily.

🐧 Deep Dive into the Linux File System: A Beginner-Friendly Guide

DevOps Man — Fri, 11 Apr 2025 06:14:10 +0000

Linux is known for its powerful file system and command-line tools. Whether you're an aspiring DevOps engineer or just curious about Linux internals, understanding the file system is foundational.

In this guide, we'll explore everything from the Linux file hierarchy to file manipulation commands.

📁 1. What is the Linux File System?

The Linux file system is a structured arrangement for storing and organizing data. Everything in Linux is treated as a file, whether it’s a document, directory, or hardware device.

🌲 2. File Hierarchy System

Linux follows a hierarchical file structure starting from the root directory /.
/
├── bin/ → Essential user binaries
├── boot/ → Boot loader files
├── dev/ → Device files
├── etc/ → Configuration files
├── home/ → User directories
├── lib/ → Essential shared libraries
├── tmp/ → Temporary files
├── usr/ → User programs
├── var/ → Variable data (logs, spool)

🛣️ 3. Absolute vs Relative Path

📌 Absolute Path
An absolute path always starts from the root directory / and points to a file or directory regardless of the current working directory.

/home/username/docs/file.txt

📌 Relative Path
A relative path starts from the current directory (denoted by .) and is relative to where you are in the filesystem.

./docs/file.txt

🔗 4. Hard Links, Inodes, and Symbolic Links

📦 Inodes:
Every file has an inode storing metadata (permissions, size, timestamps, etc.)

🔁 Hard Link:
Creates another name pointing to the same inode.

ln original.txt hardlink.txt

🔗 Symbolic (Soft) Link:
Points to the file path, not the inode.

ln -s original.txt symlink.txt

🛠️ 5. Useful File System Commands

📄 ls – List Files

ls -l          # Long listing
ls -a          # Include hidden files

🕒 atime, mtime, ctime

atime: Last accessed
mtime: Last modified
ctime: Last changed (metadata)

Check using:

stat filename

Update time using:

touch filename

Check date/time:

date

🔃 Sorting Files

ls -lt      # Sort by modified time
ls -lSr     # Sort by size, reverse

🐈 File Viewing: cat, less, tail, head, watch

cat file.txt
less file.txt
tail -f file.txt        # Follow file
head -n 10 file.txt
watch -n 2 ls -l        # Refresh every 2s

📂 Create Files & Dirs: touch, mkdir

touch newfile.txt
mkdir newfolder

✂️ File Ops: cp, mv, rm, shred

cp file1.txt backup/
mv file.txt renamed.txt
rm unwanted.txt
shred -u sensitive.txt  # Secure delete

🔗 Pipes and Word Count: |, wc

cat file.txt | wc -l

➕ Redirection: >, >>, 2>, &>, cut, tee

echo "Hello" > file.txt       # Overwrite
echo "World" >> file.txt      # Append
command 2> error.log          # Errors only
command &> all.log            # Stdout + stderr
cut -d':' -f1 /etc/passwd
command | tee output.log      # View + save

🔍 which & plocate

which bash
plocate file.txt

🔎 find with exec

find . -name "*.log" -exec rm {} \;

🔍 grep – Search Text

grep "error" logfile.txt

🔤 strings – Extract printable text from binary

strings binaryfile

🔁 Compare Files: cmp, diff, sha256sum

cmp file1.txt file2.txt
diff file1.txt file2.txt
sha256sum file.txt

✍️ VIM Editor Basics

Open file:

vim file.txt

Common commands:

i: Insert mode
Esc: Exit insert
:w: Save
:q: Quit
:wq: Save and quit

📦 Archiving: tar and gzip

Create archive:

tar -cvf archive.tar folder/

Extract:

tar -xvf archive.tar

Compress:

gzip file.txt
gunzip file.txt.gz

🔚 Conclusion

The Linux file system is a powerful and flexible system, and the commands above will help you navigate and control it like a pro. If you're pursuing DevOps, mastering these tools is essential.

Understanding Ansible Configuration Priority

DevOps Man — Thu, 03 Apr 2025 15:15:13 +0000

When working with Ansible, it's crucial to understand how configuration settings are applied and in what order. Ansible allows multiple ways to configure its behavior, and these settings follow a specific priority order. This ensures flexibility while also allowing users to override defaults as needed.

Configuration Sources and Their Priority Order

Ansible configurations can be defined in various locations. The following list shows the priority order from lowest to highest, meaning settings at a higher level override those at a lower level:

Default Settings (Lowest Priority)

Ansible has built-in defaults that apply if no other configuration is specified.
These are hardcoded within Ansible and serve as fallback values.

System-Wide Configuration (/etc/ansible/ansible.cfg)

This configuration file is used for system-wide Ansible settings.
Any user on the system running Ansible will inherit these settings unless overridden.

User-Specific Configuration (~/.ansible.cfg)

This file allows a user to define their own Ansible settings.
It overrides the system-wide configuration but can still be overridden by more specific settings.

Project-Specific Configuration (ansible.cfg in the Current Directory)

If an ansible.cfg file exists in the working directory where the Ansible command is executed, it takes precedence over both system-wide and user-specific configurations.
This is useful for project-specific configurations where different settings may be required.

Environment Variables

Ansible allows configuration via environment variables, such as ANSIBLE_CONFIG.
If set, these variables override any file-based configurations.

Command-Line Options (Highest Priority)

The highest priority is given to command-line options specified when running Ansible.
For example, using ansible-playbook -i custom_inventory site.yml overrides inventory settings from configuration files.

Best Practices:

Use project-specific ansible.cfg to maintain consistency within a project.
Avoid modifying system-wide configuration unless necessary.
Use environment variables for temporary changes without modifying configuration files.
Always check the effective configuration using:
ansible-config dump --only-changed

Conclusion

Understanding Ansible’s configuration priority ensures that you apply settings in the most effective way. By following this hierarchy, you can control and override configurations efficiently, improving your automation workflows.

DEV Community: DevOps Man

Mastering CMake: A Practical Guide for DevOps Engineers and Developers

Table of Contents

Basic CMake Commands

Project Structure

Commonly Used Variables

Best Practices

Compiler Warnings

🚀 Common Compiler Flags (for GCC/Clang)

Configuring Files with configure_file

Unit Testing with Catch2

CMakeLists.txt

Compile Features and Definitions

Sanitizers

IPO & LTO

Generator Expressions

External Libraries

Useful CMake CLI Flags

Dependency Graph

Conclusion

Useful links:

🔧 GitLab CI/CD — The Complete Guide

🚀 Why GitLab CI/CD Over Jenkins and GitHub Actions?

Why choose GitLab CI/CD?

📁 Pipeline Basics

🗃️ Introduction to Database Design: Concepts Every Beginner Should Know

📌 What is Data?

📌 What is a Database?

📌 What is a Relational Database?

📌 What is an Entity?

📌 What are Attributes?

📌 What is a Table?

📌 What are Entity Types and Attribute Types?

📌 What is DBMS and RDBMS?

📌 What is SQL?

📌 What are SQL Languages?

📌 Why Database Design?

📌 What are Schemas and Types?

📌 Naming Convention

📌 What are Data Types?

📌 What is Data Integrity and its Types?

📌 What are Atomic Values?

📌 What is Normalization?

🔗 Relationships in a Relational Database

Mastering Git: The Ultimate Developer's Guide to Version Control

What is Git? What is Version Control?

Branching and Merging

Adding Remotes (git remote add)

Pushing Changes (git push)

Pulling Changes (git pull)

Fetching Changes (git fetch)

Tracking Branches

Git Tags (git tag, git tag -a)

Reverting Changes (git revert, git reset)

Difference Between reset, revert, checkout

Rewriting History

Cleaning Up (git clean)

Detached HEAD

git log with Options

git blame – Who Changed What, When

git bisect – Find the Bug

git reflog – Recover Lost Commits

Submodules and Subtrees

🧩 Git Submodules

🌳 Git Subtrees

Mastering Linux File Permissions: The Ultimate Guide.

Why File Permissions Matter

Viewing File Permissions

Modifying Permissions

Octal (Absolute) Mode

Special Permissions

UMASK: Default Permissions

Ownership Commands

Bonus: File Attributes (Advanced Layer)

Cheat Sheet Summary

Conclusion

💥 Master Jenkins Pipelines in One Go – With a Universal Jenkinsfile Template

🚀 What is a Jenkinsfile?

🧠 Declarative vs Scripted Pipeline: What’s the Difference?

📚 Groovy Concepts You Need to Know for Jenkins Pipelines