DEV Community: Adarsh BP

Designing Zero-Trust CI/CD Pipelines with GitHub Actions and AWS

Adarsh BP — Mon, 09 Feb 2026 10:04:22 +0000

TL;DR
Stop storing static cloud secrets in CI/CD. Use GitHub OIDC + AWS STS to authenticate using identity, not passwords.

Why This Matters (A Real Problem)

For years, CI/CD pipelines have relied on static cloud credentials:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

Even when stored in secret managers, these credentials create long-lived attack surfaces.

What usually goes wrong?

Secrets accidentally committed to GitHub
Leaked through logs or misconfigured pipelines
Keys valid for months or years
Manual rotation (often forgotten)

A leaked AWS key in a public repo can be exploited within minutes by automated bots.

This is why the industry is moving toward identity-based security.

The Shift: Credentials → Identity

Traditional security answers:

“Do you have the key?”

Modern security asks:

“Who are you, and can you prove it right now?”

This is the foundation of workload identity federation.

Airport security analogy ✈️

Old Model	New Model
Permanent key	Passport (identity)
Unlimited access	Boarding pass (temporary)
Trust forever	Trust just-in-time

What Is OIDC?

OpenID Connect (OIDC) is an identity protocol built on OAuth 2.0.

In CI/CD:

GitHub becomes the Identity Provider
AWS becomes the Identity Verifier
Authentication happens using signed JWT tokens
No secrets are stored anywhere

🧠 High-Level Architecture

What’s Inside the OIDC Token?

The JWT issued by GitHub contains claims like:

Repository name
Organization
Branch or tag
Workflow reference

AWS verifies:

Token signature
Issuer (token.actions.githubusercontent.com)
Audience (sts.amazonaws.com)
Repository & branch constraints

Why This Is More Secure

🔥 Short-Lived Credentials

Tokens expire in minutes
Stolen tokens are useless almost immediately

🎯 Granular Access Control

Restrict access to:
- A specific repo
- A specific branch
- A specific environment

🔄 Automatic Rotation

New credentials per job
Zero operational overhead

📜 Superior Auditability

Every action is traceable in AWS CloudTrail
Tied back to a specific GitHub workflow run

🛠 Implementation Guide

Step 1: Create an OIDC Identity Provider in AWS

In AWS IAM:

Identity providers → Add provider
Provider type: OpenID Connect
Provider URL:

  https://token.actions.githubusercontent.com

Audience:

  sts.amazonaws.com

This allows AWS to trust GitHub-issued tokens.

Step 2: Create a Secretless IAM Role

Attach this trust policy to the role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:ORG/REPO:ref:refs/heads/main"
        }
      }
    }
  ]
}

🔐 This enforces Zero Trust:

Even if someone knows the Role ARN, they cannot assume it unless identity claims match.

Step 3: Update GitHub Actions Workflow

name: OIDC Test

on: [push]

permissions:
  id-token: write   # Required for OIDC
  contents: read

jobs:
  aws-identity-test:
    runs-on: ubuntu-latest
    steps:
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::<ACCOUNT_ID>:role/GitHub-OIDC-Role
          aws-region: us-east-1

      - name: Verify Identity
        run: aws sts get-caller-identity

✅ No secrets
✅ No vaults
✅ No key rotation

Key Benefits Summary

Benefit	Impact
No static secrets	Zero credential leaks
Short-lived tokens	Minimal blast radius
Automatic rotation	No ops burden
Fine-grained trust	True Zero Trust
Full audit trail	Better compliance

Why This Is Mandatory

Industry data consistently shows:

Stolen credentials are a top breach vector
Breaches take months to detect
Cloud misuse leads to massive financial damage

Static credentials are now a security anti-pattern.

Secretless CI/CD is no longer a “nice to have”.
It’s a baseline security requirement.

Credits & Thanks

Huge thanks to Arun Santhosh R.A. for his excellent write-up on secretless CI/CD and workload identity federation.
This post builds upon his ideas and aims to make them more accessible to the broader community.

Final Thoughts

OIDC-based authentication between GitHub and AWS gives you:

Strong identity guarantees
Zero secret sprawl
Safer pipelines
Happier security teams

If your CI/CD pipeline still uses long-lived cloud credentials, now is the time to upgrade.

The Problems with dotenv and How dotenvx Solves Them

Adarsh BP — Thu, 27 Jun 2024 03:31:23 +0000

Managing environment variables is crucial but can be fraught with challenges. The traditional dotenv approach, while popular, has notable shortcomings:

Leaking Your .env File: This is the most significant risk, potentially exposing sensitive information.
Juggling Multiple Environments: Managing different configurations for development, testing, and production can be cumbersome.
Inconsistency Across Platforms: Behavior can vary depending on the operating system or environment.

Introducing dotenvx: A Comprehensive Solution

dotenvx addresses these issues effectively with three key features: Run Anywhere, Multiple Environments, and Encryption.

1. Run Anywhere: Consistency Across Platforms
dotenvx ensures consistent behavior across all languages, frameworks, and platforms. By using the command dotenvx run -- your-cmd, you can inject your environment variables at runtime, ensuring uniformity.

Example:

$ echo "Name=Adarsh" > .env
$ echo "console.log('Name' + process.env.Name)" > index.js

$ node index.js
Name undefined # without dotenvx

$ dotenvx run -- node index.js
Name Adarsh # with dotenvx

This consistency means your Python, Node.js, and Rust applications will behave the same way when using dotenvx. Install dotenvx via npm, brew, curl, docker, Windows, and more.

2. Multiple Environments: Simplified Environment Management
Managing multiple environments is straightforward with dotenvx. Create different .env files for each environment and use the -f flag to specify which one to load.

Example:

$ echo "HELLO=production" > .env.production
$ echo "console.log('Hello ' + process.env.HELLO)" > index.js

$ dotenvx run -f .env.production -- node index.js
[dotenvx][info] loading env (1) from .env.production
Hello production

You can also compose multiple environments by using multiple -f flags:

$ echo "HELLO=local" > .env.local
$ echo "HELLO=World" > .env
$ echo "console.log('Hello ' + process.env.HELLO)" > index.js

$ dotenvx run -f .env.local -f .env -- node index.js
[dotenvx] injecting env (1) from .env.local, .env
Hello local

This flexibility cleanly solves the problem of juggling multiple environments.

3. Encryption: Securing Your .env Files
The most groundbreaking feature of dotenvx is the ability to encrypt your .env files with a single command, significantly enhancing security.

Example:

$ dotenvx encrypt
✔ encrypted (.env)
#/-------------------[DOTENV_PUBLIC_KEY]--------------------/
#/            public-key encryption for .env files          /
#/       [how it works](https://dotenvx.com/encryption)     /
#/----------------------------------------------------------/
DOTENV_PUBLIC_KEY="03f8b376234c4f2f0445f392a12e80f3a84b4b0d1e0c3df85c494e45812653c22a"

# Database configuration
DB_HOST="encrypted:BNr24F4vW9CQ37LOXeRgOL6QlwtJfAoAVXtSdSfpicPDHtqo/Q2HekeCjAWrhxHy+VHAB3QTg4fk9VdIoncLIlu1NssFO6XQXN5fnIjXRmp5pAuw7xwqVXe/1lVukATjG0kXR4SHe45s4Tb6fEjs"
DB_PORT="encrypted:BOCHQLIOzrq42WE5zf431xIlLk4iRDn1/hjYBg5kkYLQnL9wV2zEsSyHKBfH3mQdv8w4+EhXiF4unXZi1nYqdjVp4/BbAr777ORjMzyE+3QN1ik1F2+W5DZHBF9Uwj69F4D7f8A="
DB_USER="encrypted:BP6jIRlnYo5LM6/n8GnOAeg4RJlPD6ZN/HkdMdWfgfbQBuZlo44idYzKApdy0znU3TSoF5rcppXIMkxFFuB6pS0U4HMG/jl46lPCswl3vLTQ7Gx5EMT6YwE6pfA88AM77/ebQZ6y0L5t"
DB_PASSWORD="encrypted:BMycwcycXFFJQHjbt1i1IBS7C31Fo73wFzPolFWwkla09SWGy3QU1rBvK0YwdQmbuJuztp9JhcNLuc0wUdlLZVHC4/E6q/K7oPULNPxC5K1LwW4YuX80Ngl6Oy13Twero864f2DXXTNb"
DB_NAME="encrypted:BGtVHZBbvHmX6J+J+xm+73SnUFpqd2AWOL6/mHe1SCqPgMAXqk8dbLgqmHiZSbw4D6VquaYtF9safGyucClAvGGMzgD7gdnXGB1YGGaPN7nTpJ4vE1nx8hi1bNtNCr5gEm7z+pdLq1IsH4vPSH4O7XBx"

# API Keys
API_KEY="encrypted:BD9paBaun2284WcqdFQZUlDKapPiuE/ruoLY7rINtQPXKWcfqI08vFAlCCmwBoJIvd2Nv3ACiSCA672wsKeJlFJTcRB6IRRJ+fPBuz2kvYlOiec7EzHTT8EVzSDydFun5R5ODfmN"
STRIPE_API_KEY="encrypted:BM6udWmFsPaBzlND0dFBv7R55JiaA+cZnbun8DaVNrEvO+8/k+lsXbZQ0bCPks8kUsdD2qrSp/tii0P8gVJ/gp+pdDuhdcJj91hxJ7nzSFf6h0ofRb38/2WHFhxg77XExxzui1s3w42Z"

# Logging
LOG_LEVEL="encrypted:BKmgv5E7/l1FnSaGWYWBPxxagdgN+KSEaB+va3PePjwEp7CqW6PlysrweZq49YTB5Fbc3UN/akLVn1RZ2AO4PyTVqgYYGBwerjpJiou9R2KluNV3T4j0bhsAkBochg3YpHcw3RX/"

dotenvx generates a DOTENV_PUBLIC_KEY for encryption and a DOTENV_PRIVATE_KEY for decryption using public-key cryptography. This means even if your .env file is leaked, the information remains secure without the decryption key.

Conclusion

dotenvx significantly improves the management of environment variables by addressing the three major issues with the traditional dotenv approach. With consistent behavior across platforms, easy management of multiple environments, and robust encryption, dotenvx sets a new standard for configuration management.

Head over to the official documentation of dotenvxfor detailed example and guide

Mastering Kubernetes Readiness Probes: A Comprehensive Guide

Adarsh BP — Fri, 19 Apr 2024 16:59:39 +0000

In the dynamic landscape of Kubernetes (K8s) orchestration, ensuring the reliability and availability of applications is paramount. One indispensable tool in achieving this is the Readiness Probe. In this guide, we'll delve deep into understanding, implementing, and optimizing Readiness Probes in Kubernetes.

What is a Readiness Probe in Kubernetes?

Imagine an application within your Kubernetes cluster that requires some initialization tasks before it can effectively handle incoming traffic. These tasks could range from loading data to establishing connections with external services. Deploying such an application without ensuring it's fully ready might lead to errors or performance degradation. This is precisely where Readiness Probes come into play.

A Readiness Probe in Kubernetes serves as a health check mechanism. It allows you to define conditions that Kubernetes will use to determine if a container is ready to receive traffic. Typically, this involves checking a specific TCP port endpoint or an HTTP request that the container should respond to successfully.

Distinguishing Between Startup, Readiness, and Liveness Probes

Understanding the differences between these probes is essential for effectively managing container lifecycles:

Startup Probes: Ideal for legacy applications that require an extended duration for initialization. These probes are executed once during the initialization phase.
Readiness Probes: Periodically check the container's readiness to serve traffic throughout its lifecycle. They ensure that traffic is directed only to containers capable of handling requests.
Liveness Probes: Determine if a container is still running and functioning correctly. They are crucial for detecting and recovering from unresponsive or error-prone containers.

When to Utilize Readiness Probes?

Readiness Probes are indispensable in various scenarios:

Initialization Tasks: Ensure containers are fully prepared to handle requests after completing initialization tasks like loading configurations or establishing database connections.
Microservices Orchestration: Coordinate the availability of different components within stateful applications to prevent premature interactions.
Scaling and Updates: Facilitate smooth scaling and rolling updates by delaying traffic to pods until they're ready.

Implementing Readiness Probes: Best Practices and Strategies

Now, let's explore best practices for configuring and optimizing Readiness Probes:

Define for All Containers: Ensure readiness probes are defined for all containers within your pods for granular control.
Choose the Right Probe Type: Based on your application's nature, opt for HTTP, TCP, or Command probes.
Configure Appropriately: Set initialDelaySeconds to accommodate container startup time and adjust periodSeconds based on probe frequency.
Design Lightweight Endpoints: Create dedicated lightweight endpoints within your application specifically for readiness checks.
Regular Reviews and Testing: Continuously review and optimize probe configurations in non-production environments, adjusting parameters as needed.

How to configure readiness probes?

Let’s take a look at an example YAML configuration for each type of Readiness probe.

HTTP readiness probe example

In this example we have a container running a web server, and the readiness probe is configured to check if the web server is responding with an HTTP status code of 200 on the “/testpath” endpoint.

httpGet specifies an HTTP check.
path is set to “/testreadinesspath,” which is the endpoint where the readiness check will be performed.
port is set to 8080, which corresponds to the container’s port.
initialDelaySeconds specifies that the probe should start 15 seconds after the container starts.
periodSeconds specifies that the probe will be repeated every 10 seconds after the initial delay.

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
    - name: ex-container
      image: ex-image
      ports:
        - containerPort: 8080
      readinessProbe:
        httpGet:
          path: /testreadinesspath
          port: 8080
        initialDelaySeconds: 15
        periodSeconds: 10

TCP readiness probe example

With this configuration, Kubernetes will periodically check if the container is able to accept TCP connections on port 8080.

tcpSocket specifies a TCP check.
port is set to 8080, matching the container’s port.
initialDelaySeconds specifies that the probe should start 15 seconds after the container starts.
periodSeconds specifies that the probe will be repeated every 10 seconds after the initial delay.

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
  - name: ex-container
    image: ex-image
    ports:
    - containerPort: 8080
    readinessProbe:
      tcpSocket:
        port: 8080
      initialDelaySeconds: 15
      periodSeconds: 10

Command readiness probe example

In this example, the readiness probe is configured to run a custom command to run a script inside the container. Inside the container, the custom script should be responsible for performing the readiness check. The script can return a non-zero exit code if the readiness check fails and a zero exit code if it succeeds.

exec specifies a Command check.
Command is an array of commands to run inside the container. In this example, we run a shell script named “check-script.sh.”
initialDelaySeconds specifies that the probe should start 20 seconds after the container starts.
periodSeconds specifies that the probe will be repeated every 15 seconds after the initial delay.


apiVersion: v1
kind: Pod
metadata:
  name: my-app-pod
spec:
  containers:
  - name: my-app-container
    image: my-app-image
    ports:
    - containerPort: 80
    readinessProbe:
      exec:
        command:
        - /bin/sh
        - -c
        - check-script.sh
      initialDelaySeconds: 20
      periodSeconds: 15

Addressing Common Readiness Probe Failures

Several issues may arise when implementing readiness probes:

Long Startup Time: Adjust initialDelaySeconds and optimize container startup processes.
Dependency Readiness: Ensure external dependencies are ready before the container starts.
Misconfiguration: Double-check readiness probe configuration to rectify any errors.
Application Bugs: Debug and resolve application issues affecting readiness.
Resource Constraints: Adjust resource limits to provide necessary resources to containers.
Probe Interference: Avoid conflicts between liveness and readiness probes through proper configuration.
Cluster Issues: Monitor and address Kubernetes cluster issues affecting probe functionality.

Conclusion

Mastering readiness probes is essential for ensuring the stability and availability of applications within Kubernetes clusters. By following best practices and addressing common failures, you can optimize the performance of your applications and maintain a robust Kubernetes environment. Incorporate readiness probes alongside startup and liveness probes to create a resilient and efficient deployment strategy.

Deciphering Open-Source Licenses: An In-Depth Exploration

Adarsh BP — Wed, 27 Mar 2024 10:14:23 +0000

In the realm of software development, open-source licenses serve as the bedrock for collaboration, innovation, and the free exchange of ideas. These licenses are not merely legal jargon; they are the blueprint that delineates how software can be utilized, modified, and shared within the developer community and beyond. In this article, we embark on a comprehensive journey to unravel the nuances of open-source licenses, dissecting their types, characteristics, and providing illustrative examples to illuminate their significance.

Understanding Open-Source Licenses:

Open-source licenses are legal frameworks governing the usage, distribution, and modification of software. These licenses are designed to uphold the principles of openness, transparency, and collaboration, empowering developers to leverage, study, modify, and distribute software freely. They typically encompass the following fundamental freedoms and responsibilities:

Freedom to Use: Users are granted the liberty to utilize the software for any purpose, whether it be for personal, commercial, or educational endeavors.
Freedom to Study: Access to the source code enables users to delve into the inner workings of the software, fostering a deeper understanding and facilitating learning.
Freedom to Modify: Open-source licenses empower users to modify the source code to tailor the software to their specific requirements or to rectify any deficiencies.
Freedom to Distribute: Users have the prerogative to distribute the software and its modified versions, fostering collaboration and the dissemination of knowledge.

Categories of Open-Source Licenses:

Open-source licenses span a spectrum of categories, each delineating distinct rights and obligations. Here are the primary types of open-source licenses:

1 >> Permissive Licenses:

Permissive licenses, often referred to as "BSD-style" or "MIT-style" licenses, embody a laissez-faire approach, offering maximal freedom with minimal restrictions. These licenses typically necessitate the retention of the original copyright notice and disclaimer in derivative works. Examples include:

MIT License: Renowned for its simplicity and permissiveness, the MIT License grants users carte blanche to utilize the software for any purpose, provided that the original copyright notice is retained.
Example: React.js, Node.js
BSD License (2-Clause and 3-Clause): BSD licenses share similarities with the MIT License but include a warranty disclaimer. They find widespread adoption in academic and industrial spheres.
Example: FreeBSD, SQLite

Detailed example : Permissive licenses are like giving someone a gift with very few strings attached. They allow users to use, modify, and distribute the software almost however they want, with minimal obligations. Think of it as borrowing a book from a library and being told you can keep it for as long as you want and even write your own stories in it, as long as you acknowledge the original author.

2 >> Copyleft Licenses:

Copyleft licenses, epitomized by the GNU General Public License (GPL), espouse a philosophy of reciprocity, mandating that derivative works be licensed under the same terms. These licenses aim to safeguard the openness of software and ensure that modifications are perpetually accessible. Examples include:

GNU General Public License (GPL): Among the most prevalent copyleft licenses, the GPL stipulates that any derivative works derived from GPL-licensed software must also be distributed under the GPL.
Example: Linux Kernel, GNU Compiler Collection (GCC)
GNU Lesser General Public License (LGPL): A variant of the GPL, the LGPL permits the linking of proprietary software with open-source libraries, offering greater flexibility.
Example: Qt, GTK+

Detailed example :Copyleft licenses are like a potluck dinner where everyone contributes and benefits. They require that any modified or derivative works also be shared under the same open-source terms. It's like agreeing to share your secret recipe at a potluck, but only if everyone else agrees to share their recipes too. If someone makes changes to your recipe and shares it with others, they must also share their version under the Copyleft licenses . It's like saying, "Sure, you can use my recipe, but if you change it and share it, you have to let others use it freely too."

3 >> Strongly Protective Licenses:

Strongly protective licenses impose more stringent restrictions on usage and distribution, ensuring the preservation of software freedoms. Examples include:

Mozilla Public License (MPL): The MPL is a hybrid license that amalgamates elements of copyleft and permissive licenses, enabling the integration of MPL-licensed code with proprietary software.
Example: Mozilla Firefox, Thunderbird
Affero General Public License (AGPL): A derivative of the GPL, the AGPL extends its purview to software distributed over a network, compelling users to disclose modifications made to AGPL-licensed software accessible via a network.
Example: GitLab, Nextcloud

Detailed example :Strongly protective licenses are like setting boundaries to protect something valuable. They impose more restrictions compared to permissive licenses, ensuring that the software remains open-source and accessible. It's like putting a lock on your bike to prevent anyone from taking it without permission.Imagine you developed a tool and released it under the MPL. This license allows others to use and modify your tool, but if they make changes and distribute it, they must release those changes under the MPL too. It's like saying, "You can use my tool and make it better, but if you share those improvements, they have to remain open-source like the original."

Selecting the Appropriate License:

Choosing the apt open-source license for a project necessitates careful consideration of various factors, including project objectives, community dynamics, commercial implications, and legal ramifications. Here are some pivotal considerations:

Project Goals: Determine whether the aim is to foster widespread adoption and collaboration (permissive licenses) or to safeguard the integrity and openness of the software (copyleft licenses).
Community Dynamics: Take into account the prevailing preferences and expectations of the open-source community, as well as the compatibility with existing licenses of dependencies or related projects.
Commercial Considerations: Assess the impact on commercial utilization and proprietary integration, opting for licenses that strike a balance between openness and commercial viability.
Legal Guidance: Seek legal counsel to ensure compliance with licensing requirements and gain insights into the repercussions of different licensing choices on the project's trajectory.

Conclusion:

Open-source licenses serve as the linchpin of collaborative software development, underpinning innovation, inclusivity, and knowledge-sharing within the global developer ecosystem. Whether it's the unfettered liberty of the MIT License, the egalitarian ethos of the GPL, or the nuanced provisions of the MPL, the choice of license profoundly influences the trajectory and ethos of a software project. By comprehending the intricacies of open-source licenses and making informed decisions, developers can cultivate an environment conducive to innovation, collaboration, and collective progress in the ever-evolving landscape of software development.

Mastering Docker: A Comprehensive Guide to Docker Commands

Adarsh BP — Thu, 21 Mar 2024 17:00:42 +0000

Docker has revolutionized the way developers build, ship, and run applications by providing a consistent environment across different machines. Whether you're a seasoned Docker user or just getting started, mastering Docker commands and best practices is essential for efficient containerization and management of your applications. In this guide, we'll explore a comprehensive list of Docker commands and best practices to help you navigate the Docker ecosystem effectively.

General Commands

Viewing Docker Information

docker version: Provides detailed information about your Docker CLI and daemon versions.
docker system info: Lists data about your Docker environment, including active plugins, containers, and images.
docker help: Displays the help index, a reference of all supported commands.
docker <command> --help: Shows help information about a particular command, including detailed information on supported option flags.

Building Images

Building Docker Images from Dockerfiles

docker build .: Builds the Dockerfile in your working directory into a new image.
docker build -t example-image:latest .: Builds the Dockerfile and tags the resulting image as example-image:latest.
docker build -f docker/app-dockerfile: Builds the Dockerfile at a specific path.
docker build --build-arg foo=bar .: Sets a build argument while building an image.
docker build --pull .: Instructs Docker to pull updated versions of images referenced in FROM instructions before building.
docker build --quiet .: Builds an image without emitting any output during the build process.

Creating Images

docker commit <container> new-image:latest: Saves the current state of a container to a new image.

Running Containers

Managing Containers

docker run example-image:latest: Runs a new container using the specified image.
docker run --rm example-image:latest: Automatically removes the container when it exits.
docker run -d example-image:latest: Detaches your terminal from the running container, leaving it in the background.
docker run -it example-image:latest: Attaches your terminal’s input stream and TTY to the container for interactive commands.
docker run --name my-container example-image:latest: Names the new container.
docker run --hostname my-container example-image:latest: Sets the container’s hostname.
docker run --env foo=bar example-image:latest: Sets environment variables inside the container.
docker run -p 8080:80 example-image:latest: Binds ports between the host and container.
docker run -v /host-directory:/container-directory example-image:latest: Sets up volume mounts.
docker run --network my-network example-image:latest: Connects the container to a Docker network.

Inspecting Containers

docker inspect <container>: Retrieves detailed information about a container in JSON format.

Pausing and Unpausing Containers

docker pause <container> and docker unpause <container>: Pauses and unpauses the processes running within a container.

Restart Policies

docker run --restart unless-stopped example-image:latest: Sets the container to start automatically when the Docker daemon starts, unless manually stopped. Other restart policies are also supported.

Managing Running Containers

docker ps: Lists all running containers.
docker ps -a: Lists all containers, including stopped ones.
docker attach <container>: Attaches your terminal to a container’s foreground process.
docker kill <container>: Sends a SIGKILL signal to force stop a container.
docker stop <container> and docker start <container>: Stops and starts a container.
docker rm <container>: Deletes a container.

Copying to and from Containers

Copying Files

docker cp example.txt my-container:/data: Copies a file from the host to a container.
docker cp my-container:/data/example.txt /demo/example.txt: Copies a file from a container to the host.

Executing Commands in Containers

Running Commands

docker exec my-container demo-command: Runs a command inside a running container.
docker exec -it my-container demo-command: Runs a command interactively.

Accessing Container Logs

Viewing Logs

docker logs <container>: Streams log output from a container.
docker logs <container> --follow: Streams new logs as they’re stored.
docker logs <container> -n 10: Gets the last 10 logs from a container.

Managing Images

Working with Images

docker images: Lists all stored images.
docker rmi <image>: Deletes an image by its ID or tag.
docker tag <image> example-image:latest: Adds a new tag to an existing image.

Pulling and Pushing Images

Interacting with Registries

docker push example.com/user/image:latest: Pushes an image to a remote registry.
docker pull example.com/user/image:latest: Pulls an image from a remote registry.

Managing Networks

Configuring Networks

docker create network my-network: Creates a new network.
docker network connect <network> <container>: Connects a container to a network.
docker network disconnect <network> <container>: Removes a container from a network.
docker network ls: Lists all Docker networks.
docker network rm <network>: Deletes a network.

Managing Volumes

Handling Storage

docker volume create my-volume: Creates a new volume.
docker volume ls: Lists volumes on the host.
docker volume rm: Deletes a volume.

Using Configuration Contexts

Managing Docker Daemons

docker context create my-context --host=tcp://host:2376: Creates a new context.
docker context update <context>: Modifies a context’s configuration.
docker context ls: Lists available contexts.
docker context use <context>: Switches to a named context.
docker context rm <context>: Deletes a context.

Creating SBOMs

Generating Software Bill of Materials

docker sbom example-image:latest: Generates an SBOM for an image.
docker sbom example-image:latest --output sbom.txt: Saves an SBOM to a file.
docker sbom example-image:latest --format spdx-json: Generates an SBOM in a standard format.

Scanning for Vulnerabilities

Security Scanning

docker scan example-image:latest: Scans for vulnerabilities in an image.
docker scan example-image:latest --file Dockerfile: Provides detailed vulnerability information when the Dockerfile is available.
docker scan example-image:latest --severity high: Reports only high severity vulnerabilities.

Volume Management

Inspecting Volumes

docker volume inspect <volume>: Displays detailed information about a volume, including its mount point and configuration.

Docker Compose

Running Docker Compose

docker-compose up: Starts services defined in a Docker Compose file.
docker-compose down: Stops and removes containers, networks, and volumes defined in a Docker Compose file.

Docker Swarm

Initializing Docker Swarm

docker swarm init: Initializes a Docker Swarm on the current node.

Joining Nodes to Swarm

docker swarm join: Adds a node to an existing Docker Swarm as either a manager or worker.

Deploying Services in Swarm

docker service create: Creates a new service in a Docker Swarm.

Scaling Services

docker service scale <service>=<replicas>: Scales the number of replicas for a service in a Docker Swarm.

Listing Services

docker service ls: Lists all services in a Docker Swarm.

Inspecting Services

docker service inspect <service>: Displays detailed information about a service in a Docker Swarm.

Docker Network Operations

Inspecting Networks

docker network inspect <network>: Shows detailed information about a Docker network, including its configuration and connected containers.

Pruning Networks

docker network prune: Removes unused networks.

Managing Docker Contexts

Setting Default Context

docker context use default: Sets the default context for Docker commands.

Docker Hub Account

Interacting with Docker Hub

docker login: Logs in to your Docker Hub account.
docker logout: Logs out of your Docker Hub account.
docker search nginx: Searches Docker Hub for images.

Cleaning Up Unused Resources

Resource Management

docker system prune: Removes unused data.
docker system prune -a: Deletes all unused images.
docker system prune --volumes: Includes volume data in the prune process.
docker image prune, docker network prune, docker volume prune: Prunes unused images, networks, and volumes respectively.
docker system df: Reports total disk usage.

Docker Machine

Managing Docker Hosts

docker-machine create: Creates a new Docker host using Docker Machine.
docker-machine ls: Lists all Docker hosts created with Docker Machine.
docker-machine ssh <machine>: SSH into a Docker Machine.

Docker BuildKit

Using BuildKit

DOCKER_BUILDKIT=1 docker build .: Enables Docker BuildKit for building Docker images, providing advanced features like caching and parallel execution.

By mastering these Docker commands and best practices, you can streamline your containerization workflow, optimize resource utilization, and enhance the security of your Dockerized applications. Experiment with these commands in your development environment to become proficient in Docker management and deployment.

Getting hands-on with local LLMs using OLLAMA

Adarsh BP — Wed, 20 Mar 2024 03:41:23 +0000

This quick rundown explores Ollama and my experience with local Language Model Microservices (LLMs) and their uses in inference-based tasks. Among local LLM options, LMStudio was my first encounter, finding it easy to use. Yet, I've been intrigued by Ollama for its simplicity and adaptability.

In a nutshell, Ollama has its own collection of models that users can access. These models can be downloaded to your computer and interacted with through a simple command line. Alternatively, Ollama offers a server for inference, usually at port 11434, allowing interaction via APIs and libraries like Langchain.

Currently, Ollama offers a variety of models, including embedding models.

Getting started with OLLAMA

To start using Ollama, download the version for your operating system.

After installation, check if it's working by typing 'ollama', which should show a help menu.

ollama

To make use of any model, the initial step involves "pulling" it from Ollama, which is akin to downloading an image from Dockerhub (assuming you're acquainted with that platform) or a service like Elastic Container Registry (ECR). Additionally, Ollama comes pre-loaded with several default models, one of which is llama2, recognized as Facebook's open-source LLM.

To Download the model that would like to interact use the command to download , Assume if you want to download llama2

ollama pull llama2

You can also check the different version of model and download which is required according to your usecase.

Once the download is complete, you can check to see whether the model is available

ollama list

Now you can run the model in your local machine

ollama run llama2

Now you can ask any questions and the response speed will be based on your system configurations and model

The precision of the responses may not always be optimal, but this can be mitigated by selecting different models or employing techniques such as fine-tuning. Alternatively, implementing a RAG-like solution independently could enhance accuracy.

If you examine Llama's inference server, you'll notice that there are programmatic methods available to access it by connecting to port 11434.

Now this could be extended more to integrate with langchain and create a application based on your usecase.

Exploring 5 Docker Alternatives: Containerization Choices for 2024

Adarsh BP — Tue, 19 Mar 2024 05:22:29 +0000

In the realm of containerization, Docker has long been hailed as the go-to platform for developers. However, the landscape has evolved, presenting a plethora of alternatives tailored to various needs and preferences. In this comprehensive round-up, we'll delve into 5 Docker alternatives that offer diverse capabilities for building and deploying containers in 2024.

Is Docker Still the Best Choice in 2024?

While Docker remains a stalwart in the containerization domain, its supremacy is no longer unchallenged. Opting for an alternative tool can mitigate Docker's limitations, cater to specific requirements, and foster consistency across different environments. Whether it's circumventing the Docker daemon on host machines or ensuring uniform container technology usage from development to production, alternatives offer tailored solutions.

Dispelling the Myth: Using Containers Beyond Docker

Once synonymous with containerization, Docker's dominance has waned with the emergence of a more diversified ecosystem. Container basics are now standardized by the Open Container Initiative (OCI), facilitating interoperability among various tools. Consequently, Docker's exclusivity is dispelled, and alternatives can seamlessly handle existing container content, including images sourced from popular registries like Docker Hub.

Exploring the Top Docker Alternatives

Embarking on a journey to explore containerization options reveals a rich tapestry of tools, each addressing distinct use cases and functionalities. Here, we spotlight 5 noteworthy alternatives that encompass a spectrum of features and capabilities:

Podman

This open-source container engine is a lightweight and daemonless alternative to Docker. It utilizes containerd (which we'll discuss next) under the hood, offering a familiar Docker command-line experience for managing containers. Podman is ideal for those seeking a Docker-like experience without the requirement of a background daemon.

Podman’s CLI is Docker-compatible; most commands can be converted by simply replacing docker with podman, such as podman ps and podman run instead of docker ps and docker run. Additionally, Podman has a graphical desktop application called Podman Desktop, serving as an open-source alternative to Docker Desktop. This graphical interface simplifies the management of container resources, providing an intuitive experience without the need to learn complex command line syntax

Containerd and nerdctl

This CNCF (Cloud Native Computing Foundation) project is a lightweight container runtime. Unlike Docker, it focuses solely on runtime functionalities and doesn't handle image building or registry management. However, containerd is the runtime engine used by Docker itself, making it a powerful and secure foundation for containerization.

Nerdctl is deliberately designed to be fully Docker-compatible. Docker commands can be directly translated to their Nerdctl counterparts by simply replacing docker with nerdctl—try nerdctl build instead of docker build, for example. Docker Compose commands are supported too.

Setting up containerd and nerdctl is slightly more complex than just using Docker. However, this approach gives you more control over your container stack: you can easily replace the containerd runtime or nerdctl CLI in the future if you need to. It also allows you to access new containerd features that haven’t yet made it into Docker.

Rancher Desktop

This open-source desktop application provides a comprehensive container development environment. It bundles Docker along with Kubernetes, the renowned container orchestration platform, alongside other essential tools. This integrated approach offers a convenient one-stop solution for developers engaged in containerized application development. Rancher Desktop serves as an excellent choice for individuals seeking a user-friendly platform that streamlines the process of building and deploying containerized applications.

Buildah

Buildah is a tool specifically designed to streamline the process of building OCI-compliant container images. Unlike other container tools, it focuses solely on image creation and lacks features for running containers.

Ideal for those seeking a lightweight solution for image management, Buildah offers a daemonless architecture and a straightforward CLI. It seamlessly integrates into custom tooling environments, providing flexibility and ease of use. Additionally, Buildah enables direct interaction with OCI images, allowing for modifications such as adding supplementary content or executing additional commands.

Whether assembling images from existing Dockerfiles or through CLI commands, Buildah simplifies the image creation process. Furthermore, it offers the capability to mount filesystems generated during the build process onto the host system, facilitating convenient inspection of output image content.

LXC

Linux Containers (LXC) represents an OS-level containerization solution deeply integrated into Linux systems. Positioned between full virtualization and the lightweight application containers provided by OCI tools like Docker, LXC offers a unique approach to containerization.

LXC containers are characterized as system containers, encompassing a complete operating system within. This architecture allows for the installation of diverse software workloads within the container environment. Once created, LXC containers persist on the host system for the duration of their necessity. The management experience resembles that of traditional virtual machines, offering robust control and persistence.

In contrast, application or process containerization tools like Docker prioritize executing a single process within an ephemeral environment. These containers are typically short-lived, serving a specific task before terminating. While suitable for many modern development and cloud deployment scenarios, this model can prove restrictive for more intricate software requirements.

Choosing LXC over Docker may be preferable in situations demanding the execution of multiple workloads within containers, necessitating deeper access to the container's operating system, or favoring VM-like administration techniques. Notably, LXC does not directly support OCI containers. However, it is feasible to create an LXC container from an OCI image utilizing specialized templates.

Streamlining Software Development: The Power of .gitignore Templates

Adarsh BP — Thu, 14 Mar 2024 04:43:33 +0000

In the intricate world of software development, managing repositories efficiently is paramount. With the proliferation of diverse technology stacks, frameworks, and programming languages, developers often find themselves grappling with files that clutter repositories but serve no purpose in the shared codebase. Enter .gitignore templates – a developer's best friend in keeping repositories clean and focused.

One shining beacon in this realm is the Gitignore repository on GitHub. Boasting a staggering more than 150K stars and more than 80K forks, this repository is a treasure trove for developers seeking to declutter their projects. Managed and maintained actively, it offers an extensive collection of nearly 200 .gitignore templates, covering a wide array of languages and frameworks.

What is .gitignore file?

At its core, a .gitignore file serves a simple yet crucial purpose: it specifies files and directories that Git should ignore, preventing them from being tracked by version control systems. Whether it's log files, build artifacts, or IDE-specific configurations, these templates provide a comprehensive solution for excluding unnecessary files from repositories.

Here is an example .gitignore file:

For developers navigating the complex landscape of modern software development, the value of .gitignore templates cannot be overstated. By leveraging these templates, developers can streamline their workflow, focusing on code that matters while effortlessly filtering out extraneous files. Moreover, with a permissive CC0–1.0 license, developers have the freedom to adapt and integrate these templates into their projects without restriction.

A prime example of the utility of .gitignore templates lies in their versatility. Whether you're developing a web application in JavaScript, a mobile app in Kotlin, or a data science project in Python, there's a template tailored to your needs. This accessibility empowers developers to hit the ground running, eliminating the hassle of manually crafting .gitignore files from scratch.

Furthermore, the collaborative nature of the Gitignore repository fosters a vibrant ecosystem of contribution and improvement. Developers from around the globe actively contribute updates, ensuring that the templates remain current and relevant in an ever-evolving landscape. This collaborative ethos not only enhances the quality of the templates but also cultivates a sense of community among developers.

In conclusion, the Gitignore repository stands as a testament to the power of collective knowledge and collaboration in software development. By providing a centralized repository of .gitignore templates, it empowers developers to streamline their workflow, maintain cleaner repositories, and focus on what they do best – writing exceptional code. As the software development landscape continues to evolve, the significance of .gitignore templates as indispensable tools for developers is set to endure.

Repository Link : Gitignore

Start using it, and keep enhancing/updating it.

Leveraging Gzip Compression for Enhanced Performance in Spring Boot Applications

Adarsh BP — Tue, 05 Mar 2024 06:11:37 +0000

Introduction: In the world of web development, optimizing the performance of applications is paramount. One effective technique for improving performance is Gzip compression, which significantly reduces the size of HTTP responses, thereby reducing bandwidth usage and improving page load times. In this article, we'll explore how to integrate Gzip compression into Spring Boot applications, enhancing their efficiency and responsiveness.

Understanding Gzip Compression: Gzip compression is a method used to compress files, particularly those transmitted over the internet, to reduce their size. This compression is achieved by replacing repetitive sequences of data with references to a single copy of that data. When a client requests a compressed file, the server compresses the response using Gzip, reducing the amount of data sent over the network. The client then decompresses the file upon receipt, restoring it to its original form.
Benefits of Gzip Compression: Implementing Gzip compression in Spring Boot applications offers several benefits:

Benefits of Gzip Compression: Implementing Gzip compression in Spring Boot applications offers several benefits:

1.Reduced Bandwidth Usage: By compressing HTTP responses, the amount of data transferred between the server and the client is significantly reduced, leading to lower bandwidth usage.
2.Faster Page Load Times: Smaller file sizes mean quicker downloads, resulting in faster page load times for users. This is particularly crucial for improving the user experience and SEO rankings.
3.Improved Performance: With reduced network latency and faster data transfer, applications experience improved performance, leading to higher user satisfaction and engagement.

Integrating Gzip Compression in Spring Boot: Integrating Gzip compression into a Spring Boot application is straightforward and can be achieved using either built-in features or third-party libraries. Here's how you can do it:

Spring Boot Built-in Support: Spring Boot provides built-in support for Gzip compression through its auto-configuration feature. By default, Spring Boot automatically compresses responses for MIME types such as text/html, text/css, text/javascript, and application/javascript. You can customize this behavior by configuring the properties server.compression.enabled and server.compression.mime-types in your application.properties file.
Using Servlet Filters: Another approach to enabling Gzip compression is by implementing a Servlet Filter in your Spring Boot application. This filter intercepts requests and applies Gzip compression to the response before sending it to the client. You can create a custom filter by extending the javax.servlet.Filter interface and registering it in your application's configuration.
Third-party Libraries: Additionally, you can leverage third-party libraries like spring-content-negotiation or compression-configuration to configure Gzip compression in your Spring Boot application. These libraries offer more advanced configuration options and flexibility, allowing you to fine-tune compression settings based on your specific requirements.

Enabling Gzip Compression in Spring Boot:

Spring Boot simplifies the integration of Gzip compression through its built-in support and auto-configuration capabilities. By following a few simple steps, you can enable Gzip compression in your Spring Boot application effortlessly.

Add Dependency:
Start by ensuring that your Spring Boot project includes the necessary dependencies. If you're using Maven, add the following dependency to your pom.xml file:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

For Gradle users, include the following dependency in your build.gradle file:

implementation 'org.springframework.boot:spring-boot-starter-web'

Configure Compression Properties:

Open your application.properties file and configure the properties related to Gzip compression. Spring Boot provides straightforward properties to enable compression and specify the MIME types to be compressed.

# Enable Gzip compression
server.compression.enabled=true

# Define MIME types to be compressed
server.compression.mime-types=text/html,text/css,application/javascript,application/json

Customize the server.compression.mime-types property to include additional MIME types that you want to compress.

Run Your Application:
With the dependencies added and properties configured, you can now run your Spring Boot application as usual. Spring Boot's auto-configuration will take care of the rest, automatically enabling Gzip compression for the specified MIME types.

Verify Compression:
To ensure that Gzip compression is functioning as expected, inspect the response headers using browser developer tools or network monitoring tools. Look for the presence of the Content-Encoding: gzip header in the response, indicating that the content has been compressed successfully.

Best Practices and Considerations: While Gzip compression offers significant performance improvements, it's essential to consider some best practices and potential pitfalls:

Selective Compression: Avoid compressing already compressed file formats such as images, videos, and binary files, as it can lead to negligible savings and unnecessary CPU overhead.
Compression Threshold: Set an appropriate compression threshold to avoid compressing small responses, as the overhead of compression may outweigh the benefits for tiny files.
Monitoring and Tuning: Regularly monitor the performance of your application with and without compression enabled to ensure optimal configuration and identify any potential bottlenecks.

Conclusion: Integrating Gzip compression into your Spring Boot applications is a simple yet effective way to enhance performance and improve user experience. By reducing bandwidth usage, accelerating page load times, and optimizing network performance, Gzip compression contributes to the overall efficiency and responsiveness of your applications. Whether leveraging Spring Boot's built-in features or third-party libraries, implementing Gzip compression is a worthwhile investment that can yield significant benefits for both developers and end-users alike.