DEV Community: Jayesh Nalawade

Mastering Kubernetes Traffic: Ingress, Controllers, and Gateway API Explained

Jayesh Nalawade — Sun, 29 Dec 2024 14:20:49 +0000

Ingress -

Make your HTTP (or HTTPS) network service available using a protocol-aware configuration mechanism, that understands web concepts like URIs, hostnames, paths, and more.
The Ingress concept lets you map traffic to different backends based on rules you define via the Kubernetes API.

An ingress is an alternative to creating a dedicated load balancer in front of Kubernetes services, or manually exposing services within a node. It lets you flexibly configure routing rules, greatly simplifying your production environment.

Key Components of Ingress

Ingress Resource: A YAML/JSON configuration that specifies rules for routing traffic to the appropriate services.
Ingress Controller: A pod that implements the Ingress rules and manages the actual HTTP/HTTPS traffic. Examples include NGINX Ingress Controller, Traefik, or HAProxy.

Why Use Ingress?

Before Ingress we have been using services but for services we have to buy separate static IP for each service that is being very costly compared traditional LB.
Also there are lot of features were available on traditional load balancing rather than only round-robin in LoadBalancer service.
Consolidates multiple service routes under one IP address.
Supports advanced HTTP routing features like path-based routing, host-based routing, SSL termination, etc.
Replaces or complements other access methods like NodePort and LoadBalancer.

What is Ingress?

Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.

Here is a simple example where an Ingress sends all its traffic to one Service:

Prerequisites

You must have an Ingress controller to satisfy an Ingress. Only creating an Ingress resource has no effect.(Ingress controller forwards request to ingress)

You may need to deploy an Ingress controller such as ingress-nginx. You can choose from a number of Ingress controllers.

Ideally, all Ingress controllers should fit the reference specification. In reality, the various Ingress controllers operate slightly differently.

Types of Ingress

1) Single Service Ingress

A single service ingress exposes only one service to external users.
To enable a single service ingress, you must define a default backend—if the ingress object’s host or path does not match information in the HTTP message, traffic is forwarded to this default backend.
The default backend does not have any routing rules.

2) Simple Fanout Ingress

A simple fan-out ingress allows you to expose multiple services using a single IP address
This makes it easier to route traffic to destinations based on the type of request.
This type of ingress simplifies traffic routing while reducing the total number of load balancers in the cluster.

3) Name-based Virtual Hosting

Name-based virtual hosting makes it possible to route HTTP traffic to multiple hostnames with the same IP address.
This ingress type usually directs traffic to a specific host before evaluating routing rules.
Example:
- Requests to app1.example.com are routed to Service app1.
- Requests to app2.example.com are routed to Service app2.

Kubernetes Ingress Controller

In order for an Ingress to work in your cluster, there must be an ingress controller running.

An ingress controller is a dedicated load balancer for Kubernetes clusters.

Ingress controller implements a Kubernetes Ingress and works as a load balancer and reverse proxy entity. It abstracts traffic routing by directing traffic coming towards a Kubernetes platform to the pods inside and load balancing it.

The controller’s routing directions come from the Ingress resource configurations.

Kubernetes cluster can have multiple associated Ingress controllers, and each one works similarly to a deployment controller.
It is event-driven and gets triggered when, for example, a user creates, updates or deletes an Ingress.
Once triggered, an Ingress controller tracks a cluster’s Ingress resources and reads the Ingress specifications. Then, it makes the configuration (in YAML or JSON) intelligible for the reverse proxy so it can provide the cluster with the resources it requires.

The main functions of a Kubernetes ingress controller are to receive and load-balance traffic from outside Kubernetes to pods running in a Kubernetes cluster, and manage egress traffic from services within the cluster to services outside the cluster. They monitor pods running on Kubernetes and automatically update load balancing rules when pods are added or removed from a service.

Kubernetes Ingress Controller Benefits and Limitations

Technically, an ingress is not a service, it is a layer 7 (application layer) router that is typically exposed through a load balancer service. It is cheaper than using a cloud load balancer for each service as it relies on an ingress controller which is hosted together with the application.
It ensures each service has only one IP that can be accessed from the internet, and ==traffic intended for this IP is routed to the correct service by the ingress controller.==

Benefits of ingress controllers include:

Provides secure access to services over HTTP or HTTPS paths, instead of using direct connections.
Creates access routes to multiple services, with full control over routing of services and conditions for external access.
Creates a single path for ingress traffic, which can be modified based on conditions defined by the operator, instead of opening many connections to access a Kubernetes application.
Simplifies management of complex ingress processes, with the ability to easily manage access to multiple services in one system. This can have a significant impact on performance and management costs of large systems.
All ingress controllers support a set of annotations that enable specific software-based features. For example, in the Traefik ingress controller, users can use annotations to add middleware to ingress, even if not supported by the Ingress specification.

Limitations of ingress controllers:

The Ingress controller only handles layer 7 traffic==, while an ingress routes HTTP and HTTPS traffic. This means it is ==not possible to route TCP and UDP traffic.
Ingress is used in a single namespace. This means that ==anything within a Kubernetes namespace can only reference services within the same namespace==. To solve this problem, Kubernetes has introduced a gateway API specification, which enables cross-namespace communication. This specification is still in alpha status. A Kubernetes-native API Gateway can be used today for cross-namespace communications.

How to choose an ingress controller

There are several ingress controllers available in Kubernetes, each has advantages and preferred use cases. It can be challenging to select the ingress controller suitable for your needs.
If you’re managing a small production environment and don’t plan to scale it, you probably need only the standard features such as load balancing, flow control, and flow splitting. Be careful not to select a controller that has more functionality than you need, because this can negatively impact performance and create unneeded management complexity.

However, if your application runs in a distributed, multi-cloud environment, you may need an enterprise-grade, high-performance networking solution.

https://www.solo.io/topics/kubernetes-api-gateway/kubernetes-ingress

Kubernetes Gateway API

Current State of Ingress in Kubernetes

After you deploy a Kubernetes application, you typically need to expose it to end users. This is usually done using an Ingress Controller. The Ingress API object defines the routing and mapping of external traffic to the Kubernetes service. It also provides load balancing, SSL termination, and name-based virtual hosting. Currently, the native Ingress API is very limited in scope.
While this basic set of ingress controls was enough to get started, the need for more sophisticated controls over incoming traffic led to the development of additional tools to add more control over ingress.

What is the Kubernetes Gateway API?

The Kubernetes Gateway API is the first step toward adding additional control over ingress patterns directly in Kubernetes.
It represents a new standard in defining how to configure and manage how traffic is defined and routed in Kubernetes.
This extensible API is a collection of resources that models a network of services in Kubernetes and provides a standard way to describe how inbound traffic routes can be defined.
The Gateway API consists of three new resources to define.

1) GatewayClass: This exists at a cluster level to describe the set of common configurations and behaviors for a set of Gateways.
2) Gateway: A gateway defines how traffic can connect to the services that exist in the cluster.
3) Routes: These describe how to map incoming requests to the services. These can be defined as HTTPRoute, TLSRoute, TCPRoute/UDPRoute or GRPCRoute.

![[Pasted image 20241226114019.png]]

GatewayClass must be defined in order to have any Gateways in a cluster. Each Gateway defines ==how incoming traffic will connect through a Route to the service it is destined to connect with.==

Here's a comparison table highlighting the differences between Kubernetes Ingress and Gateway API:

Feature	Ingress	Gateway API
Introduction	Native Kubernetes object for managing HTTP/HTTPS routing.	Modern, extensible API designed to replace Ingress over time.
Traffic Routing	Basic routing based on hostnames and paths.	Advanced routing with support for headers, query params, etc.
Protocol Support	Primarily HTTP/HTTPS.(Only layer 7)	Supports HTTP, HTTPS, TCP, UDP, and more.
Multi-tenancy	Limited, requires custom implementations.	Built-in support with `GatewayClass` and delegation.
Extensibility	Limited to annotations for customizations.	Modular design with support for custom route types.
Multi-listener Support	Not supported.	Supports multiple listeners (e.g., HTTP, HTTPS, TCP) on different ports.
Separation of Concerns	Cluster admins and developers manage a single object.	Separation between infrastructure (`Gateway`) and app routing (`HTTPRoute`).
Traffic Splitting	Limited (requires custom annotations or controllers).	Native support for traffic splitting (e.g., ==canary deployments).==
TLS Configuration	Basic support with limited flexibility.	Granular TLS configuration, including per-route TLS settings.
Controller Support	Widely supported by most ingress controllers.	==Requires Gateway-compatible controllers (e.g., Istio, NGINX).==
Scalability	Suitable for basic use cases in small/medium clusters.	Designed for ==large-scale and complex routing scenarios.==
Standardization	Relies on annotations, leading to inconsistent behavior between controllers.	Consistent behavior with standardized CRDs.
Custom Resource Definitions (CRDs)	No CRDs, uses `Ingress` resource.	CRDs like `GatewayClass`, `Gateway`, `HTTPRoute`, etc.
Maturity	Stable and widely used.	Newer, under active development, and rapidly gaining adoption.

When to Use Each

Ingress:
- Use for simple HTTP/HTTPS routing in smaller or less complex clusters.
- Ideal for straightforward use cases with limited advanced routing needs.
Gateway API:
- Use for advanced routing, multi-tenancy, scalability, and extensibility.
- Ideal for large, multi-team environments or when you need protocol-agnostic routing.

Resources -

1) https://www.solo.io/topics/kubernetes-api-gateway/kubernetes-ingres

(They have very good articles)

2) kubernetes docs

Thank you and subscribe for more updates .

Services in Kubernetes

Jayesh Nalawade — Sun, 29 Dec 2024 09:49:07 +0000

In Kubernetes, a Service is an abstraction that defines a logical set of Pods and a policy to access them. Services provide a stable endpoint to applications running on Pods, even if the underlying Pods are dynamically created or destroyed.

Key Concepts

1. Purpose of Services:

- Pods in Kubernetes are ephemeral, meaning their IPs can change if the Pod is restarted. Services provide a consistent way to access Pods.

- They enable communication between components of an application (e.g., frontend talking to backend).

- They provide load balancing across multiple Pods.

2. Service Selector:

- The set of Pods targeted by a Service is usually determined by a selector that you define

- A Service uses selectors to identify the Pods it routes traffic to, based on Pod labels.

3. ClusterIP and Endpoints:

- When you create a Service, Kubernetes assigns a ClusterIP to the Service, which acts as a virtual IP inside the cluster.

- The Service points to a set of Endpoints, which are the IPs of the Pods matching the selector.

There are 3 types of services mainly used in k8s :

a. ClusterIP (Default)-

Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster.
What it does:

- Exposes the Service internally within the Kubernetes cluster.

- Only accessible from within the cluster (e.g., by other pods).

Use cases:

- For internal communication between microservices.

- Backend services like databases or caches that do not need to be exposed to the internet.

b. NodePort -(Not safe cause we expose the port)

Exposes the Service on each Node's IP at a static port (the NodePort). To make the node port available
What it does:

- Exposes the Service externally on a static port (between 30000--32767) on each node in the cluster.

- Accessible using <NodeIP>:<NodePort>.

Use cases:

- For development or debugging purposes where direct node access is acceptable.

- For exposing services in environments without a LoadBalancer.

C. LoadBalancer-

On cloud providers which support external load balancers, setting the type field to LoadBalancer provisions a load balancer for your Service.(if you have their services like EKS,GKE)

- cloud-controller-manager component then configures the external load balancer to forward traffic to that assigned node port.

What it does:

- Exposes the Service externally using a cloud provider's load balancer (e.g., AWS ELB, Google Cloud LB).(for automatic load balancer -EKS )

- Automatically provisions a load balancer and assigns an external IP.

Use cases:

- For production environments where external traffic needs to reach the application.

- Scenarios requiring high availability and load distribution.

3. Core Concepts of Kubernetes Services

Selectors: Define the pods the Service routes traffic to, based on labels.
Endpoints: Represents the set of IPs and ports for the pods selected by the Service.
TargetPort: The port on the pod where the Service forwards traffic.
ClusterIP: Internal IP assigned to the Service for communication

In Kubernetes, a Service provides a stable endpoint (IP address and port) to access a set of pods. A Service routes traffic to the correct pods based on labels and selectors, and the port configuration plays a key role in how traffic is handled within the cluster. The ports in a Kubernetes Service configuration define the communication between different components.

Key Ports in Kubernetes Service

1. port

2. targetPort

3. nodePort (for NodePort services)

4. protocol

5. name

1. port

Definition: This is the port on which the Service is exposed within the Kubernetes cluster.
Usage: The port is the port that other services or pods in the cluster use to access the Service.
Example: If you define port: 80, any service or pod that communicates with your Service would connect to port 80.
`ports:

port: 80 # Exposed port inside the cluster`

2. targetPort

Definition: This is the port on the pods that the Service forwards traffic to. This is where your application is listening.(Expose used in docker image)
Usage: The targetPort should match the port the container inside the pod is listening on. If this is not specified, it defaults to the same value as the port.
Example: If you set targetPort: 8080, Kubernetes will route traffic that hits port 80 to port 8080 inside the pods.
ports:

port: 80 # Exposed port on the service

targetPort: 8080 # Port where the pod is actually listening

3. nodePort (only for `NodePort` Service type)

Definition: This is the port on each node in the cluster that exposes the service externally (outside the Kubernetes cluster).(port of your host)
Usage: The nodePort is used in NodePort type services. It allows you to access the service from outside the Kubernetes cluster by connecting to nodeIP:nodePort.
Port Range: By default, the valid range for nodePort is from 30000 to 32767, but you can specify a custom range if needed.
Example: If you set nodePort: 30080, traffic hitting port 30080 on any node's external IP will be forwarded to the service.

- ports:

port: 80 # service port (port on which service is exposed)

targetPort: 8080 # To this port service forward traffic (application port)

nodePort: 30080 # Port on each node forward's traffic to target port

Mastering Ansible: Quick and Powerful Ad-Hoc Commands for Efficient Automation

Jayesh Nalawade — Fri, 20 Dec 2024 17:37:34 +0000

Ansible Ad-hoc commands

An Ansible ad hoc command uses the /usr/bin/ansible command-line tool to automate a single task on one or more managed nodes. ad hoc commands are quick and easy, but they are not reusable.

you could execute a quick one-liner in Ansible without writing a playbook. An ad hoc command looks like this:
$ ansible [pattern] -m [module] -a "[module options]"

The -a option accepts options either through the key=value syntax or a JSON string starting with { and ending with } for more complex option structure. You can learn more about patterns and modules on other pages.

Ansible ad-hoc commands have a simple syntax that you can use to perform quick tasks across your infrastructure

ansible <host-pattern> -i <inventory-file> -m <module-name> -a "<module-arguments>" [options]

Explanation of Parameters:

ansible: The Ansible command-line tool.
<host-pattern>: Specifies the target hosts (e.g., all, web, db, or an individual host).
-i <inventory-file>: Specifies the inventory file that contains the hosts you want to target.
-m <module-name>: Specifies the Ansible module you want to use (e.g., apt, yum, file, ping, copy, etc.).
-a "<module-arguments>": Arguments passed to the module (e.g., the package name, file path, or other options).
[options]: Optional flags like --become for privilege escalation, --check for dry-run, etc.

Common Options for Ad-Hoc Commands:

--become: Elevates privileges to execute tasks as a superuser (==useful for package installations or service management==).
--check: Performs a ==dry-run== of the playbook or command, showing what changes would be made without actually applying them.
--diff: Shows a diff of what changes will be made (e.g., when modifying files).
--ask-become-pass: Prompts for the sudo password (if required) when using --become

Host Patterns:

all: Targets all hosts in the inventory.
web: Targets the web group in the inventory.
db[0:5]: Targets a range of hosts, like db0, db1, db2, etc.
localhost: Targets the local machine (useful for testing or local actions).
<hostname>: Targets a specific host by name or IP address.

Important Ad-hoc commands

1. Basic Commands for System Management

1) Ping all hosts
- ansible all -i inventory.ini -m ping
* * for particular host
ansible <hostname> -i inventory.ini -m ping

2) Gather Facts (System Information):
- ansible all -i inventory.ini -m setup
- for specific tasks use filter
ansible all -i inventory.ini -m setup -a "filter=<fact_name>
eg .
1) Gather only memory-related facts:
ansible all -i inventory.ini -m setup -a "filter=ansible_memory_mb"
2) Gather only network-related facts:
ansible all -i inventory.ini -m setup -a "filter=ansible_default_ipv4,ansible_all_interfaces
3) Gather only disk-related facts:
`ansible all -i inventory.ini -m setup -a "filter=ansible_devices"

3) Check System Uptime
- ``ansible all -i inventory.ini -m command -a "uptime

2.Package Management (Install/Remove Software)

1) Install Package
- ansible all -i inventory.ini -m apt -a "name=nginx state=present" --become (debian based eg. ubuntu) change to apt for Redhat based
- became uses root permissions

2) Remove a package
- ansible all -i inventory.ini -m apt -a "name=nginx state=absent" --become

3) Upgrade All Packages- Upgrades all packages on all hosts.
- ansible all -i inventory.ini -m apt -a "upgrade=dist" --become

3.File and Directory Management

1) Create a Directory
- ansible all -i inventory.ini -m file -a "path=/tmp/testdir state=directory mode=0755
- Creates a directory at testdir

2) Copy a File to Remote Hosts
- ansible all -i inventory.ini -m copy -a "src=/local/file dest=/remote/file mode=0644

3) Delete a File or Directory
- ansible all -i inventory.ini -m file -a "path=/tmp/testfile state=absent

4) Change File Permissions
- ansible all -i inventory.ini -m file -a "path=/tmp/testfile mode=0755

4. Service Management

1) Start a service
- ansible all -i inventory.ini -m service -a "name=nginx state=started" --become

2) Stop a Service:
- ansible all -i inventory.ini -m service -a "name=nginx state=stopped" --become

3) Restart a service:
- ansible all -i inventory.ini -m service -a "name=nginx state=restarted" --become

4) Enable a service from boot:
- ansible all -i inventory.ini -m service -a "name=nginx enabled=yes" --become

- enabled=no - Disable a Service from boot

5. User and Group Management

1) Create a User:
- ansible all -i inventory.ini -m user -a "name=deploy state=present" --become
- Creates the deploy user on all hosts.

2) Delete a User:
- ansible all -i inventory.ini -m user -a "name=deploy state=absent" --become

3) Add user to group:
- ansible all -i inventory.ini -m user -a "name=deploy groups=sudo append=yes" --become
- Adds the deploy user to the sudo group on all hosts

6. Network Management

1) Check if a Port is Open:
- ansible all -i inventory.ini -m shell -a "netstat -tuln | grep :80"

 - Checks if port `80` is open on all remote hosts.

2) Download a File from the Internet:
- ansible all -i inventory.ini -m get_url -a "url=https://example.com/file.tar.gz dest=/tmp/file.tar.gz"

 -  Downloads a file from a URL to the `/tmp` directory on all hosts.

7.Process and System Management

1) Reboot Host
- ansible all -i inentory.ini -m reboot --become

2) Kill a process
- ansible all -i inventory.ini -m shell -a "pkill -f provess_name" --become

3) Check a disk space
- ansible web -i inventory.ini -m shell -a "df -h"
- uses only web tagged machines in inventory file

4) Show running processes:
- ansible all -i inventory.ini -m shell -a "ps aux"

8.Docker Management

1) Start Docker Container:
- ansible all -i inventory.ini -m docker_container -a "name=my_cont image=nginx state=started"

2) Stop a docker container
- ansible all -i inventory.ini -m docker_container -a "name=my_cont state=stopped"

9. Log Management

1) Tail log file
- ansible all -i inventory.ini -m shell -a "tail -n 10 /var/log/nginx/access.log" --become

10.Other

1) Execute a Shell Command:
- ansible all -i inventory.ini -m shell -a "echo Hello, World!"
- Executes a simple shell command (echo Hello, World!) on all hosts

2) Create file with content
- ansible all -i inventory.ini -m copy -a "content='Hello, World!' dest=/usr/share/nginx/html/hello.html"

Docker ENV vs ARG Explained: Key Differences to Know

Jayesh Nalawade — Tue, 10 Dec 2024 17:52:24 +0000

When working with Docker, two essential instructions, ENV and ARG, are often used to handle variables in your Dockerfile. While they may seem similar, they serve different purposes. Let's dive into their differences, use cases, and best practices!

Overview

Feature	`ARG`	`ENV`
Scope	Build-time	Run-time
Default Value	Yes (via `ARG name=default_value`)	Yes (via `ENV name=value`)
Accessibility	Available only during build	Available during build and run
Persistency	Not persisted in the final image	Persisted in the final image

What is ARG?

The ARG instruction is used to define build-time variables. These variables are available only during the build process of the Docker image and cannot be accessed once the image is running.

Key Features of ARG:

Used for passing build-time information like application versions, flags, or secrets.
Can have a default value.
Can be overridden via the --build-arg flag when running docker build.

Example: Using ARG

ARG APP_VERSION=1.0.0
FROM alpine:${APP_VERSION}

RUN echo "Building with version ${APP_VERSION}"

To build the image with a custom version:

docker build --build-arg APP_VERSION=3.15 -t my-alpine .

What is ENV?

The ENV instruction is used to define environment variables that are available both during the image build process and at runtime. These variables are persisted in the final image.

Key Features of ENV:

Used to configure application behavior or pass runtime variables.
Can be overridden at runtime with the docker run -e flag or Docker Compose.
Useful for defining default values for environment variables.

Example: Using ENV


FROM alpine:3.15

ENV APP_ENV=production

RUN echo "Environment is set to ${APP_ENV}"

CMD ["sh"]

To override the variable at runtime:

docker run -e APP_ENV=development my-alpine

Comparison: When to Use ARG vs ENV

Use Case	Instruction to Use
Pass secrets during build	`ARG`
Specify application version	`ARG`
Configure runtime application behavior	`ENV`
Define default runtime variables	`ENV`
Variables not needed after build	`ARG`

Key Notes and Best Practices

Security:

-   Avoid using "ENV" for sensitive information (e.g., passwords or API keys), as they persist in the image and can be inspected.

-   Use "ARG" for sensitive build-time secrets and tools like Docker BuildKit for better security.

Default Values:\
Both ARG and ENV allow default values, but ARG defaults are accessible only during the build stage, while ENV defaults persist through runtime.
Combination:\
You can combine ARG and ENV to define runtime environment variables using build-time arguments:
```
 ARG APP_ENV=production
 ENV APP_ENV=${APP_ENV}
```
Overriding:

-   "ARG" values can be overridden at build time with `--build-arg`.

-   "ENV" values can be overridden at runtime with `docker run -e`.

Conclusion

In summary:

Use ARG for build-time variables (temporary use).
Use ENV for runtime variables (persistent use).

Understanding when to use each is essential for writing efficient and secure Dockerfiles. Keep these distinctions in mind, and you'll be crafting Dockerfiles like a pro

ENTRYPOINT vs CMD in Dockerfile

Jayesh Nalawade — Wed, 04 Dec 2024 19:19:37 +0000

CMD (Command)

Purpose: Provides default arguments for the container.
Usage: Use CMD when you want to provide default behavior but allow it to be overridden by arguments supplied when running the container (docker run).

Example 1: Default command with override

FROM ubuntu:latest CMD ["echo", "Hello, World!"]

If you run the container as docker run <image>, it will print Hello, World!.
If you provide a different command, like docker run <image> echo "Goodbye", it will override the CMD.

ENTRYPOINT

Purpose: Sets the main executable for the container that is not easily overridden.
Usage: Use ENTRYPOINT when you want to enforce a specific command or script as the container's entry point. Additional arguments provided with docker run are passed as arguments to the ENTRYPOINT.

Example 2: Enforce entry point

FROM ubuntu:latest ENTRYPOINT ["echo"]

Running docker run <image> Hello, World! will always use echo, so the result will be Hello, World!.
You cannot replace echo unless you use the --entrypoint flag.

Combining `ENTRYPOINT` and `CMD`

Combine ENTRYPOINT and CMD for flexible and reusable images.
ENTRYPOINT specifies the main command, and CMD provides default arguments that can be overridden.

Example 3: Flexible combination

FROM ubuntu:latest ENTRYPOINT ["echo"] CMD ["Hello, World!"]

Running docker run <image> will print Hello, World!.
Running docker run <image> Goodbye will print Goodbye, overriding CMD but not ENTRYPOINT.

Key Differences

Feature	CMD	ENTRYPOINT
Overridable	Yes (via `docker run` arguments).	No, unless `--entrypoint` is used.
Intended Purpose	Default behavior or arguments.	Main executable/script.
Flexibility	More flexible (easier to override).	More rigid (forces specific usage).

When to Use Which?

Use CMD when you want to provide a default, flexible behavior that users can easily override.
Use ENTRYPOINT when you want to enforce a specific script or command while still allowing extra arguments.

Dockerfile Instructions Guide

Jayesh Nalawade — Tue, 26 Nov 2024 19:02:53 +0000

This guide provides a comprehensive overview of the commonly used Dockerfile instructions with examples to help you create efficient and optimized Docker images.

1. FROM

Specifies the base image for the Docker image.

FROM ubuntu:latest

Always the first instruction in a Dockerfile.
Example: FROM python:3.9 for Python-based projects.

2. WORKDIR

Sets the working directory inside the container.

WORKDIR /app

Ensures subsequent commands (like COPY or RUN) operate in this directory.

3. COPY

Copies files or directories from the build context into the container.

COPY src/ /app/

Use COPY for local files.
It's simpler and faster than ADD when no archive extraction or URL downloads are required.

4. ADD

Copies files or downloads remote resources. Also extracts archives.

ADD myfile.tar.gz /app/

Automatically extracts compressed files (e.g., .tar.gz).
Supports downloading files from remote URLs.

5. RUN

Executes commands during the build process (e.g., installing software).

RUN apt-get update && apt-get install -y curl

Each RUN instruction creates a new image layer.
Combine commands with && to minimize layers.

6. CMD

Specifies the default command to run when the container starts.

CMD ["python", "app.py"]

Can be overridden at runtime using docker run <image> <command>.

7. ENTRYPOINT

Defines the main application or command to run in the container.

ENTRYPOINT ["nginx", "-g", "daemon off;"]

Works with CMD to provide default arguments.
Use ENTRYPOINT for containers meant to act like executables.

8. ENV

Sets environment variables.

ENV NODE_ENV=production

Variables can be accessed inside the container.

9. EXPOSE

Documents the port(s) the container listens on.

EXPOSE 8080

Does not publish the port; you need -p in docker run.

10. LABEL

Adds metadata to the image.

LABEL maintainer="you@example.com"

Use for descriptions, versioning, or maintainers.

11. ARG

Defines build-time variables.

ARG APP_VERSION=1.0

Passed during the build process (docker build --build-arg APP_VERSION=2.0).

12. VOLUME

Creates a mount point for persistent data.

VOLUME /data

Ensures the directory persists outside the container.

13. USER

Specifies the user for running container processes.

USER appuser

Use to improve security by avoiding root privileges.

14. SHELL

Changes the shell used in RUN commands.

SHELL ["/bin/bash", "-c"]

15. ONBUILD

Sets triggers for dependent images.

ONBUILD RUN apt-get update

Executes only when the image is used as a base for another Dockerfile.

Best Practices

Use Multi-Stage Builds: Reduce image size by separating build and runtime stages.
Minimize Layers: Combine commands in RUN instructions.
Leverage .dockerignore: Exclude unnecessary files from the build context.
Use Tags for Base Images: Avoid latest for more consistent builds.

Example Dockerfile

Here's a complete example:

# Use Python as the base image
FROM python:3.9

# Set environment variables
ENV APP_HOME=/app

# Set the working directory
WORKDIR $APP_HOME

# Copy application files
COPY . $APP_HOME

# Install dependencies
RUN pip install -r requirements.txt

# Expose the port
EXPOSE 5000

# Run the application
CMD ["python", "app.py"]

Resources

Dockerfile Reference
Docker Best Practices

Docker commands (part-1)

Jayesh Nalawade — Sat, 16 Nov 2024 13:51:49 +0000

Beginner Commands:

docker --version
- Check the Docker version installed on your system.
- Example: docker --version
docker pull <image>
- Download a Docker image from Docker Hub.
- Example: docker pull ubuntu
docker build -t <tag> <path>
- Build a Docker image from a Dockerfile located at a specified path.
- Example: docker build -t myimage .
docker images
- List all locally stored Docker images.
- Example: docker images
docker rmi <my_image/id>
- Remove an image
- Example: docker rmi ubuntu
docker ps
- List running containers.
- Example: docker ps
docker ps -a
- List all containers (including stopped ones).
- Example: ``docker ps -a
docker run <image>
- Run a container from a specified image.
- Example: docker run ubuntu
docker run -d <image_nam/id>
- Run a container in detached mode (background). #usethis beacause without -d powershell getting error
- Example: ``docker run -d ubuntu
docker run --name <container_name> <image_name>
- Run container with specific name
- Example: docker run --name mycontainer nginx
- It will create container named mycontainer with base image of nginx
docker run -it --name mycont nginx bash
- Runs container named mycont with base image nginx and also gives bash shell(command prompt inside container)
docker run -p <host_port>:<container_port> <image_name>
- Example:docker run -p 8080:80 nginx
- This runs an Nginx container and maps port 8080 on the host to port 80 inside the container.

bestway -best way to launch container of nginx or any other

docker run -d -p 8080:80 --name myweb nginx
- run an Nginx container in detached mode and expose port 80 from the container to port 8080 on the host.
`docker run -it -p 8080:80 --name myweb nginx bash
- This will start the Nginx container but override the default entry point (which is Nginx) with a bash shell instead, allowing you to interact with the container's file system or run commands inside it. ` ---
docker exec -it <container_id> <command>
- Run commands inside a running container.
- Example: docker exec -it mycontainer bash
- This command open up interactive bash terminal inside container
docker stop <container_id>
- Stop a running container.
- Example: docker stop mycontainer
`docker stop
- Stop multiple containers at once:
- Example:docker stop cont1 cont2 1233aec2 (with name or id)
docker stop $(docker ps -q)
- docker ps -q lists all the running container IDs, and docker stop stops them
docker rm $(docker ps -a -q)
- Remove All Containers: Once the containers are stopped, you can remove them with this command
docker rm <container_id>
- Remove a stopped container.
- Example: docker rm mycontainer
docker rmi $(docker images -q)
- To remove all Docker images from your system:
docker rmi -f <image_id_or_name>
- To remove an image that is currently being used by a container, you would typically encounter an error because Docker prevents you from removing an image that’s being used.
docker rm -f <container_id_or_name>
- remove a running container, you can't do so directly without stopping it first so we use -f/--force

Docker Port Mapping

Jayesh Nalawade — Thu, 14 Nov 2024 13:35:49 +0000

Docker port mapping allows your containerized application to be accessed from outside the container by linking a port on the host machine to a port inside the container.

Why Port Mapping?

Containers run in isolated environments, so they don’t automatically have access to your host’s network. Port mapping lets you “expose” specific ports from the container so external requests can reach it.

For example, if you’re running a web app in a container on port 80, you can use port mapping to make it available on a specified port on your computer, like port 8080.

How to Use Docker Port Mapping

When running a container, you can map ports with the -p option in the following format:

docker run -p :

Example

Let's say you have a web server running on port 80 inside a Docker container, and you want to access it on port 8080 on your local machine.

docker run -p 8080:80 my_web_server

8080: Port on the host machine (your computer).
80: Port inside the container where the application is running.
my_web_server: Name of the Docker image.

Now, if you open http://localhost:8080 in a web browser, you’ll be able to access the application running in the Docker container.

Multiple Port Mappings

You can also map multiple ports if needed. Here’s an example where we map ports 8080 and 443 (for HTTPS):

docker run -p 8080:80 -p 8443:443 my_web_server

8080:80 maps port 80 in the container to 8080 on the host.
8443:443 maps port 443 in the container to 8443 on the host.

Check Mapped Ports

To check the mapped ports of running containers, use:

docker ps

This command shows the container’s details, including the port mappings under the “PORTS” column.

Common Use Cases

Web Applications: Expose port 80 or 443 from the container to make the app accessible via HTTP or HTTPS.
Database Containers: Map standard ports like 3306 (MySQL) or 5432 (PostgreSQL) for database access.
API Services: Expose specific ports so other services can access the API.

Summary

Use -p : to map a container port to a host port.
You can map multiple ports by specifying -p multiple times.
Check your mappings with docker ps.

Port mapping in Docker allows applications inside containers to communicate with the outside world, enabling easy access and integration into your development or production environment.

Git & Github(part-1)

Jayesh Nalawade — Wed, 13 Nov 2024 18:17:11 +0000

what is Git-

Git is a distributed version control system (VCS) that helps developers track and manage changes to their code over time. It allows multiple developers to work on the same codebase simultaneously and keeps a history of changes, making it easy to revert to previous versions if needed.

Key Features of Git:

Version Control: Tracks changes in files, allowing developers to save different versions of code, known as commits.
Branching: Enables developers to create isolated branches for new features or bug fixes, keeping the main codebase clean.
Merging: Combines changes from different branches, which is essential for collaboration and feature integration.
Distributed System: Every developer has a full copy of the codebase, which allows for offline work and reduces reliance on a single server.

What is GitHub?

GitHub is a cloud-based platform that hosts Git repositories, making it easy to share code, collaborate, and manage projects. GitHub provides a web-based interface for working with Git, along with additional features like issue tracking, pull requests, and access controls.

Key Features of GitHub:

Remote Repository Hosting: Stores your code in the cloud, accessible from anywhere.
Collaboration: Developers can work together on projects, review each other’s code, and manage project tasks.
Pull Requests: A feature that allows developers to propose changes, review code, and discuss modifications before merging them into the main codebase.
Issues: Built-in bug tracking and task management to help organize and prioritize development work.
GitHub Actions: Provides automation for CI/CD workflows directly in GitHub.

Configure Git with Username and Email-

1)$ git config --global user.name "Paul Philips

What it does:

Sets your name to "Paul Philips" for all the commits you make in any Git repository on your computer.

Why it’s useful:

When you make a commit, Git will associate your name with that commit. This is especially important when you're collaborating with others and want to keep track of who made each change.

Example Scenario: You are working on a project with others, and when you make a change and commit it, your name "Paul Philips" will show up in the commit history.

2)$ git config --global user.email paulphilips@email.com

What it does:

Sets your email for all commits, so when you push changes to a repository (e.g., GitHub), your email will be associated with those changes.

Why it’s useful:

Your email helps identify you in the commit history, and on platforms like GitHub, it will be used to link your commits to your account.

Example Scenario: When you push code to GitHub, your email (e.g., paulphilips@email.com) will show who made the changes, and it helps you keep track of contributions in a public repository.

3)$ git config --list

What it does:

Displays all the Git configuration settings that are currently applied. This includes your user.name, user.email, and any other configurations, like your preferred text editor or merge tool.

Why it’s useful:

It lets you check if the user information you've set is correct and if any other Git settings are applied. You can also verify if there are conflicting settings between global and local repositories.

Docker Basics

Jayesh Nalawade — Wed, 13 Nov 2024 17:37:00 +0000

Docker Basics: An Introduction to Containers

Docker is a powerful platform that allows developers to package, ship, and run applications in isolated environments called containers. Containers bundle all the necessary dependencies, configurations, and application code, making it easy to run applications consistently across different systems. Here’s an overview to get started with Docker.

What is Docker?

Docker enables containerization, which allows applications to run consistently across various environments. Unlike virtual machines, Docker containers are lightweight and share the host OS kernel, leading to faster startup times and reduced resource usage.

Key Concepts

Images: Templates used to create containers. An image is like a blueprint that includes your application code and dependencies.
Containers: Running instances of Docker images. A container is a lightweight, portable encapsulation of an environment for running applications.
Dockerfile: A text file with instructions for building a Docker image, specifying commands for setting up the environment, installing dependencies, and configuring the app.
Docker Hub: A public repository where Docker images are stored and shared.

Why Use Docker?

Docker offers several key benefits:

Consistency: Ensures applications run the same way across different environments.
Portability: Docker containers can be run on any system that supports Docker.
Efficiency: Containers use less memory and resources than virtual machines by sharing the host OS kernel.
Scalability: Applications can be easily scaled by deploying multiple containers.

Basic Docker Commands

Here are some essential Docker commands to get you started:


bash
# Check Docker version
docker --version

# Pull an image from Docker Hub
docker pull <image_name>

# List all Docker images
docker images

# Run a container from an image
docker run <image_name>

# Run a container and directly access bash
docker run -it <image_name> /bin/bash

# Access bash of an existing container
docker exec -it <container_id> /bin/bash

# List all running containers
docker ps

# Stop a running container
docker stop <container_id>

# Remove a container
docker rm <container_id>

# Remove an image
docker rmi <image_name>

Virtualization vs. Containerization in DevOps: Differences, Tools, and Use Cases

Jayesh Nalawade — Mon, 11 Nov 2024 12:11:36 +0000

Introduction

As organizations look to optimize their software development and deployment processes, virtualization and containerization have become key technologies. Both play significant roles in DevOps, enhancing flexibility, scalability, and resource efficiency. But what exactly are virtualization and containerization, how do they differ, and what tools can we use to implement them in a DevOps environment?

In this article, we’ll break down virtualization and containerization, explore their differences, and discuss how they contribute to DevOps.

What is Virtualization?

Virtualization is the process of creating virtual versions of physical hardware resources, including servers, storage, and networks. With virtualization, multiple operating systems can run on a single physical machine by abstracting the hardware layer. Each virtual machine (VM) functions independently, with its own OS and resources, making it a complete, isolated environment.

Key Benefits of Virtualization in DevOps

Resource Efficiency: VMs share resources, reducing the need for additional physical hardware.
Isolation: VMs run independently, providing full isolation of applications and their dependencies.
Flexibility: Virtualization allows DevOps teams to spin up multiple environments quickly and efficiently.

What is Containerization?

Containerization is a lightweight alternative to virtualization. Instead of virtualizing the entire hardware, containerization virtualizes the OS. Containers package an application along with its dependencies and libraries into a single, lightweight executable unit that runs consistently across environments.

Key Benefits of Containerization in DevOps

Lightweight: Containers share the OS kernel, making them significantly lighter than VMs.
Portability: Containers work the same across development, testing, and production environments, ensuring consistency.
Scalability: Containers are easy to replicate and scale horizontally, enabling flexible deployment.

Virtualization vs. Containerization: Key Differences

Aspect	Virtualization	Containerization
Resource Utilization	VMs are resource-intensive due to separate OS instances.	Containers are lightweight, sharing the OS kernel.
Isolation Level	Each VM is fully isolated with its own OS.	Containers share the same OS kernel but are isolated at the application level.
Performance	VMs consume more resources, affecting performance.	Containers are more performant due to shared resources.
Portability	VMs are less portable across environments.	Containers offer high portability across environments.
Boot Time	VMs take minutes to start.	Containers start in seconds.
Use Case	Ideal for running multiple OS instances.	Ideal for microservices and cloud-native applications.

Tools for Virtualization

Virtualization tools help create and manage VMs, making it easier to create isolated environments for testing, staging, and production.

Popular Virtualization Tools

VMware: A widely used platform for managing virtual machines in enterprise environments.
Oracle VirtualBox: An open-source tool for creating and managing VMs on desktops.
Microsoft Hyper-V: A Microsoft tool for virtualization, often used in Windows-based infrastructures.
KVM (Kernel-based Virtual Machine): An open-source virtualization technology for Linux environments.

Tools for Containerization

Containerization tools allow DevOps teams to build, deploy, and manage containers at scale. Containers are ideal for microservices, where applications are broken down into smaller, manageable components.

Popular Containerization Tools

Docker: The most popular containerization platform, allowing you to package applications with their dependencies.
Kubernetes: An open-source container orchestration platform that automates deployment, scaling, and management of containerized applications.
OpenShift: A Kubernetes-based platform developed by Red Hat, offering additional DevOps tools.
Rkt (Rocket): An alternative to Docker, designed with a focus on security.

Virtualization vs. Containerization in DevOps

In DevOps, both virtualization and containerization play crucial roles:

Environment Consistency: Containers ensure consistency between development, testing, and production environments, reducing “works on my machine” issues.
Efficient Testing: VMs allow developers to test across multiple operating systems, while containers provide lightweight and isolated environments for application-level testing.
Continuous Integration/Continuous Deployment (CI/CD): Containers support fast, reliable CI/CD by enabling DevOps teams to deploy and scale applications quickly and consistently.
Microservices Architecture: Containerization is essential for microservices-based applications, where each service can be isolated and managed independently.
Scalability and Flexibility: Both virtualization and containerization enable DevOps teams to scale resources based on demand, ensuring applications are always available.

Choosing Between Virtualization and Containerization

Use Virtualization when you need to run multiple operating systems on the same hardware, or if your applications require full OS isolation.
Use Containerization when you’re working with microservices, need fast start-up times, and want to optimize resource usage.

In many cases, DevOps environments use both virtualization and containerization together, depending on the specific needs of the application and infrastructure.

Conclusion

Virtualization and containerization are foundational technologies in DevOps. Each offers unique benefits and use cases, from creating isolated VMs to enabling scalable containerized applications. By choosing the right tools, DevOps teams can enhance collaboration, streamline workflows, and optimize resource use, ultimately delivering software faster and more efficiently.

Let us know which tools you’re using in your DevOps journey! Are you team VM or team Container?

What is DevOps and What Tools Do We Use?

Jayesh Nalawade — Mon, 11 Nov 2024 11:22:22 +0000

Introduction

In today’s fast-paced tech environment, delivering high-quality software quickly is essential. DevOps—a combination of “Development” and “Operations”—emerges as a solution that bridges the gap between development and operations teams. By focusing on collaboration, automation, and continuous integration/continuous deployment (CI/CD), DevOps enables teams to work more efficiently and deliver software faster with higher quality.

What is DevOps?

DevOps is a set of practices, tools, and cultural philosophies that bring development (software engineering) and IT operations together to shorten the development lifecycle. DevOps emphasizes communication and collaboration, continuous integration and delivery (CI/CD), and ongoing monitoring. This approach ensures that software is always ready for release with minimal errors, resulting in faster, more reliable deployments.

Key objectives of DevOps include:

Improved Collaboration: Breaking down silos between teams.
Faster Releases: Reducing the time it takes to bring new features and updates to production.
Enhanced Stability: Increasing the reliability of systems through automated testing and monitoring.
Higher Efficiency: Automating repetitive tasks and minimizing human errors.

DevOps Lifecycle

The DevOps lifecycle can be broken down into stages: Plan, Code, Build, Test, Release, Deploy, Operate, and Monitor. Each stage represents a phase in the continuous delivery process.

Plan - Define requirements and identify tools.
Code - Develop code and automate testing.
Build - Compile code to create artifacts.
Test - Automatically test code to identify and fix issues.
Release - Ensure that code is ready for deployment.
Deploy - Push code to production or other environments.
Operate - Maintain, troubleshoot, and support the system.
Monitor - Track system performance and feedback for improvements.

Essential DevOps Tools

The success of DevOps heavily relies on the tools used for automation, collaboration, and monitoring. Here’s a breakdown of some popular DevOps tools for each lifecycle stage.

1. Source Code Management

Git: Git is a widely used version control system that helps manage code across multiple contributors.
GitHub, GitLab, Bitbucket: These platforms provide cloud-hosted repositories and support for pull requests, code reviews, and integrations with CI/CD tools.

2. Continuous Integration and Continuous Deployment (CI/CD)

Jenkins: An open-source automation server that supports building, deploying, and automating applications.
GitLab CI/CD: A built-in CI/CD tool in GitLab that allows seamless integration with repositories.
CircleCI and Travis CI: CI/CD platforms that offer fast builds and robust integration options.

3. Configuration Management

Ansible: An automation tool for configuration management, application deployment, and task automation.
Chef and Puppet: Provide infrastructure as code, making it easier to manage server configurations at scale.

4. Containerization and Orchestration

Docker: A platform that allows applications to be packaged with their dependencies in containers.
Kubernetes: A container orchestration tool that automates deploying, scaling, and managing containerized applications.

5. Monitoring and Logging

Prometheus: An open-source monitoring tool that helps monitor system performance.
Grafana: A visualization tool that works well with Prometheus and other data sources to provide real-time system insights.
ELK Stack (Elasticsearch, Logstash, Kibana): A set of tools for search, logging, and visualization, often used for log management.

6. Collaboration and Communication

Slack and Microsoft Teams: Communication platforms that support DevOps notifications, updates, and discussions.
JIRA: A project management tool that helps track issues, manage agile workflows, and support collaboration between teams.

Why DevOps Matters

DevOps offers several key advantages:

Faster Time to Market: Shorter release cycles allow companies to respond to user needs more quickly.
Improved Product Quality: Continuous testing and monitoring help ensure high standards.
Better Team Collaboration: Teams work together, reducing silos and enhancing transparency.
Increased Reliability: Automated deployments reduce the risk of human error and downtime.

Conclusion

Adopting DevOps is essential for organizations that want to stay competitive in today’s software-driven world. With the right practices and tools, DevOps enables faster development cycles, more reliable releases, and improved collaboration across teams. Start exploring DevOps tools today to enhance your workflow and increase productivity.

DEV Community: Jayesh Nalawade

Mastering Kubernetes Traffic: Ingress, Controllers, and Gateway API Explained

Ingress -

Key Components of Ingress

Why Use Ingress?

What is Ingress?

Prerequisites

Types of Ingress

1) Single Service Ingress

2) Simple Fanout Ingress

3) Name-based Virtual Hosting

Kubernetes Ingress Controller

In order for an Ingress to work in your cluster, there must be an ingress controller running.

Ingress controller implements a Kubernetes Ingress and works as a load balancer and reverse proxy entity. It abstracts traffic routing by directing traffic coming towards a Kubernetes platform to the pods inside and load balancing it.

The controller’s routing directions come from the Ingress resource configurations.

Kubernetes Ingress Controller Benefits and Limitations

How to choose an ingress controller

Kubernetes Gateway API

Current State of Ingress in Kubernetes

What is the Kubernetes Gateway API?

When to Use Each

Services in Kubernetes

In Kubernetes, a Service is an abstraction that defines a logical set of Pods and a policy to access them. Services provide a stable endpoint to applications running on Pods, even if the underlying Pods are dynamically created or destroyed.

Key Concepts

There are 3 types of services mainly used in k8s :

a. ClusterIP (Default)-

b. NodePort -(Not safe cause we expose the port)

C. LoadBalancer-

3. Core Concepts of Kubernetes Services

Key Ports in Kubernetes Service

1. port

2. targetPort

3. nodePort (only for NodePort Service type)

Mastering Ansible: Quick and Powerful Ad-Hoc Commands for Efficient Automation

Ansible Ad-hoc commands

Ansible ad-hoc commands have a simple syntax that you can use to perform quick tasks across your infrastructure

Explanation of Parameters:

Common Options for Ad-Hoc Commands:

Host Patterns:

Important Ad-hoc commands

1. Basic Commands for System Management

2.Package Management (Install/Remove Software)

3.File and Directory Management

4. Service Management

5. User and Group Management

6. Network Management

7.Process and System Management

8.Docker Management

9. Log Management

10.Other

Docker ENV vs ARG Explained: Key Differences to Know

Overview

What is ARG?

Key Features of ARG:

Example: Using ARG

What is ENV?

Key Features of ENV:

Example: Using ENV

Comparison: When to Use ARG vs ENV

Key Notes and Best Practices

Conclusion

ENTRYPOINT vs CMD in Dockerfile

CMD (Command)

Example 1: Default command with override

ENTRYPOINT

Example 2: Enforce entry point

Combining ENTRYPOINT and CMD

Example 3: Flexible combination

Key Differences

When to Use Which?

Dockerfile Instructions Guide

1. FROM

2. WORKDIR

3. COPY

4. ADD

5. RUN

6. CMD

7. ENTRYPOINT

8. ENV

9. EXPOSE

3. nodePort (only for `NodePort` Service type)

Combining `ENTRYPOINT` and `CMD`

- This runs an Nginx container and maps port `8080` on the host to port `80` inside the container.