DEV Community: Grigor Khachatryan

Advanced Docker Networking: A Complete Guide

Grigor Khachatryan — Sat, 05 Oct 2024 04:00:35 +0000

Docker has revolutionized the way we develop, ship, and run applications. While many are familiar with containerizing applications, networking within Docker remains a topic that can be daunting for both newcomers and seasoned developers. This guide aims to demystify Docker networking, offering insights that cater to all levels of expertise.

Introduction to Docker Networking
Docker Network Drivers
- Bridge Networks
- Host Networks
- Overlay Networks
- Macvlan Networks
Custom Network Configurations
- Creating a User-Defined Bridge Network
- Connecting Containers to Multiple Networks
Advanced Networking Concepts
- Network Namespaces
- IP Address Management (IPAM)
- Exposing and Publishing Ports
Best Practices and Tips
Conclusion

1. Introduction to Docker Networking

Docker networking allows containers to communicate with each other, the host system, and external networks. Understanding how Docker handles networking is crucial for building scalable and efficient applications.

By default, Docker uses the bridge network driver, creating an isolated network namespace for each container. However, Docker provides several network drivers and options to customize networking to suit various application needs.

2. Docker Network Drivers

Docker offers multiple network drivers, each suited for different use cases.

Bridge Networks

Default network type for standalone containers.
Containers on the same bridge network can communicate using IP addresses or container names.
Suitable for applications running on a single host.

# Create a new bridge network
docker network create my-bridge-network

Host Networks

Containers share the host's network stack.
No network isolation between the container and the host.
Offers performance benefits by reducing network overhead.

# Run a container using the host network
docker run --network host my-image

Overlay Networks

Used for swarm services and multi-host networking.
Enables containers running on different Docker hosts to communicate securely.
Requires a key-value store like Consul or etcd for coordination.

# Create an overlay network
docker network create -d overlay my-overlay-network

Macvlan Networks

Assigns a MAC address to each container, making it appear as a physical device on the network.
Containers can be directly connected to the physical network.
Useful for legacy applications requiring direct network access.

# Create a macvlan network
docker network create -d macvlan \
  --subnet=192.168.1.0/24 \
  --gateway=192.168.1.1 \
  -o parent=eth0 my-macvlan-network

3. Custom Network Configurations

Customizing Docker networks allows for greater control over container communication and network policies.

Creating a User-Defined Bridge Network

User-defined bridge networks provide better isolation and enable advanced networking features like service discovery.

# Create a user-defined bridge network
docker network create my-custom-network

# Run containers on the custom network
docker run -d --name container1 --network my-custom-network my-image
docker run -d --name container2 --network my-custom-network my-image

Connecting Containers to Multiple Networks

Containers can be connected to multiple networks, allowing them to communicate across different network segments.

# Connect an existing container to a network
docker network connect additional-network container1

4. Advanced Networking Concepts

Diving deeper into Docker networking reveals concepts that offer enhanced control and flexibility.

Network Namespaces

Each Docker container runs in its own network namespace, isolating its network environment. Understanding namespaces helps in troubleshooting and customizing network configurations.

# Inspect a container's network namespace
docker inspect -f '{{ .NetworkSettings.SandboxKey }}' container_name

IP Address Management (IPAM)

Docker's IPAM driver manages IP address allocation for networks. Custom IPAM drivers can be used for specific networking requirements.

# Create a network with a specific subnet and gateway
docker network create --subnet=172.20.0.0/16 --gateway=172.20.0.1 my-net

Exposing and Publishing Ports

Exposing Ports: Makes a port available to linked containers.
Publishing Ports: Maps a container port to a host port, making it accessible externally.

# Publish a container's port to the host
docker run -d -p 8080:80 my-image

5. Best Practices and Tips

Use User-Defined Networks: Avoid the default bridge network for better isolation and name resolution.
Leverage DNS: Docker provides built-in DNS for container name resolution within user-defined networks.
Monitor Network Performance: Use tools like docker stats and network plugins to monitor and optimize performance.
Secure Your Networks: Implement network policies and use overlay networks with encryption for secure communication.
Clean Up Unused Networks: Regularly remove unused networks to prevent clutter.

# Remove an unused network
docker network rm my-unused-network

6. Conclusion

Docker networking is a powerful feature that, when understood and utilized correctly, can significantly enhance the scalability and performance of containerized applications. Whether you're just starting out or looking to optimize complex deployments, mastering Docker networking opens up a world of possibilities.

Remember, the key to effectively using Docker networking lies in understanding your application's requirements and choosing the right tools and configurations to meet those needs.

Docker Compose vs Kubernetes: When to Use What

Grigor Khachatryan — Sat, 05 Oct 2024 03:55:55 +0000

In the ever-growing world of containerization, two major players dominate the scene: Docker Compose and Kubernetes. Both are powerful tools, but they serve different purposes depending on the scale and complexity of the project. In this article, we will explore the key differences, use cases, and best practices for using Docker Compose and Kubernetes, helping you decide which tool is best suited for your needs.

Understanding the Basics

Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications. It allows you to manage your services through a simple YAML file, configuring networks, volumes, and dependencies. With a single command, your entire application can be spun up locally, making Docker Compose an excellent choice for development environments and smaller-scale applications.

Kubernetes

Kubernetes, often referred to as K8s, is an open-source container orchestration platform designed for automating deployment, scaling, and managing containerized applications across clusters of machines. While Kubernetes is more complex and requires a steeper learning curve, it offers a powerful suite of features that make it the go-to tool for managing large-scale, production-level applications.

Key Differences Between Docker Compose and Kubernetes

Complexity
- Docker Compose: Simple to set up and manage, making it ideal for smaller projects or development environments.
- Kubernetes: More complex, with a higher learning curve, but designed for large-scale, production deployments.
Scalability
- Docker Compose: Limited scalability, best suited for smaller environments.
- Kubernetes: Highly scalable and capable of managing thousands of containers across clusters of machines.
Service Discovery and Load Balancing
- Docker Compose: Basic service discovery and load balancing, sufficient for smaller setups.
- Kubernetes: Built-in service discovery, load balancing, and routing, making it ideal for managing complex networks.
High Availability and Fault Tolerance
- Docker Compose: Minimal built-in support for high availability and fault tolerance.
- Kubernetes: Offers robust high availability and self-healing features.
Resource Management
- Docker Compose: Basic resource management (CPU and memory limits can be specified).
- Kubernetes: Advanced resource management with options like auto-scaling, resource quotas, and limits.
Updates and Rollbacks
- Docker Compose: Manual updates and rollbacks.
- Kubernetes: Supports automated rolling updates and easy rollbacks with minimal downtime.

When to Use Docker Compose

Ideal Use Cases:

Local Development: Docker Compose makes it easy to set up local environments quickly, ensuring consistency across your team.
Small to Medium Applications: If your application has fewer services and doesn't require complex scaling or high availability, Docker Compose is a great choice.
Prototyping and Testing: Its simplicity allows for rapid prototyping and testing, making it an excellent tool for validating new ideas.
Simple CI/CD Pipelines: Docker Compose can integrate easily into CI/CD pipelines for straightforward application deployments.

Best Practices for Docker Compose:

Version control your Docker Compose files to keep track of changes.
Use environment variables to adapt configurations across different environments.
Leverage YAML anchors and aliases to keep your configuration DRY (Don’t Repeat Yourself).
Implement health checks to ensure all services are up and running as expected.

When to Use Kubernetes

Ideal Use Cases:

Large-Scale Production Deployments: If you're managing a large-scale application with hundreds or thousands of containers across multiple machines, Kubernetes provides the orchestration you need.
Microservices Architectures: For applications built on microservices, Kubernetes excels in handling complex service discovery and load balancing.
High Availability Requirements: Kubernetes offers built-in fault tolerance and self-healing features that ensure your application is highly available.
Auto-Scaling Needs: If your application requires auto-scaling based on demand or resource utilization, Kubernetes is the better option.
Multi-Cloud and Hybrid Cloud Deployments: Kubernetes provides a consistent platform across different cloud providers, making it perfect for multi-cloud and hybrid cloud environments.

Best Practices for Kubernetes:

Use declarative configuration through YAML files for better maintainability.
Implement proper resource requests and limits to optimize performance.
Use namespaces to organize your resources and manage different environments.
Regularly monitor and log your Kubernetes clusters to maintain system health.
Utilize Helm charts to manage complex Kubernetes applications and simplify deployments.

Conclusion

Both Docker Compose and Kubernetes are essential tools in the containerization ecosystem, but they serve different purposes depending on your project’s needs. Docker Compose shines in local development, small applications, and testing scenarios, offering simplicity and speed. Kubernetes, on the other hand, is the go-to solution for large-scale production environments, offering advanced features like auto-scaling, service discovery, and fault tolerance.

Many teams use both tools: Docker Compose for local development and testing, and Kubernetes for production. By understanding the strengths and limitations of each, you can make an informed decision to enhance your development workflow and improve operational efficiency.

Whether you use Docker Compose, Kubernetes, or both, the key is to leverage these tools effectively to ensure your applications are robust, scalable, and easy to manage.

Docker Best Practices: Security

Grigor Khachatryan — Mon, 30 Jan 2023 09:11:24 +0000

Docker is a popular tool for creating and managing containerized applications. However, as with any technology, there are security best practices that should be followed to ensure that your applications and systems remain secure. In this article, we will discuss some of the best practices for securing Docker containers and applications.

1. Use official images

One of the best ways to ensure the security of your Docker containers is to use official images from trusted sources. Official images are those that have been created and maintained by the upstream software vendor or a trusted third-party organization. These images are typically more secure than those that have been created by individuals or untrusted sources, as they are regularly updated and patched to address known vulnerabilities.

For example, when running a containerized version of the Apache web server, you should use the official Apache image from the Docker Hub rather than a version created by an individual or untrusted organization. This will ensure that the image you are using is up-to-date and free from known vulnerabilities.

2. Minimize the attack surface

Another important aspect of securing Docker containers is to minimize the attack surface. This can be achieved by running only the necessary processes and services within a container and by using minimalistic images. Additionally, it is important to keep the number of open ports to a minimum and to use network segmentation to limit the exposure of your containers to the internet.

For example, instead of running a full-fledged Linux distribution within a container, you can use a minimalistic image that includes only the necessary components for your application. Additionally, you can use network segmentation to limit the number of open ports and to restrict access to your container from the internet.

3. Keep your images and containers up-to-date

Keeping your images and containers up-to-date is another important aspect of securing Docker. This includes updating your images to the latest version and patching any known vulnerabilities. Additionally, it is important to regularly update the software and libraries within your containers to ensure that they are free from known vulnerabilities.

For example, if you are running an older version of the Apache web server within a container, it is important to update the image to the latest version to ensure that it is free from known vulnerabilities. Additionally, it is important to regularly update the software and libraries within your container to ensure that they are free from known vulnerabilities.

4. Use container orchestration

Container orchestration tools, such as Kubernetes, can help you to manage and secure your Docker containers at scale. These tools provide features such as automatic scaling, automatic updates, and automatic failover, which can help to ensure that your containers are always running and are free from known vulnerabilities. Additionally, container orchestration tools can be configured to automatically update your images and containers to the latest version and to patch any known vulnerabilities.

5. Use a container firewall

Another important aspect of securing Docker is to use a container firewall. Container firewalls can be configured to restrict incoming and outgoing network traffic and to limit access to your containers from the internet. Additionally, container firewalls can be configured to automatically block any traffic that is deemed to be malicious.

For example, you can use a container firewall to restrict incoming and outgoing network traffic and to limit access to your containers from the internet. Additionally, you can configure the firewall to automatically block any traffic that is deemed to be malicious, such as traffic from known malicious IP addresses or traffic that is trying to exploit known vulnerabilities.

6. Use a vulnerability scanner

A vulnerability scanner is a tool that can scan your images and containers for known vulnerabilities. Vulnerability scanners can be configured to automatically scan your images and containers and to alert you to any known vulnerabilities that they find. Additionally, they can be configured to automatically update your images and containers to the latest version and to patch any known vulnerabilities. Some examples of vulnerability scanners include Aqua Security’s Trivy, Sysdig Secure, and StackRox.

For example, you can use a vulnerability scanner such as Trivy to scan your images and containers for known vulnerabilities. If the scanner finds any vulnerabilities, it will alert you and provide information on how to patch or update the image or container. Additionally, Trivy can also be configured to automatically update your images and containers to the latest version and to patch any known vulnerabilities.

In conclusion, securing Docker containers and applications requires a multi-faceted approach. By following best practices such as using official images, minimizing the attack surface, keeping your images and containers up-to-date, using container orchestration, using a container firewall, and using a vulnerability scanner, you can significantly reduce the risk of vulnerabilities and attacks on your containerized applications. It is important to regularly review and update your security practices, as new vulnerabilities and attack vectors are discovered and the threat landscape changes.

For example, in January 2021, the CVE-2021–3450 vulnerability affected the Docker Engine, allowing an attacker to execute arbitrary code. The vulnerability was patched by Docker in the same month, but it highlights the importance of keeping your images and containers up-to-date and using vulnerability scanners to detect and remediate known vulnerabilities.

Running a GraphQL endpoint with Serverless

Grigor Khachatryan — Mon, 30 Jan 2023 09:07:54 +0000

GraphQL?
In case you are not familiar with it, GraphQL is a data query language created by Facebook. GraphQL allows the client to select ad-hoc data. This differentiates it from REST API’s that expose pre-set data structures. More here

Serverless?
Serverless pattern encourages development focus on well-defined units of business logic, without premature optimization decisions related to how this logic is deployed or scaled.

Setting up the Serverless Framework

Before we start, make sure you have Node.js and npm installed on your machine. You can check this by running the following commands:

node -v
npm -v

Next, install the Serverless Framework globally:

npm install -g serverless

Now that we have the Serverless Framework installed, let’s create a new Serverless project:

serverless create --template aws-nodejs --path graphql-serverless

This will create a new Serverless project using the AWS Node.js template in a directory named graphql-serverless.

Installing the Required Libraries

To run a GraphQL endpoint, we need to install the following libraries:

npm install apollo-server-lambda graphql

apollo-server-lambda is a library that makes it easy to run an Apollo GraphQL server on AWS Lambda. graphql is the library that provides the GraphQL implementation.

Writing the GraphQL Schema

The next step is to write the GraphQL schema. This schema defines the types and the operations that can be performed on these types.

Create a schema.js file in the graphql-serverless directory with the following code:

const { gql } = require("apollo-server-lambda");

const typeDefs = gql`
  type Query {
    hello: String
  }
`;

module.exports = typeDefs;

This is a simple schema that defines a single operation hello that returns a string.

Writing the GraphQL Resolvers

Next, we need to write the resolvers. Resolvers are the functions that implement the operations defined in the schema.

Create a resolvers.js file in the graphql-serverless directory with the following code:

const resolvers = {
  Query: {
    hello: (_, __, context) => {
      return "Hello, Serverless!";
    },
  },
};

module.exports = resolvers;

This resolver implements the hello operation from the schema and returns the string "Hello, Serverless!".

Setting up the Apollo Server

Finally, we need to set up the Apollo Server.

Create a handler.js file in the graphql-serverless directory with the following code:

const { ApolloServer } = require("apollo-server-lambda");
const typeDefs = require("./schema");
const resolvers = require("./resolvers");

const server = new ApolloServer({
  typeDefs,
  resolvers,
});

exports.graphqlHandler = server.createHandler();

This sets up the Apollo Server using the typeDefs and resolvers from the previous sections.

Configuring the Serverless Function

Now that we have the GraphQL endpoint set up, we need to configure the Serverless function to run it.

Update the serverless.yml file in the graphql-serverless directory with the following code:

service: graphql-serverless
provider:
  name: aws
  runtime: nodejs18.x

functions:
  graphql:
    handler: handler.graphqlHandler
    events:
      - http:
          path: graphql
          method: post

plugins:
 - serverless-apollo-middleware

This configures a single Serverless function named graphql that will be triggered by a HTTP POST request to the /graphql endpoint. It also includes the serverless-apollo-middleware plugin, which is required for the Apollo Server to work on AWS Lambda.

Deploying the Serverless Function

The final step is to deploy the Serverless function to AWS. Run the following command in the graphql-serverless directory:

serverless deploy

This will deploy the function to AWS and return the URL of the GraphQL endpoint.

Testing the GraphQL Endpoint

You can use a tool like Insomnia or Postman to test the GraphQL endpoint.

Create a new POST request to the URL of the GraphQL endpoint with the following query in the request body:

query {
  hello
}

Send the request and you should receive the following response:

{
  "data": {
    "hello": "Hello, Serverless!"
  }
}

Continuous Integration and Deployment (CI/CD) using GitHub Actions

To automate the deployment process, we can set up a CI/CD pipeline using GitHub Actions. GitHub Actions allows you to automate your workflow by creating custom actions that run whenever a specific event occurs in your GitHub repository.

Create a new file named .github/workflows/deploy.yml in your GitHub repository with the following code:

name: Deploy

on:
  push:
    branches:
      - main

env:
  AWS_REGION: us-east-1
  AWS_PROFILE: default

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Deploy to AWS
        uses: anothrNick/serverless-action@v1
        with:
          command: deploy

This action is triggered whenever a push is made to the main branch of the GitHub repository. It uses the anothrNick/serverless-action action to deploy the Serverless function to AWS.

Make sure to replace AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the code above with the actual values from your AWS account. You can store these values as secrets in your GitHub repository.

Conclusion

In this article, we showed how to run a GraphQL endpoint with Serverless on AWS Lambda. We covered the basics of setting up an Apollo Server and deploying it as a Serverless function. We also showed how to automate the deployment process using GitHub Actions, allowing for a seamless CI/CD pipeline.

By using Serverless, you can take advantage of the scalability and cost-effectiveness of AWS Lambda while keeping the development process simple and streamlined. The combination of GraphQL and Serverless provides a powerful platform for building and deploying modern web applications.

With the code and steps outlined in this article, you should be able to get started with running a GraphQL endpoint with Serverless and continue building upon this foundation for your own projects.

GraphQL vs REST

Grigor Khachatryan — Mon, 30 Jan 2023 08:54:12 +0000

GraphQL and REST are two popular API architectures for building web applications. Both have been widely used for building modern applications and have their own advantages and disadvantages. In this article, we will compare and contrast GraphQL and REST, explore the benefits of each, and provide code examples to illustrate the differences.

What is GraphQL? GraphQL is a query language for APIs that was developed by Facebook in 2012. It provides a more efficient and flexible alternative to traditional REST APIs by allowing clients to request exactly the data they need, reducing the risk of over-fetching or under-fetching data. GraphQL also includes a type system that defines the data that can be fetched and the operations that can be performed, making it easier for clients and servers to understand the API.

What is REST? REST, or Representational State Transfer, is an architectural style for building web services that was first described by Roy Fielding in 2000. REST APIs use HTTP methods (such as GET, POST, PUT, and DELETE) to perform operations on resources. REST APIs typically return all the data associated with a resource, which can result in over-fetching data.

Code Examples

Let’s look at a simple code example to see the difference between a GraphQL API and a REST API.

Here is an example of a simple GraphQL query to retrieve information about a user:

query {
  user(id: 1) {
    name
    email
  }
}

And here is an equivalent REST API call to retrieve the same information:

GET /users/1

As you can see, the GraphQL query allows you to specify exactly what data you want to retrieve, while the REST API call returns all the data associated with the user resource.

Benefits of GraphQL

Flexibility: GraphQL allows clients to request exactly the data they need, reducing the risk of over-fetching or under-fetching data. This makes it easier to optimize performance and reduce network overhead.
Efficient: GraphQL allows clients to fetch multiple resources in a single request, reducing the number of round trips to the server. This can greatly improve performance, especially for mobile devices with slow or unreliable network connections.
Type System: GraphQL includes a type system that defines the data that can be fetched and the operations that can be performed, making it easier for clients and servers to understand the API.
Versioning: GraphQL has a built-in versioning system through its schema, which allows for new fields and types to be added without breaking existing clients.

Benefits of REST

Simple: REST is a simple and well-established architecture that provides a straightforward way to build APIs.
Backwards Compatible: REST APIs are typically designed to be backwards compatible, which means that new versions of the API can be introduced without breaking existing clients.
Widely Adopted: REST is a widely adopted architecture and has a large community of developers and tools that support it.
Caching: REST APIs can be cached, reducing network overhead and improving performance.

N+1 Problem

The n+1 problem is a common issue in REST API design, where multiple API calls are required to retrieve related data. For example, to retrieve information about a user and their posts, you might need to make two separate API calls: one to retrieve the user information and another to retrieve the list of posts.

In GraphQL, related data can be retrieved in a single API call, avoiding the n+1 problem and improving performance. For example, the following GraphQL query retrieves information about a user and their posts:

query {
  user(id: 1) {
    name
    email
    posts {
      title
      body
    }
  }
}

Overfetching and Underfetching

Overfetching occurs when an API returns more data than a client needs, which can result in increased network overhead and slower performance. Underfetching occurs when an API does not return enough data for a client to complete its task, which can result in additional API calls and reduced performance.

In REST, overfetching and underfetching can be a common issue as the API often returns a fixed set of data for each resource. In GraphQL, the client has control over what data is retrieved, reducing the risk of overfetching or underfetching.

Backward Compatibility

Backward compatibility is an important aspect of API design, as it allows for new versions of the API to be introduced without breaking existing clients. REST APIs are typically designed to be backwards compatible, while GraphQL has a built-in versioning system through its schema. This allows for new fields and types to be added without breaking existing clients, making it easier to evolve the API over time.

Benefits of Schema and Type System

The schema and type system in GraphQL provide a clear and concise definition of the data that can be fetched and the operations that can be performed. This makes it easier for clients and servers to understand the API and reduces the risk of misunderstandings or errors. In addition, the type system helps to ensure that the data returned from the API is consistent and accurate.

In conclusion, GraphQL and REST are two popular API architectures that each have their own advantages and disadvantages. GraphQL provides a more efficient and flexible alternative to REST, while REST is a simple and well-established architecture that is widely adopted. Ultimately, the choice between GraphQL and REST will depend on the specific needs of the project and the preferences of the development team.

About Rust Programming Language

Grigor Khachatryan — Mon, 30 Jan 2023 08:48:38 +0000

Rust is a modern programming language that has become increasingly popular in recent years. It is a statically typed, multi-paradigm language that focuses on speed, reliability, and safety. Rust was created by Graydon Hoare in 2006 and was officially released in 2010. Over the years, Rust has gained a reputation as a language that provides developers with low-level control, strong memory safety guarantees, and excellent performance.

Rust is designed to be an alternative to traditional systems programming languages such as C and C++. These languages provide low-level control and access to the underlying hardware, but they also come with a high risk of bugs, crashes, and security vulnerabilities. Rust aims to provide the same level of low-level control and performance, but with improved memory safety and reliability.

Memory Safety in Rust:

Memory safety is a critical aspect of software development, and it is one of the areas where Rust excels. Unlike C and C++, Rust has built-in memory safety guarantees that prevent common programming errors such as null pointer dereferences and buffer overflows. This is achieved through a combination of strict type checking, ownership, and borrowing rules.

In Rust, each value has a unique owner, and the owner is responsible for freeing the memory when the value is no longer needed. When a value is shared, Rust uses a borrowing system to ensure that the value can only be used in a controlled and safe manner. This helps to prevent bugs and crashes that can occur when multiple parts of a program attempt to access the same memory location at the same time.

Concurrency in Rust:

Another area where Rust excels is concurrency. Concurrent programming involves running multiple parts of a program in parallel, allowing for faster and more efficient processing. Rust supports concurrent programming through its ownership and borrowing model, which helps developers write efficient and correct parallel code.

Rust’s ownership and borrowing system ensures that multiple parts of a program cannot access the same data simultaneously, preventing race conditions and other concurrency-related bugs. Additionally, Rust has a built-in system for synchronizing access to shared data, allowing developers to write safe and efficient concurrent code.

Performance in Rust:

Rust is known for its excellent performance, and it is often compared to C and C++. Rust’s low-level control and memory safety guarantees help to ensure that programs run quickly and efficiently. Additionally, Rust has faster compile times and lower memory overhead than C and C++, making it a great choice for high-performance systems.

Rust’s performance is also helped by its built-in concurrency support, which allows developers to write efficient parallel code. With the rise of multi-core processors, concurrent programming is becoming increasingly important for achieving high performance, and Rust provides the tools and features necessary to build high-performance systems.

Syntax and Ecosystem:

Rust’s syntax is similar to C++, making it easier for C++ developers to learn. This is an important consideration, as it makes it easier for developers to transition from one language to another, and it helps to ensure that the knowledge and experience gained in one language can be easily applied to another.

Rust also has a growing ecosystem, with many popular libraries and tools available. This ecosystem is supported by a large and active community of developers, who contribute to the development of the language and its libraries. This makes it easy for developers to find the tools and resources they need to build their projects.

Conclusion:

Rust is a modern programming language that provides developers with the speed, reliability, and safety they need to build high-performance systems. Whether you’re building a game, a web service, or a system utility, Rust is a great choice for your next project. Its combination of low-level control, memory safety, and concurrency support makes it well suited for systems programming, while its growing ecosystem and active community provide developers with the tools and resources they need to be productive.

Additionally, Rust’s syntax and design philosophy make it a language that is easy to learn and use, while its performance and reliability make it a language that is well suited for production use. Whether you’re a seasoned systems programmer or just starting out, Rust is a language that you should consider for your next project.

In conclusion, Rust is a fast, safe, and reliable programming language that provides developers with the low-level control and performance they need, while also providing the memory safety and concurrency support they need to build robust and scalable systems. With its growing ecosystem, active community, and excellent performance, Rust is a language that is worth exploring for any programmer looking to build high-performance systems.

Dockerfile: ADD vs COPY

Grigor Khachatryan — Thu, 17 Mar 2022 07:23:39 +0000

COPY and ADD are both Dockerfile instructions that serve similar purposes. They let you copy files from a specific location into a Docker image.

COPY

The COPY instruction copies new files or directories from <src> and adds them to the filesystem of the container at the path <dest>.

COPY has two forms:

COPY [--chown=<user>:<group>] <src>... <dest>
COPY [--chown=<user>:<group>] ["<src>",... "<dest>"] (this form is required for paths containing whitespace)

ADD

ADD has two forms:

ADD [--chown=<user>:<group>] <src>... <dest>
ADD [--chown=<user>:<group>] ["<src>",... "<dest>"] (this form is required for paths containing whitespace)

Dockerfile best practice for copying from a URL

Docker suggests that it is often not efficient to copy from a URL using ADD, and it is best practice to use other strategies to include the required remote files.

COPY only supports the basic copying of local files into the container, while ADD has some features (like local-only tar extraction and remote URL support) that are not immediately obvious. Consequently, the best use for ADD is local tar file auto-extraction into the image, as in ADD rootfs.tar.xz /.

— Dockerfile Best Practices
Because image size matters, using ADD to fetch packages from remote URLs is strongly discouraged; you should use curl or wget instead. That way you can delete the files you no longer need after they’ve been extracted and you don’t have to add another layer in your image. For example, you should avoid doing things like:

ADD http://example.com/big.tar.xz /usr/src/things/
RUN tar -xJf /usr/src/things/big.tar.xz -C /usr/src/things
RUN make -C /usr/src/things all

And instead, do something like:

RUN mkdir -p /usr/src/things \
    && curl -SL http://example.com/big.tar.xz \
    | tar -xJC /usr/src/things \
    && make -C /usr/src/things all

For other items (files, directories) that do not require ADD’s tar auto-extraction capability, you should always use COPY.

Like to learn?

Follow me on twitter where I post all about the latest and greatest AI, DevOps, VR/AR, Technology, and Science! Connect with me on LinkedIn too!

Fix: tzdata hangs during Docker image build

Grigor Khachatryan — Mon, 07 Mar 2022 16:24:39 +0000

During the installation of a few packages, Ubuntu usually installs the tzdata package. It’s usually included in some PHP or Python packages dependencies. The issue with it is that it hangs and waits for user input to continue the installation. It’s ok until we are using Docker and trying to build images (it’s hanging or even throwing errors in newer versions of Ubuntu). We will try to reproduce the situation and try to fix it.

To reproduce the hanging situation, we can use this Docker image:

FROM ubuntu:20.04
RUN apt update
RUN apt install -y tzdata

Here is the logs that we see in terminal:

Step 1/3 : FROM ubuntu:20.04
 ---> 1e4467b07108
Step 2/3 : RUN apt update
 ---> Using cache
 ---> 174ce3e1bb84
Step 3/3 : RUN apt install -y tzdata
...
Configuring tzdata
------------------

Please select the geographic area in which you live. Subsequent configuration
questions will narrow this down by presenting a list of cities, representing
the time zones in which they are located.

  1. Africa      4. Australia  7. Atlantic  10. Pacific  13. Etc
  2. America     5. Arctic     8. Europe    11. SystemV
  3. Antarctica  6. Asia       9. Indian    12. US
Geographic area:

And here it hangs waiting for us enter data, and even after you’ll enter a region — the process will not resume.

To fix this situation we need to add lines 3 and 4 to our Dockerfile. We will create a variable called $TZ which will hold our timezone, and the create a /etc/timezone file:

FROM ubuntu:20.04

ENV TZ=Asia/Dubai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN apt update
RUN apt install -y tzdata

And after building image we will see this output:

Step 1/5 : FROM ubuntu:20.04
 ---> 1e4467b07108
Step 2/5 : ENV TZ=Asia/Dubai
 ---> Using cache
 ---> 7f4c85bd0d3e
Step 3/5 : RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
 ---> Using cache
 ---> f6f784dfbad5
Step 4/5 : RUN apt update
 ---> Using cache
 ---> 5b1b5617eaa5
Step 5/5 : RUN apt install -y tzdata
 ---> Running in e71a917a9b6b
Current default time zone: 'Asia/Dubai'
Local time is now:      Tue Aug  4 12:14:55 +04 2020.
Universal Time is now:  Tue Aug  4 08:14:55 UTC 2020.
Run 'dpkg-reconfigure tzdata' if you wish to change it.

Removing intermediate container e71a917a9b6b
 ---> 3d29f4e8f7eb
Successfully built 3d29f4e8f7eb
Successfully tagged tzdata:latest

So it’s used the timezone that we provide and nothing hangs.

Here is the list of Timezones that you can pick one for you:
List of tz database time zones

Like to learn?

Follow me on twitter where I post all about the latest and greatest AI, DevOps, VR/AR, Technology, and Science! Connect with me on LinkedIn too!

Access services in k8s that are not exposed publicly

Grigor Khachatryan — Sat, 05 Mar 2022 20:57:21 +0000

Services running in Kubernetes are not exposed to the public by default so, no one can access them from outside.To access services running in K8s that are not exposed publicly we have few ways which are secure and will not bring to opening security holes in our systems and services. One of them is the Ingress Controller or API Gateway that most of the service meshes are using which basically mapping domains and subdomains to the services.

Another option is Kubernetes CLI (kubectl)using which you can bind any services or POD/container port to your localhost and access private service from your localhost.

Let’s assume we have a service called customer-dashboard and we want to access it using Kubernetes CLI (kubectl).

Connecting to Pod

If we want to connect to Pod/Container we should know the exact name of the Pod, for that we can run this command:

kubectl get pod | grep customer-dashboard

the output would be something similar:

customer-dashboard-7945c779f4-27c5k     1/1   Running   0   4d
customer-dashboard-7945c779f4-885h9     1/1   Running   0   4d

Now when you know the pod name you can bind pods specific port to your localhost using this command:

kubectl port-forward pod/customer-dashboard-7945c779f4-27c5k 8080:80

You will see this output:

Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80

So first port is the port to which you want to bind the service in localhost (so you can access it using 127.0.0.1:8080 or localhost:8080), second port is the pod/container port number.

If you need to open few ports you can do the same for another port too, for that you need to open new terminal tab and type kubectl port-forward customer-dashboard-7945c779f4–27c5k 8081:81 to bind 81 port as well to your 8081 on localhost.

Connecting to Service

Connecting to Service (or internal K8s load balancer) is almost same as connecting to the pod but in this case you will need the service name instead of Pod name. To get the service name type:

kubectl get service | grep customer-dashboard

and you will get the service names:

customer-dashboard  ClusterIP 10.24.2.124  <none>   80/TCP,81/TCP

To bind it to your localhost type:

kubectl port-forward service/customer-dashboard 8080:80

You will see this output:

Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80

Like to learn?

Follow me on twitter where I post all about the latest and greatest AI, DevOps, VR/AR, Technology, and Science! Connect with me on LinkedIn too!

How to Install Docker on Ubuntu 21.10

Grigor Khachatryan — Fri, 04 Mar 2022 14:16:33 +0000

Introduction

Docker is an application that makes it simple and easy to run application processes in a container, which are like virtual machines, only more portable, more resource-friendly, and more dependent on the host operating system.
In this tutorial, you’ll learn how to install and use it on an existing installation of Ubuntu 21.10.

Note: Docker requires a 64-bit version of Ubuntu as well as a kernel version equal to or greater than 3.10. The default 64-bit Ubuntu 21.10 server meets these requirements.

Installing Docker

Note: All the commands in this tutorial should be run as a non-root user. If root access is required for the command, it will be preceded by sudo.

The Docker installation package available in the official Ubuntu 21.10 repository may not be the latest version. To get the latest and greatest version, install Docker from the official Docker repository. This section shows you how to do just that.

First, add the GPG key for the official Docker repository to the system:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add the Docker repository to APT sources:

sudo bash -c 'echo "deb [arch=$(dpkg --print-architecture)] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker-ce.list'

Next, update the package database with the Docker packages from the newly added repo:

sudo apt-get update

Make sure you are about to install from the Docker repo instead of the default Ubuntu 21.10 repo:

apt-cache policy docker-ce

You should see output similar to the follow:

docker-ce:
  Installed: (none)
  Candidate: 5:20.10.12~3-0~ubuntu-impish
  Version table:
     5:20.10.12~3-0~ubuntu-impish 500
        500 https://download.docker.com/linux/ubuntu impish/stable amd64 Packages
     5:20.10.11~3-0~ubuntu-impish 500
        500 https://download.docker.com/linux/ubuntu impish/stable amd64 Packages
     5:20.10.10~3-0~ubuntu-impish 500
        500 https://download.docker.com/linux/ubuntu impish/stable amd64 Packages

Notice that docker-ce is not installed, but the candidate for installation is from the Docker repository for Ubuntu 21.10. The docker-ce version number might be different.
Finally, install Docker:

sudo apt-get install -y docker-ce

Docker should now be installed, the daemon started, and the process enabled to start on boot. Check that it’s running:

sudo systemctl status docker

The output should be similar to the following, showing that the service is active and running:

● docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-02-28 06:54:43 UTC; 8s ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
   Main PID: 2398 (dockerd)
      Tasks: 8
     Memory: 29.9M
        CPU: 375ms
     CGroup: /system.slice/docker.service
             └─2398 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.547798752Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.547839103Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/conta>
Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.547857684Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.601928785Z" level=info msg="Loading containers: start."
Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.803682067Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. D>
Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.912000602Z" level=info msg="Loading containers: done."
Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.935127854Z" level=info msg="Docker daemon" commit=459d0df graphdriver(s)=overlay2 version=20.10.12
Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.935361734Z" level=info msg="Daemon has completed initialization"
Feb 28 06:54:43 ubuntu systemd[1]: Started Docker Application Container Engine.
Feb 28 06:54:43 ubuntu dockerd[2398]: time="2022-02-28T06:54:43.978811872Z" level=info msg="API listen on /run/docker.sock"

Installing Docker now gives you not just the Docker service (daemon) but also the docker command line utility, or the Docker client. Have fun!

Like to learn?

Follow me on twitter where I post all about the latest and greatest AI, DevOps, VR/AR, Technology, and Science! Connect with me on LinkedIn too!

Resilience Engineering – Don't Be Afraid to Show Your Vulnerable Side!

Grigor Khachatryan — Wed, 10 Feb 2021 13:02:54 +0000

Every software developer’s primary goal is to come up with a practical, intuitive, and robust product, a platform or service lots of people can use without any major issue. The problem is that what happens out there with real users is a lot more, well, chaotic than in a control environment developers initially work in. That’s why more and more devs have been using specialized techniques to test out their handiwork and ensure optimal reliability.

Resilience engineering is a practice within Site Reliability Engineering (SRE), closely related to Chaos Engineering. If you’re having trouble wrapping your head around all these terms, don’t worry, we’ll cover each aspect separately and then show you the real magic behind properly applied resilience engineering.

First, here’s a short history lesson.

Where Did SRE Come from?

SRE dates back to almost two full decades ago when a ragtag group of Google devs tried to find a way to improve the reliability of the company’s sites and keep them working smoothly as they grew. Needless to say, these guys were so effective that their techniques and strategies were turned into an IT subset of its own.

It’s an important part of modern DevOps and helps bridge the gap between the initial framework created by developers and the highly practical concerns of real-life system administration.

The Advent of Chaos Engineering

These days, it’s not easy to see an issue coming a mile away and address them in advance to keep a company’s cloud-based platform up and running. And with even just 10-20 minutes of downtime, large corporations stand to lose a lot of potential business, as well as their brand equity. Enter the creatively destructive art of Chaos Engineering.

Think of it as handing the keys to your finely-tuned sedan to a rally driver to run it through its paces on the track and see what breaks first when the system is pushed to its limits.
If you do this on your first batch of sedans, you can go back to the shop and tweak out all the little glitches and potential weak spots, ensuring that the cars run like greased lightning for miles without breaking down when you actually start driving people in them.

The first example of this approach was Netflix’s surprisingly aptly named Chaos Monkey, back in 2010, and Simian Army a year later. It was simple, but it got the job done – it simulated a server failure by shutting down instances at random.

The Practice of Resilience Engineering

Resilience Engineering is all about building systems that can adapt and automatically take the best course of action when common issues occur. Any inadequacies found through testing are ironed out before the system can become truly resilient.

The Flow of a Basic Chaos Experiment

There are several basic steps to testing out the system’s vulnerabilities:

Define a baseline measurement for the system when things are running smoothly.
Come up with an idea of a potential failure.
Test for said failure on a small enough scale so as not to disrupt the whole system but still get measurable data you can act on.
Proceed to compare any issues that have popped up with the baseline performance.
Scale up your tests if no issues were found initially.

What happens with the system is often not the same as what the developers hypothesized would happen, making it an excellent learning opportunity.

The most common issues distributed systems are tested for are server failures – either a single server simply not responding, the network working periodically and crashing, or an entire host of servers going out.

The Ideal System Response to Shoot For

A resilient and well-put-together system will have a quick answer for the most common issues outlined above. For instance, if the cloud provider no longer permits access to a CPU, for whatever reason, the system should respond by connecting to the next best thing.

Also, if the entire network of servers in a particular time zone goes out, the system should look for servers in another region. If the number of users hits a peak in a very short time, the system should naturally scale up and start using more servers to compensate.

New Technologies Allow for Scalability and Automation, Cutting Down on Human Interventions

With the advent of Kubernetes, continuous delivery has been made much easier. The system’s response can be automated – the necessity for human intervention goes down dramatically, and the whole system experiences less downtime as a result. The ability to quickly scale up with sudden bursts of user traffic is incredibly important for cloud-based services.
Imagine giants like Netflix or Blizzard being unable to accommodate all the new users logging on, especially now that everyone is online throughout the day. Any amount of downtime would lead to hoards of unsatisfied customers ready to move on to other services who understand the power of continuous delivery.

Resilient Systems are Reliable and Competitive Systems

While we may have had a little bit of fun flexing our bad pun muscles in the title, the fact is that some companies really are afraid to look for vulnerabilities and address them early on. It’s tempting to start believing the myth of the flawless cloud environment, where all the servers work all the time, there’s no latency, and the number of users using your service barely fluctuates.

However, the reality of it is that if something can go wrong, it eventually will, and chances are you’re not going to be ready for it. Well, if you play it smart and invest in resilience engineering, you’ll save yourself a whole lot of headache down the line.

What is Cloud-Native Computing and How CNCF Contributes to Industry

Grigor Khachatryan — Wed, 09 Dec 2020 16:41:21 +0000

Cloud computing has become the leading method for scaling up workloads and growing businesses at a steady rate. It allows companies to build and run scalable applications in dynamic environments known as clouds.
Cloud technologies allow integration of multiple systems, offering a new platform designed to enable easy management and detailed reporting. With an emphasis on automation, these services allow engineers to make huge changes quickly and effectively. The Cloud-Native Computing Foundation or CNFC is making a push to create an open-source system that allows all users to access new technologies and improve their platforms.

The Importance of Cloud Computing

Scaling up and growing a business requires expanding servers and other technologies designed to handle large amounts of data. However, with ultra-high internet speeds and massive amounts of data generated by websites, extracting the right data can take a long time and a lot of money.
That’s why more and more businesses choose to migrate their websites, data analytics, and other business details onto cloud services. These services are designed to allow fast data analytics and results with automation features designed to speed things up.
The process of migrating data to cloud services provides all kinds of benefits that can help you make better business decisions in the future. Here’s a quick overview of the things you’ll get by using cloud technologies:

Access to new markets, customer base, and previously untargeted areas;
The ability to deliver more value to your customers by understanding their needs better;
The means to directly influence your customer’s behavior;

However, even though cloud services improve resilience and allow better scalability, you need the right management to enjoy the benefits listed above. You have to start small and scale the application up as the need presents itself. Companies such as Twitter and Google both used this method to scale their operations globally and find the right management solutions to keep everything going.

The thing is - cloud technologies are only as useful and beneficial as the managers behind them. If your company doesn’t have the right technologies, training methods, and management needs, you won’t be able to get the most out of this new technology. You must make sure that all internal systems work together, and only then will you be able to scale up your cloud services.

CNCF Making The Transition Much Smoother

As mentioned, making the switch from traditional systems to cloud services can be a difficult, time-consuming task that depends on the volume of the data. You have to tune your entire system with the new, modern distributed system environments that can help you scale things up almost indefinitely.
That’s where CNCF can help you a lot. Its primary goal is to allow merging multiple projects in one native cloud space, such as orchestration services, containers, microservices, etc. These services are then moved to one cloud, allowing seamless integration with current IT solutions already in use. As a result, the migration process becomes much faster and easier to manage.
CNCF offers a set of comprehensive solutions that include features such as container runtime, container orchestration & networking, service development, and management. In other words, it allows full integration with tools such as Kubernetes, CNI, OpenTracing, Prometheus, gRPC, and many others. CNCF can help you to get in tune with the new technologies faster, speeding up the transition from classic systems to cloud services.

The Benefits of Using CNCF

Most major CSPs are already a part of CNCF. They are working together to create new standards for the latest cloud-native technologies and implement them into their CSP platforms. They are working hard to make the integration process as simple as possible, allowing other businesses and enterprises to make the switch to the cloud standard without too much hassle. In other words, CNCF allows direct upgrades and migration to cloud services much more comfortable than it ever was.
Since all major cloud service providers are already a part of the CNCF open standards, companies can use these new technologies to speed up the cloud-merging process and get constant updates with new cloud technologies in the future.
Some cloud-native technologies, including containers, service mash, orchestration, etc., are already available as different services that are provided by various clouds. With a pay-as-you-go model, you can scale your business whenever the need arises.
The entire process allows enterprises to try out new ideas and methods to improve growth by developing new applications all within the cloud itself. You won’t have to invest in new ideas and setups since it’s an open-source type of deal. If your ideas turn out to be useful, the cloud will validate them, allowing you to scale to new applications without changing your existing systems. That is only possible because the entire application is built within the cloud.
By coupling CNCF and CSPs, the real potential of cloud computing technologies will become apparent. It will allow large enterprises to improve sales, drive growth, and create more stable systems within this new dynamic environment called cloud computing.

Changing The Way We Create Software Solutions Forever

Like it or not, the need for cloud services is growing steadily during the past few years. With new technologies reaching the market, and vast amounts of data needed to scale any business and promote growth in the online environment, CNCF will make things much faster and more manageable.
By offering an open-source software stack where companies can work together to improve existing systems and develop new technologies that promote efficiency, enterprises will increase their reach and income without spending massive amounts of money on potential software solutions.
The bottom line is that CNCF allows the industry’s top developers, users, and vendors to put their knowledge together and create new technologies designed to promote growth and scaling much easier than ever before in human history.

DEV Community: Grigor Khachatryan

Advanced Docker Networking: A Complete Guide

Table of Contents

1. Introduction to Docker Networking

2. Docker Network Drivers

Bridge Networks

Host Networks

Overlay Networks

Macvlan Networks

3. Custom Network Configurations

Creating a User-Defined Bridge Network

Connecting Containers to Multiple Networks

4. Advanced Networking Concepts

Network Namespaces

IP Address Management (IPAM)

Exposing and Publishing Ports

5. Best Practices and Tips

6. Conclusion

Docker Compose vs Kubernetes: When to Use What

Understanding the Basics

Docker Compose

Kubernetes

Key Differences Between Docker Compose and Kubernetes

When to Use Docker Compose

Ideal Use Cases:

Best Practices for Docker Compose:

When to Use Kubernetes

Ideal Use Cases:

Best Practices for Kubernetes:

Conclusion

Docker Best Practices: Security

1. Use official images

2. Minimize the attack surface

3. Keep your images and containers up-to-date

4. Use container orchestration

5. Use a container firewall

6. Use a vulnerability scanner

Running a GraphQL endpoint with Serverless

Setting up the Serverless Framework

Installing the Required Libraries

Writing the GraphQL Schema

Writing the GraphQL Resolvers

Setting up the Apollo Server

Configuring the Serverless Function

Deploying the Serverless Function

Testing the GraphQL Endpoint

Continuous Integration and Deployment (CI/CD) using GitHub Actions

Conclusion

GraphQL vs REST

Code Examples

Benefits of GraphQL

Benefits of REST

N+1 Problem

Overfetching and Underfetching

Backward Compatibility

Benefits of Schema and Type System

About Rust Programming Language

Memory Safety in Rust:

Concurrency in Rust:

Performance in Rust:

Syntax and Ecosystem:

Conclusion:

Dockerfile: ADD vs COPY

COPY

ADD

Dockerfile best practice for copying from a URL

Like to learn?

Fix: tzdata hangs during Docker image build

Like to learn?

Access services in k8s that are not exposed publicly

Connecting to Pod

Connecting to Service

Like to learn?

How to Install Docker on Ubuntu 21.10

Introduction

Installing Docker

Like to learn?

Resilience Engineering – Don't Be Afraid to Show Your Vulnerable Side!

Where Did SRE Come from?

The Advent of Chaos Engineering