DEV Community: Syed Sadat Ali

How to Monitor Network Bandwidth of Docker Containers

Syed Sadat Ali — Mon, 16 Sep 2024 17:44:10 +0000

To check the network bandwidth of a Docker instance, you can use several methods depending on what specifics you want to monitor (e.g., overall network usage, per-container bandwidth, etc.). Here’s a step-by-step guide for different approaches:

1. Using Docker Stats Command

The docker stats command provides real-time statistics for containers, including network I/O.

docker stats

This command shows information including network input and output for each running container. The NET I/O column shows the amount of data received and sent by each container.

2. Using Docker Network Inspect

If you need detailed information about a specific network:

docker network inspect <network_name>

This will give you details about the network configuration and connected containers but does not directly show bandwidth usage.

3. Using Network Monitoring Tools

For more detailed bandwidth monitoring, you may need external tools:

iftop or nload: These tools can be installed on the host machine to monitor network traffic. You’ll need to look at traffic on the Docker network interfaces (e.g., docker0 or br-<network_id>).

  sudo iftop -ni docker0

  sudo nload

Prometheus & Grafana: For more comprehensive monitoring, you can set up Prometheus with a Docker exporter and visualize the metrics using Grafana. The Docker exporter can provide metrics on network usage.

4. Using cAdvisor

cAdvisor is a container monitoring tool that provides detailed metrics about Docker containers, including network usage.

Install cAdvisor:

  docker run -d \
    --name=cadvisor \
    --volume=/:/rootfs:ro \
    --volume=/var/run:/var/run:ro \
    --volume=/sys:/sys:ro \
    --volume=/var/lib/docker/:/var/lib/docker:ro \
    --publish=8080:8080 \
    google/cadvisor:latest

Access the cAdvisor Web Interface: Visit http://localhost:8080 to view metrics, including network statistics.

5. Custom Monitoring with Network Tools

You can use network traffic monitoring tools directly within containers to get more granular data:

Install iperf or similar tools within your containers and run tests to measure bandwidth.

  docker exec -it <container_id> iperf -s

  docker exec -it <container_id> iperf -c <server_ip>

Reference : https://docs.docker.com/reference/cli/docker/network/

Mindful Activities

Syed Sadat Ali — Sat, 07 Sep 2024 20:58:51 +0000

Four data structures in Python

Syed Sadat Ali — Sat, 07 Sep 2024 20:45:36 +0000

Four data structures in Python

List:
- Mutable: You can change, add, or remove items after the list creation.
- Ordered: The order of items is maintained, and items can be accessed by their index.
- Syntax: Created using square brackets [] or the list() function.
- Duplicates: Allows duplicate elements.
  - Example: [1, 2, 3, 'apple', 'banana']
Tuple:
- Immutable: Once created, you cannot change, add, or remove items.
- Ordered: Like lists, the order is maintained, and index access is possible.
- Syntax: Created using parentheses () or the tuple() function.
- Duplicates: Allows duplicate elements.
  - Example: (1, 2, 3, 'apple', 'banana')
Set:
- Mutable: You can add or remove items, but you cannot change individual items.
- Unordered: No index access because sets do not record element position.
- Syntax: Created using curly braces {} or the set() function.
- Duplicates: Does not allow duplicate elements; only unique items.
  - Example: {1, 2, 3, 'apple', 'banana'}
Dictionary:
- Mutable: You can change, add, or remove items, which are key-value pairs.
- Ordered: As of Python 3.7, dictionaries maintain insertion order.
- Syntax: Created using curly braces {} with key-value pairs or the dict() function.
- Duplicates: Keys must be unique, but values can be duplicated.
  - Example: {1: 'apple', 2: 'banana', 3: 'cherry'}

Quick summary in code:

# List: Mutable, ordered, allows duplicates
my_list = [1, 2, 2, 'apple']

# Tuple: Immutable, ordered, allows duplicates
my_tuple = (1, 2, 2, 'apple')

# Set: Mutable, unordered, no duplicates
my_set = {1, 2, 'apple'}

# Dictionary: Mutable, ordered (since Python 3.7), unique keys
my_dict = {1: 'apple', 2: 'banana', 3: 'apple'}

Each structure has its own use-cases depending on the requirements of mutability, order, and uniqueness in your Python program.

How to fix RHEL file system

Syed Sadat Ali — Sat, 07 Sep 2024 15:03:03 +0000

How to fix RHEL file system

To boot into rescue mode in Red Hat Enterprise Linux (RHEL) 8 and check the file system for corruption, you can follow these steps:

Boot into Rescue Mode

Reboot the System:
Restart your system.
Access the Boot Menu:
During the boot process, press a key (usually Esc, F2, F12, or a similar key, depending on your system) to access the boot menu.
Select Rescue Mode:
If using GRUB bootloader, press e to edit the boot parameters of the default kernel. Use the arrow keys to navigate to the line that starts with linux and append rd.break at the end of the line. Then press Ctrl + X to boot with these parameters.

Check and Fix File System Corruption

Remount the Root File System: Once you are in the rescue shell, you will need to remount the root file system in read/write mode:

   mount -o remount,rw /sysroot

Change Root into the Sysroot: Change the root to the /sysroot directory:

   chroot /sysroot

Check and Repair the File System: Use the fsck command to check and repair the file system. Replace /dev/sdXN with your actual device identifier (e.g., /dev/sda1):

   fsck /dev/sdXN

For example, to check the root partition, you might run:

   fsck -f /dev/sda1

Exit and Reboot: After the file system check and repair are complete, exit the chroot environment:

   exit

Remount the file system in read-only mode:

   mount -o remount,ro /

Reboot the system:

   reboot

Alternative Method (Using Rescue Media)

Boot from Rescue Media:
Insert the RHEL 8 installation media and boot from it. Select the Troubleshooting option from the boot menu, then select Rescue a Red Hat Enterprise Linux system.
Mount the File System:
Follow the prompts to mount the file system. The installer will detect your RHEL installation and mount it under /mnt/sysimage.
Chroot into the Mounted File System:
Change root into the mounted file system:

   chroot /mnt/sysimage

Check and Repair the File System:
Follow the same steps as above to run fsck and repair the file system.
Exit and Reboot:
Exit the chroot environment and reboot the system:

   exit
   reboot

You should be able to boot into rescue mode in RHEL 8 and check and fix any file system corruption.

Tech Debt

Syed Sadat Ali — Sat, 07 Sep 2024 14:01:36 +0000

To effectively advocate for reducing technical debt, focus on communicating how it can lead to faster delivery, increased productivity, and better value for the company. Here’s how to frame your approach:

Pay Down Tech Debt to Go Faster Immediately

What is Technical Debt?
Technical debt represents the cost of quick fixes, workarounds, and suboptimal code that accumulate over time. It’s like borrowing time from future development efforts, but with "interest" in the form of slower progress, more bugs, and higher maintenance costs.

Proven Approach:

Identify High-Impact Areas: Start by identifying specific, high-impact areas of technical debt that slow down development or increase error rates. Prioritize these issues by their impact on day-to-day work, such as parts of the code that frequently cause bugs or slow down new feature development.
Demonstrate Quick Wins: Provide examples where addressing specific pieces of technical debt has led to immediate improvements, like faster builds, reduced bug counts, or quicker turnaround on feature requests. Use data to back up your points—show how much time or how many resources are spent dealing with recurring issues.
Relate to Business Objectives: Highlight how paying down technical debt aligns with broader business goals, like accelerating time-to-market or improving customer satisfaction. Frame debt reduction not just as a cleanup activity, but as a strategic move that helps the team meet delivery targets faster and with fewer issues.

Use Tech Debt to Increase Productivity

Proven Approach:

Quantify Productivity Losses: Illustrate the impact of technical debt on productivity by quantifying time lost due to rework, debugging, or navigating complex code. For instance, track the number of hours spent fixing issues that stem from technical debt versus working on new features.
Link to Team Well-Being: Emphasize how technical debt contributes to daily stress and impacts team morale. Explain that constantly working around poor code quality is frustrating and demoralizing, which can lead to burnout or reduced performance.
Propose Focused Refactoring Sessions: Suggest dedicating specific time slots, such as a portion of each sprint, to tackle high-priority debt items. This approach integrates debt reduction into the workflow without requiring major shifts in scheduling or priorities, making it easier for management to approve.

Couple Tech Debt Fixes with Value Delivery

Proven Approach:

Align with Feature Development: Suggest coupling technical debt reduction with ongoing feature development. For example, when planning a new feature that interacts with problematic code, propose refactoring that code as part of the feature work. This approach minimizes the perception of technical debt reduction as a separate, time-consuming task.
Show Value Through Examples: Provide case studies or examples where similar approaches have led to noticeable improvements in product stability, performance, or customer satisfaction. Concrete evidence of value delivery can help shift the conversation from abstract concepts to real-world benefits.
Create a Feedback Loop: Set up mechanisms to regularly review the impact of debt reduction efforts, such as retrospectives focused on technical debt or metrics that track improvements in productivity and quality. Sharing these successes can build momentum and encourage ongoing support from management.

Key Points for Management

Faster Delivery: Reducing technical debt directly correlates with faster delivery of features and bug fixes, aligning with business goals of speed and efficiency.
Increased Productivity: Paying down technical debt allows developers to spend more time on productive work and less on firefighting, leading to a more efficient and happier team.
Enhanced Product Quality: A cleaner, more maintainable codebase results in fewer bugs, better performance, and a stronger product overall, contributing to higher customer satisfaction.

By clearly linking technical debt reduction to the immediate and tangible benefits that management values—speed, productivity, and quality—you can make a more compelling case for prioritizing this work. Present your argument with data and examples that resonate with the broader business objectives, and emphasize that addressing technical debt is not just a technical necessity, but a strategic investment in the team's and the product's future success.

The impact of technical debt on a software project and its stakeholders can be significant, affecting everything from code quality to team morale.

*Understanding these impacts is crucial for making the case to reduce technical debt and to motivate teams to prioritize quality alongside speed. *

Here’s a breakdown of the various ways technical debt can impact a project:

1. Reduced Development Velocity

Technical debt often leads to slower development over time because:

Complex Code: Developers spend more time understanding and working around complex, messy, or poorly documented code.
Frequent Bugs: Increased bugs and errors mean more time spent on fixes and less on new feature development.
Rework and Refactoring: As debt accumulates, the need for rework increases, consuming time that could be spent on innovation.

Impact: Overall, this reduces the team's ability to deliver features quickly, impacting deadlines and slowing down the project’s progress.

2. Increased Maintenance Costs

As technical debt accumulates, the cost of maintaining the software rises due to:

Time-Consuming Debugging: Debugging becomes more challenging in a codebase riddled with quick fixes and workarounds.
Higher Testing Efforts: Poor code quality can lead to more complex and extensive testing requirements, as the code may behave unpredictably.
Dependencies Management: Outdated or poorly managed dependencies add complexity, often leading to compatibility issues and security risks.

Impact: The increasing need for maintenance drives up operational costs, reducing the overall return on investment for the project.

3. Poor Product Quality

Technical debt can significantly impact the quality of the product, leading to:

Performance Issues: Suboptimal code can cause performance bottlenecks, resulting in slower applications and poor user experiences.
Increased Bugs: A high volume of technical debt typically correlates with a higher incidence of bugs, negatively affecting reliability and customer satisfaction.
Scalability Challenges: As the codebase becomes more complex and intertwined, scaling the product or adding new features becomes increasingly difficult.

Impact: Reduced product quality can lead to customer dissatisfaction, increased churn, and damage to the company's reputation.

4. Lower Team Morale and Increased Stress

The presence of technical debt can have a significant impact on the development team's morale:

Frustration: Developers may feel frustrated and demotivated when constantly dealing with poor code quality and recurring issues.
Burnout: The stress of managing technical debt can contribute to burnout, especially when deadlines are tight, and the pressure to deliver is high.
Turnover: Persistent technical debt can drive talented developers to leave, seeking environments where they can work on higher-quality code and more rewarding projects.

Impact: High turnover and low morale can disrupt team cohesion, leading to a loss of institutional knowledge and further slowing down the project.

5. Difficulty in Onboarding New Developers

Technical debt can make it challenging to onboard new developers:

Steep Learning Curve: New developers may struggle to understand a complex, undocumented, or poorly structured codebase.
Reduced Productivity: It takes longer for new team members to become productive, as they spend more time deciphering existing debt-laden code.

Impact: Slower onboarding can delay project timelines and reduce the overall efficiency of the team.

6. Increased Risk and Reduced Agility

Technical debt can introduce significant risks and reduce the agility of the development process:

Unpredictable Behavior: Quick fixes and workarounds can lead to code that behaves unpredictably, making the software less reliable.
Limited Flexibility: As debt grows, making changes or adapting to new business requirements becomes increasingly difficult, reducing the team's ability to respond to market needs quickly.

Impact: The project becomes less adaptable, increasing the risk of missed opportunities or failure to meet business objectives.

7. Negative Impact on Customer Satisfaction

For end-users, the effects of technical debt can manifest as:

Decreased Reliability: Users may experience more downtime, bugs, or inconsistent performance due to the underlying technical issues.
Poor User Experience: Performance issues or frequent updates to fix bugs can disrupt the user experience, leading to frustration.

Impact: Dissatisfied customers are less likely to continue using the product, potentially harming the business’s growth and profitability.

Addressing technical debt early and systematically is essential to mitigating these impacts. By reducing debt, teams can improve productivity, lower maintenance costs, enhance product quality, and create a healthier work environment—all of which contribute to the long-term success of the project.