DEV Community: Oliver Bennet

Automate Repetitive Tasks: Creating Bash Scripts for Everyday Use 🚀

Oliver Bennet — Thu, 05 Dec 2024 07:52:49 +0000

Did you know? IT professionals spend a significant portion of their time on repetitive tasks. Imagine reclaiming that time to focus on innovation and strategic goals. That’s exactly what Bash scripting can help you achieve! 💡

In this post, I'll give you a sneak peek into the power of Bash scripting and share why learning it is an essential skill for developers, sysadmins, and automation enthusiasts alike.

Why Learn Bash Scripting?

Bash scripting is the Swiss Army knife of automation. Whether you're managing servers, processing data, or performing routine system tasks, Bash lets you:

🕒 Save time by automating repetitive workflows.
🤖 Reduce human error with consistent execution.
💻 Empower your Linux experience by turning commands into powerful scripts.

What You’ll Learn in the Full Guide

🔗 In my detailed article, I cover:

🧱 Building Blocks of Bash: Variables, loops, conditionals, and functions.
⚙️ Real-World Examples: Automating backups, system updates, and log monitoring.
🚀 Advanced Techniques: Error handling, logging, and integrating with tools like jq and cron.
✨ Best Practices: How to write clean, reusable, and efficient scripts.

Who Should Read This?

This guide is perfect for:

👶 Beginners looking to learn Bash scripting from scratch.
🛠️ Intermediate users who want to enhance their automation skills.
🌟 Professionals aiming to streamline their workflows and boost productivity.

Ready to Master Automation?

👉 Read the Full Guide: Automate Repetitive Tasks: Creating Bash Scripts for Everyday Use

Automate Repetitive Tasks: Creating Bash Scripts for Everyday Use | by Oliver Bennet | Dec, 2024 | Medium

Oliver Bennet ・ Dec 5, 2024 ・
graphpe.Medium

Feel free to ask questions, share your favorite Bash tricks, or suggest what you'd like to see in future posts. Let’s make automation accessible and fun! 💬

If you find this helpful, don’t forget to leave a reaction ❤️, comment, and share to help others discover the magic of Bash scripting!

Bash Basics: A Beginner’s Guide to Shell Scripting

Oliver Bennet — Wed, 04 Dec 2024 12:31:23 +0000

Did you know that over 90% of the world's supercomputers run on Linux. Behind the seamless performance of these systems lies the power of Bash scripting. From automating repetitive tasks to orchestrating complex workflows, Bash scripting is a cornerstone of system administration and development.

Whether you're managing servers, configuring systems, or simply looking to make your day-to-day tasks more efficient, learning Bash scripting can open doors to endless possibilities. we will talk about the step by step into the world of Bash scripting, making it approachable for absolute beginners.

1. What is Bash Scripting?

Bash scripting involves writing a sequence of commands in a plain text file and executing them in the Bourne Again Shell (Bash). Bash is the default shell on most Linux distributions, and it allows users to interact with the system via the command line or scripts.

Why Bash?
Bash is widely regarded as the standard for scripting because of its:

Ease of Use: It simplifies command-line tasks.
Ubiquity: Installed by default on most Unix-like systems.
Versatility: Can automate tasks ranging from simple file manipulations to complex system monitoring.

For instance, instead of typing multiple commands to back up files every day, you can write a Bash script to do it automatically.

2. Getting Started with Bash

Before diving into scripting, let’s set up the basics:

How to Create a Bash Script

Open a text editor (e.g., nano, vim, or gedit).
Write your commands in the file.
Save the file with a .sh extension (e.g., myscript.sh).

Making the Script Executable
To run your script, you need to make it executable:

chmod +x myscript.sh

Running a Script
Execute the script with:

./myscript.sh

First Script Example
Here’s a simple script that prints "Hello, World!" to the terminal:

#!/bin/bash
# This is a comment
echo "Hello, World!"

#!/bin/bash: This shebang line tells the system to use Bash to interpret the script.
echo: A command to print text to the terminal.

3. Understanding Bash Basics

Variables
Variables store data for reuse.

name="Alice"
echo "Hello, $name!"

Variables are case-sensitive.
Use $ to access the variable’s value.

User Input
Read user input with the read command:

echo "Enter your name:"
read name
echo "Welcome, $name!"

Conditional Statements
Bash supports if-else constructs for decision-making:

#!/bin/bash
echo "Enter a number:"
read number
if [ $number -gt 10 ]; then
    echo "The number is greater than 10."
else
    echo "The number is 10 or less."
fi

[ ]: Tests conditions.
-gt: Greater than.

Loops
Automate repetitive tasks using loops.

For Loop

#!/bin/bash
for i in {1..5}
do
    echo "Number: $i"
done

While Loop

#!/bin/bash
count=1
while [ $count -le 5 ]
do
    echo "Count: $count"
    ((count++))
done

4. Practical Applications

File Operations
Automate file management tasks:

Create a File

touch myfile.txt

Check If a File Exists

if [ -f myfile.txt ]; then
    echo "File exists."
else
    echo "File does not exist."
fi

Automated Backups
Create a backup of a directory:

#!/bin/bash
src="/home/user/documents"
dest="/home/user/backup"
tar -czf "$dest/backup_$(date +%F).tar.gz" "$src"
echo "Backup completed."

System Monitoring
Monitor disk usage:

#!/bin/bash
usage=$(df -h / | grep / | awk '{print $5}' | sed 's/%//')
if [ $usage -gt 80 ]; then
    echo "Warning: Disk usage is at $usage%!"
else
    echo "Disk usage is under control at $usage%."
fi

5. Best Practices in Bash Scripting

Use Comments: Explain your code for readability.

# This script checks disk usage

Error Handling: Anticipate and handle errors gracefully.

if [ ! -d "$src" ]; then
    echo "Source directory does not exist."
    exit 1
fi

Avoid Hardcoding: Use variables for paths and configurations.
Indentation: Keep your code organized.

6. Debugging and Troubleshooting

Debugging Tips

Run the script in debug mode:

bash -x myscript.sh

Check for Syntax errors

bash -n myscript.sh

Common Errors

Permission Denied: Ensure the script is executable with chmod +x.
Command Not Found: Verify the script’s shebang line and paths.

7. Expanding Your Skills

As you grow more comfortable with Bash scripting, explore advanced topics such as:

Functions: Organize reusable code.
Arrays: Handle lists of items.
Cron Jobs: Schedule scripts to run automatically.
Text Processing: Use tools like awk and sed.

Conclusion

Bash scripting is an invaluable skill for anyone working in Linux or Unix environments. By mastering the basics of variables, loops, and conditional statements, you can create scripts to automate tasks, save time, and increase efficiency.

Start small, experiment with practical scripts, and gradually build more complex solutions. Remember, the best way to learn is by doing—so dive in and start scripting today!

If you want to continue knowing more on Bash. Check out this Article I Published on Medium today.

Mastering Bash Variables and Parameters | by Oliver Bennet | Dec, 2024 | Medium

Oliver Bennet ・ Dec 4, 2024 ・
graphpe.Medium

Automating Network Devices with Python and Netmiko: A Comprehensive Guide

Oliver Bennet — Sat, 23 Nov 2024 06:30:43 +0000

Introduction: The Real-World Impact of Network Automation

In a 2023 report by Gartner, organizations that implemented network automation saw operational costs decrease by up to 40% and configuration errors reduce by over 70%.

Cisco further highlights that manual configuration accounts for over 75% of network downtime, costing enterprises an average of $5,600 per minute of downtime. These figures underline the critical role automation plays in modern network management.

Automation not only saves money but also reduces human error, accelerates deployment, and ensures consistent configurations across devices.

For network engineers and administrators, the demand for automation is no longer optional but a necessity in large-scale IT environments.

Among the numerous tools available for automating network devices, Python paired with Netmiko stands out for its simplicity, flexibility, and vendor-agnostic approach.

In this article, we'll explore how you can use Python and Netmiko to automate real-world network tasks, starting with basic use cases and progressing to advanced and complex scenarios.

Understanding Netmiko and Its Benefits

Netmiko is an open-source Python library built on top of Paramiko, designed specifically for managing network devices. Its abstraction layer simplifies SSH interactions and supports multiple vendors, making it an essential tool for network automation.

Key Features of Netmiko

Multi-Vendor Support: Includes Cisco, Juniper, Arista, HP, Dell, and more.
Ease of Use: Simplifies SSH-based interactions and configuration tasks.
Scalability: Automates tasks for hundreds of devices in parallel.
Customizability: Tailors scripts to specific network requirements.

Use Cases for Automating Network Devices

Let’s dive into real-world use cases that demonstrate the power of Python and Netmiko in network automation.

1. Bulk Configuration Changes

In large organizations, updating configurations across hundreds or thousands of devices is a frequent but tedious task. For example:

Updating VLANs across all switches.
Applying QoS policies to prioritize traffic.
Enabling or disabling specific protocols.

Example: Configuring VLANs Across Multiple Switches

from netmiko import ConnectHandler

# List of devices
devices = [
    {"device_type": "cisco_ios", "ip": "192.168.1.1", "username": "admin", "password": "password"},
    {"device_type": "cisco_ios", "ip": "192.168.1.2", "username": "admin", "password": "password"},
]

# VLAN configuration commands
commands = [
    "vlan 10",
    "name Sales_VLAN",
    "vlan 20",
    "name HR_VLAN",
]

# Automating the changes
for device in devices:
    connection = ConnectHandler(**device)
    connection.send_config_set(commands)
    connection.save_config()
    connection.disconnect()

Impact: Automating such tasks reduces manual effort from hours to minutes while ensuring consistency across the network.

2. Backup Configurations

Configuration backups are crucial for disaster recovery. Manual backups are time-consuming and prone to being overlooked.

Example: Automating Backup of Running Configurations

from netmiko import ConnectHandler

# Device details
device = {
    "device_type": "cisco_ios",
    "ip": "192.168.1.1",
    "username": "admin",
    "password": "password",
}

# Connect and fetch configuration
connection = ConnectHandler(**device)
config = connection.send_command("show running-config")

# Save configuration to a file
with open(f"{device['ip']}_backup.txt", "w") as file:
    file.write(config)

connection.disconnect()

Impact: Automating backups ensures that the latest configurations are always available, reducing downtime in case of failures.

3. Compliance Auditing

Compliance with organizational policies or industry regulations requires regular auditing of device configurations. For example:

Ensuring SSH and SNMP are securely configured.
Verifying firewall rules.
Detecting unauthorized changes.

Example: Auditing SNMP Configuration

from netmiko import ConnectHandler

device = {
    "device_type": "cisco_ios",
    "ip": "192.168.1.1",
    "username": "admin",
    "password": "password",
}

connection = ConnectHandler(**device)
output = connection.send_command("show running-config | include snmp")

# Verify SNMP settings
if "snmp-server community public RO" not in output:
    print(f"{device['ip']} is non-compliant!")

connection.disconnect()

Impact: Detects non-compliant devices in real-time, ensuring that configurations adhere to security policies.

4. Real-Time Device Monitoring

Network devices often require real-time monitoring to track their health and performance. This includes:

Monitoring CPU, memory, and interface utilization.
Detecting high latency or dropped packets.

Example: Checking Interface Status

from netmiko import ConnectHandler

device = {
    "device_type": "cisco_ios",
    "ip": "192.168.1.1",
    "username": "admin",
    "password": "password",
}

connection = ConnectHandler(**device)
output = connection.send_command("show ip interface brief")

# Parse the output to detect interfaces that are down
for line in output.splitlines():
    if "administratively down" in line:
        print(f"Interface issue found: {line}")

connection.disconnect()

Impact: Alerts administrators about potential issues before they escalate into outages.

Advanced and Complex Use Cases

5. Multi-Vendor Network Orchestration

Large enterprises often have multi-vendor environments with Cisco, Juniper, and Arista devices. Writing automation scripts for each vendor is cumbersome, but Netmiko simplifies this with vendor-specific device types.

Example: Retrieving Version Information Across Vendors

devices = [
    {"device_type": "cisco_ios", "ip": "192.168.1.1", "username": "admin", "password": "password"},
    {"device_type": "juniper", "ip": "192.168.1.2", "username": "admin", "password": "password"},
    {"device_type": "arista_eos", "ip": "192.168.1.3", "username": "admin", "password": "password"},
]

for device in devices:
    connection = ConnectHandler(**device)
    output = connection.send_command("show version")
    print(f"Version information for {device['ip']}:\n{output}")
    connection.disconnect()

6. Firmware Upgrades

Firmware upgrades are critical but disruptive tasks that require careful orchestration. Automating the process minimizes downtime and human error.

Example: Automating Firmware Uploads

from netmiko import ConnectHandler

device = {
    "device_type": "cisco_ios",
    "ip": "192.168.1.1",
    "username": "admin",
    "password": "password",
}

connection = ConnectHandler(**device)

# Upload firmware
connection.send_command("copy tftp://192.168.1.100/new_firmware.bin flash:")

# Reload device
connection.send_command("reload")

connection.disconnect()

Impact: Enables efficient and error-free firmware management for hundreds of devices.

7. Zero-Touch Provisioning

When new devices are added to the network, manually provisioning them can delay deployment. Automating the initial setup reduces time-to-service.

Example: Configuring a New Router

from netmiko import ConnectHandler

new_device = {
    "device_type": "cisco_ios",
    "ip": "192.168.1.50",
    "username": "admin",
    "password": "password",
}

# Initial configuration
commands = [
    "hostname NewRouter",
    "interface GigabitEthernet0/1",
    "ip address 192.168.1.1 255.255.255.0",
    "no shutdown",
    "ip route 0.0.0.0 0.0.0.0 192.168.1.254",
]

connection = ConnectHandler(**new_device)
connection.send_config_set(commands)
connection.save_config()
connection.disconnect()

Impact: Speeds up deployment while ensuring consistency in configurations.

Optimizing Automation with Advanced Techniques

1. Parallel Execution

For large-scale networks, automating sequentially can be slow. Use Python’s concurrent.futures to execute tasks in parallel.

from concurrent.futures import ThreadPoolExecutor
from netmiko import ConnectHandler

def execute_command(device):
    connection = ConnectHandler(**device)
    output = connection.send_command("show ip route")
    connection.disconnect()
    return output

devices = [
    {"device_type": "cisco_ios", "ip": "192.168.1.1", "username": "admin", "password": "password"},
    {"device_type": "cisco_ios", "ip": "192.168.1.2", "username": "admin", "password": "password"},
]

with ThreadPoolExecutor(max_workers=5) as executor:
    results = executor.map(execute_command, devices)

for result in results:
    print(result)

2. Centralized Logging

Integrate logging to track script execution and errors.

import logging

logging.basicConfig(level=logging.INFO)
logging.info("Starting network automation script")

Summary

Network automation with Python and Netmiko is transforming how organizations manage their IT infrastructure. From reducing operational costs and human errors to accelerating deployments, the benefits are undeniable. By starting with basic use cases and progressing to complex scenarios like multi-vendor orchestration and firmware upgrades, Netmiko empowers engineers to build scalable and efficient solutions.

Automation isn't just about efficiency—it’s about enabling innovation, improving reliability, and future-proofing your network in an ever-evolving landscape.

Oliver | GraphPe | Medium | Youtube

🚀 Top 10 sed Command Use Cases + 5 Real-Life Scenarios!

Oliver Bennet — Wed, 20 Nov 2024 11:32:32 +0000

The sed command is a game-changer for Linux users! Whether you're a sysadmin, developer, or open-source enthusiast, sed simplifies text processing and file management.

What’s Inside?
💡 Top 10 sed Use Cases: From basic substitutions to multi-line edits.

🔧 5 Real-Life Scenarios:
1️⃣ Updating hostnames in Nagios config files
2️⃣ Replacing SMTP server details in Grafana
3️⃣ Modifying database credentials in config files
4️⃣ Updating IPs in Nginx configurations
5️⃣ Adjusting paths in scripts after restructuring

These examples highlight how sed saves time and prevents manual errors in everyday IT tasks.

📖 Read the full article here

Top 10 SED Use Cases Every Linux User Should Know | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 20, 2024 ・
graphpe.Medium

💬 Have a favorite sed use case? Share it in the comments!

Let’s master the art of text automation together! 🚀 #Linux #DevOps #OpenSource

🔐 Why Your Team Needs Passbolt for Password Management

Oliver Bennet — Wed, 20 Nov 2024 11:28:49 +0000

Managing passwords across teams can be challenging, especially when security is a top priority. Passbolt, an open-source password manager, is here to help. Designed specifically for teams and businesses, Passbolt offers robust features to keep your credentials safe and accessible.

Why Passbolt?

End-to-End Encryption: Your passwords, encrypted and secure.
Team Collaboration: Seamlessly share passwords with team members.
Self-Hosting: Take full control of your data.
Browser Extensions: Access and autofill passwords effortlessly.
Role-Based Access Control (RBAC): Manage access with precision.

Whether you're securing API keys, database credentials, or login details, Passbolt ensures your team stays secure and productive.

💻 Check out our in-depth guide: Passbolt in detail

Passbolt: A Comprehensive Guide to Secure Password Management for Teams | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 18, 2024 ・
graphpe.Medium

🔗 Let’s discuss! How do you manage passwords in your team? Share your thoughts in the comments!

🎯 Master Wireshark: 15 Essential Filters for Cybersecurity Professionals

Oliver Bennet — Wed, 20 Nov 2024 11:26:22 +0000

Wireshark is a must-have tool for network analysis, but mastering its filters can take your skills to the next level. In this guide, we’ve compiled 15 practical Wireshark display filters every cybersecurity analyst should know.

🔍 What’s Inside?

Filter by IP addresses or ranges
Analyze specific TCP/UDP ports
Monitor HTTP methods, status codes, and cookies
Capture DNS queries or TLS handshakes
Includes real-world commands and outputs

💡 Whether you're debugging networks, analyzing attacks, or monitoring performance, these filters are your secret weapon!

📖 Read the complete guide now: Check it out here

Wireshark Display Filters: Essential Filters Every Cybersecurity Analyst Should Master | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 19, 2024 ・
graphpe.Medium

💬 What’s your favorite Wireshark filter? Drop your thoughts in the comments!

Wireshark #Cybersecurity #NetworkTools #OpenSource

Essential Linux Text Manipulation Tools: cut, sort, uniq, tr 🛠️

Oliver Bennet — Wed, 20 Nov 2024 11:23:34 +0000

Linux's text processing commands are the Swiss Army knives for developers and sysadmins. In this post, I break down four essential tools—cut, sort, uniq, and tr—that can simplify your workflows:

🔥 Key Features
🔥 40+ Practical Examples
🔥 Real-World Use Cases

💡 Whether you're a beginner or an advanced user, this guide will help you unlock new efficiencies.

👉 Check it out here: 4 Most used text manipulation tools

4 Most Essential Text Manipulation Tools in Linux: cut, sort, uniq, tr | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 20, 2024 ・
graphpe.Medium

Have any creative use cases of your own? Let's discuss in the comments!

Linux #TextProcessing #CLI #OpenSource #DevOps #BashScripting

What's a Quickie Post?

Oliver Bennet — Tue, 19 Nov 2024 05:45:41 +0000

A Quickie Post is a new, concise way to share quick tips, code snippets, or random thoughts on Dev.to. It's perfect for those bite-sized ideas that don't need a full-blown article.

Why Use Quickie Posts?

I personally think below are some of the features for Quickie Post.

Share Quickly: Get your thoughts out there fast.
Keep it Simple: No need for long-form content.
Engage the Community: Spark discussions with focused topics.
Give it a Try!

Comment and let's discuss what are the other features that you might want Dev.to team to include in Quickie Post

Linux Kernel 6.12: A Quantum Leap in Performance and Features

Oliver Bennet — Mon, 18 Nov 2024 07:54:50 +0000

Linux Kernel 6.12 is a significant release that brings a plethora of enhancements, new features, and improved hardware support. This powerful kernel is set to revolutionize the Linux experience for users across various platforms.

Real-time Performance Revolution

One of the most significant advancements in Linux Kernel 6.12 is the inclusion of PREEMPT_RT mainline support. This long-awaited feature significantly boosts real-time performance, making Linux an ideal choice for time-critical applications like industrial automation, robotics, and real-time audio/video processing.
By enabling the kernel to respond to interrupts and context switches more promptly, PREEMPT_RT minimizes latency and jitter, ensuring that time-sensitive tasks are executed with precision. This level of real-time responsiveness was previously challenging to achieve on Linux, but with PREEMPT_RT, it's now a reality.

Expanded Hardware Ecosystem

Linux Kernel 6.12 continues to expand its hardware support, embracing a diverse range of platforms:

-Raspberry Pi 5: Initial mainline support for the latest Raspberry Pi model, further solidifying Linux as a preferred OS for hobbyists and developers.

ARM-powered GameForce Ace: Optimized support for this popular gaming handheld, enhancing gaming performance and compatibility.
ODROID-M15 and ODROID-M2: Improved driver support for these versatile single-board computers, enabling smoother operation and better utilization of their capabilities.
OneXPlayer: Enhanced sensor support for this gaming device, providing a more immersive and responsive gaming experience.

Performance Tweaks and Optimizations

The kernel's performance continues to be refined with several optimizations:

Intel P-State Driver: The hybrid CPU scaling feature in the Intel P-State driver is further optimized for upcoming Intel Core Ultra 2000 chips, leading to improved power efficiency and performance.
AMD P-State Driver: Enhanced support for AMD Boost and AMD Preferred Core features allows for better utilization of AMD CPU resources, resulting in increased performance and responsiveness.

Developer Tools and Ecosystem

SWIG Bindings for libcpupower: This addition simplifies scripting and extension of libcpupower functionality, making it easier for developers to - create custom tools and scripts to manage CPU power states and performance.

Other Notable Features

QR Code Support in Kernel Panic Screens: This innovative feature facilitates easier debugging and troubleshooting by providing a QR code that can be scanned to access detailed information about the system's state at the time of the crash.
RISC-V CPU ISA Extensions: Expanded support for RISC-V architectures, ensuring compatibility with a wider range of RISC-V-based hardware.
Simplified Microcode Patch Loading: This optimization streamlines the process of loading microcode patches on AMD Zen and newer CPUs, improving system stability and security.
SPI-NAND Continuous Reads: Enhanced performance for SPI-NAND storage devices, leading to faster data transfer rates and smoother overall system performance.
Regmap IRQDomain Custom Names: This feature provides greater flexibility in configuring interrupt domains, making it easier to manage complex hardware setups.
SPI Bus Idle State Configuration: More precise control over the behavior of the SPI bus during idle periods, allowing for better power management and reduced interference with other devices.
vDSO getrandom for Various Architectures: Expanded availability of a faster random number generator, improving the security and randomness of cryptographic operations.

A Deeper Dive into Real-time Performance

To fully understand the impact of PREEMPT_RT, let's delve deeper into its technical implications:

Kernel Preemption: PREEMPT_RT enables the kernel to preempt running tasks, allowing it to switch to higher-priority tasks more quickly. This reduces latency and jitter, ensuring that time-critical tasks are executed promptly.
Latency Reduction: By minimizing the time it takes for the kernel to respond to interrupts and context switches, PREEMPT_RT significantly reduces system latency. This is crucial for applications that require low-latency responses, such as real-time audio and video processing.
Jitter Minimization: PREEMPT_RT helps to minimize jitter, which is the variation in the timing of events. This is essential for applications that require precise timing, such as robotics and industrial automation.

Real-world Use Cases and Benefits

Industrial Automation: PREEMPT_RT enables the development of highly responsive and deterministic control systems for industrial automation, ensuring precise control of machinery and processes.
Robotics: Real-time performance is critical for robotics applications, as it allows robots to react quickly to changes in their environment. PREEMPT_RT helps to improve the responsiveness and accuracy of robotic systems.
Real-time Audio/Video Processing: For applications like audio and video streaming, low-latency and jitter-free performance is essential.
PREEMPT_RT helps to minimize latency and jitter, ensuring smooth and uninterrupted playback.

The Future of Linux Kernel

Linux Kernel 6.12 represents a significant milestone in the evolution of the Linux kernel. With its focus on real-time performance, hardware support, performance optimizations, and developer tools, this release empowers users and developers alike to unlock the full potential of their Linux systems.

As the Linux kernel continues to evolve, we can expect even more exciting features and improvements in future releases. The open-source community's collaborative efforts will undoubtedly drive the kernel's development, ensuring that Linux remains a powerful and versatile operating system for years to come.

By embracing the power of Linux Kernel 6.12, users can experience a quantum leap in performance, stability, and functionality.

Effortless File Transfers: A Guide to Using scp and rsync | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 15, 2024 ・
graphpe.Medium

25 Advanced Bash Commands Every Linux User Should Know | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 8, 2024 ・
graphpe.Medium

Top 10 SED Use Cases Every Linux User Should Know | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 5, 2024 ・
graphpe.Medium

Linux Aliases You Didn’t Know You Needed: Practical Shortcuts for Every User | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 15, 2024 ・
graphpe.Medium

Prometheus 3.0: A Quantum Leap in Monitoring

Oliver Bennet — Mon, 18 Nov 2024 07:06:01 +0000

Prometheus, the open-source systems monitoring and alerting toolkit, has long been a favorite among DevOps engineers and system administrators. Its powerful query language, PromQL, and flexible data model have made it a go-to tool for monitoring complex systems. With the recent release of Prometheus 3.0, the platform has taken a significant leap forward, introducing a host of new features and improvements that elevate its capabilities to new heights.

Key Features and Enhancements

Native Histograms:

Prometheus 3.0 introduces native support for histograms, a data type that efficiently captures distribution information. This is particularly useful for metrics like response times, request durations, and latency.
By leveraging native histograms, Prometheus can provide more accurate and granular insights into the performance of your systems.

OpenTelemetry Support:

Prometheus 3.0 strengthens its integration with OpenTelemetry, a vendor-neutral observability framework. This enables seamless collection and analysis of telemetry data from various sources, including applications, infrastructure, and cloud services.
By adopting OpenTelemetry, you can simplify your monitoring setup and gain a more comprehensive view of your entire technology stack.
Performance Optimizations:
Prometheus 3.0 incorporates several performance optimizations, including:
Faster query execution, especially for complex queries.
Reduced memory usage, leading to improved scalability.
Enhanced data ingestion and storage efficiency.
These optimizations ensure that Prometheus can handle larger workloads and provide real-time insights without compromising performance.

Enhanced PromQL:

PromQL, the powerful query language for Prometheus, has been further refined in 3.0 with the addition of new functions and operators:
delta: Calculates the difference between two time series. changes: Counts the number of changes in a time series.
rate_interval: Calculates the rate of change over a specific interval. These new capabilities empower you to perform more sophisticated analysis and troubleshooting tasks.

Improved Alerting:

The alerting system in Prometheus 3.0 has been enhanced with:

Enhanced filtering: More flexible filtering options for alerts.
Improved grouping: Better grouping of alerts based on specific criteria.
Enhanced notification channels: Support for additional notification channels, including email, SMS, and PagerDuty.

These improvements streamline the alert management process and ensure timely notifications of critical issues.

New Web UI:

Prometheus 3.0 boasts a redesigned web UI with a modern look and feel. The new UI offers:

-Improved navigation and search capabilities.
-More intuitive query building and visualization tools.
-Enhanced performance and responsiveness.

The new UI makes it easier to explore and analyze your monitoring data.

Why Upgrade to Prometheus 3.0?

Upgrading to Prometheus 3.0 is highly recommended for several reasons:

Enhanced Performance: The performance optimizations in 3.0 ensure that Prometheus can handle larger workloads and provide real-time insights.
Improved Scalability: Native histograms and other enhancements make Prometheus more scalable, allowing it to monitor larger and more complex systems.
Increased Functionality: The new features in 3.0, such as OpenTelemetry support and enhanced PromQL, provide greater flexibility and power.
Better User Experience: The redesigned web UI offers a more intuitive and efficient user experience.

Breaking Changes

While most of the changes in Prometheus 3.0 are backward-compatible, there are a few breaking changes that you should be aware of. These include:

Changes to the rate function: The rate function now calculates the rate of change over a specific interval, rather than the instantaneous rate of change.
Changes to the irate function: The irate function now uses a more accurate algorithm to calculate the instantaneous rate of change.
Changes to the increase function: The increase function now returns the total increase over a specific interval, rather than the instantaneous rate of increase.
It's essential to review the migration guide provided by the Prometheus team to ensure a smooth upgrade process.

Conclusion

Prometheus 3.0 is a significant milestone in the evolution of this powerful monitoring solution. The new features and improvements make it even more robust, efficient, and user-friendly. By upgrading to 3.0, you can gain deeper insights into your systems, improve performance, and enhance your overall monitoring strategy.

Effortless File Transfers: A Guide to Using scp and rsync | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 15, 2024 ・
graphpe.Medium

25 Advanced Bash Commands Every Linux User Should Know | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 8, 2024 ・
graphpe.Medium

Top 10 SED Use Cases Every Linux User Should Know | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 5, 2024 ・
graphpe.Medium

Linux Aliases You Didn’t Know You Needed: Practical Shortcuts for Every User | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 15, 2024 ・
graphpe.Medium

🎉 Dive into Python 3.13: New Features, JIT Compiler, Free threading & Installation Guide! 🎉

Oliver Bennet — Tue, 12 Nov 2024 13:00:44 +0000

Introduction:

Hey, Python enthusiasts! The long-awaited Python 3.13 release has arrived, and it brings a host of exciting changes, from a Just-In-Time (JIT) compiler and free-threading support to a more interactive interpreter. In my latest video, I go over all the essential new features and also walk through the installation process for both Windows and Ubuntu.

Check out the video here: Python 3.13 Features & Installation Guide

What’s New in Python 3.13?

Python 3.13 introduces several game-changing updates:

Free-threading Support (PEP 703) – Experimental support to run Python without the Global Interpreter Lock (GIL).
Just-In-Time (JIT) Compiler (PEP 744) – A basic JIT compiler, currently disabled by default, but laying the groundwork for future performance boosts.
Enhanced Error Messages – Python’s interactive interpreter and error handling have leveled up, now with colorized tracebacks for easy debugging.
Library Improvements – Expect a streamlined standard library, with several legacy modules officially removed (PEP 594).
Improved Interactive Interpreter – Inspired by the PyPy project, the REPL now includes multiline editing, history browsing, and colorized prompts

Step-by-Step Installation for Windows & Ubuntu
If you’re eager to try Python 3.13, this video includes a complete installation guide for both Windows and Ubuntu. Whether you're upgrading or installing Python for the first time, follow along to set up the latest version on your system.

Watch the Full Breakdown Medium Post:

Python 3.13 is here and It could be a Game Changer | Medium

Oliver Bennet ・ Nov 12, 2024 ・
graphpe.Medium

Python 3.13 Features & Installation Guide

Key Highlights Covered in the Video

Error Message Improvements – Helpful error messages for common mistakes like naming conflicts with standard modules.
New Security Enhancements – Updates in ssl and other libraries to keep your applications secure.
Typing Improvements – Expanded typing functionality including default values for type parameters.
Deprecations and Removals – Say goodbye to legacy modules like cgi, crypt, telnetlib, and more.

Whether you’re developing on Windows or Ubuntu, this video has all the insights you need to get started with Python 3.13! Check it out and let me know your thoughts in the comments below. Happy coding!

25 Advanced Bash Commands Every Linux User Should Know

Oliver Bennet — Sat, 09 Nov 2024 07:04:07 +0000

For Linux users aiming to maximize productivity, mastering advanced Bash commands is essential. These commands provide powerful ways to search, manipulate, and automate tasks, unlocking a world of efficiency for both system administration and development workflows. Here’s a deep dive into 25 of the most advanced Bash commands, essential for any serious Linux user.

1. grep - Advanced Text Searching

Usage: grep -r "pattern" /directory
Explanation: grep searches for text patterns within files and directories. With the -r (recursive) option, it looks through subdirectories as well. It supports regex patterns, allowing for complex search capabilities.
Example: grep -r "error" /var/log/ searches all logs for "error."

2. awk - Text Processing and Data Extraction

Usage: awk '{print $1, $3}' file.txt
Explanation: awk is a powerful tool for text manipulation, allowing you to filter and reformat data. You can use it to extract columns and apply conditions on text files.
Example*: awk '{print $1, $4, $5}' /var/log/syslog extracts the first, fourth and fifth columns from syslog.

3. sed - Stream Editing for Efficient Text Manipulation

Usage: sed 's/old/new/g' file.txt
Explanation: sed allows you to find and replace text patterns within files. It’s ideal for batch substitutions across large files or groups of files.
Example: sed 's/apache/nginx/g' config.txt replaces "apache" with "nginx" in config.txt.

4. xargs - Command Chaining for Pipelining Tasks

Usage: cat files.txt | xargs rm
Explanation: xargs constructs command lines from standard input. It’s useful in combination with find or grep to perform actions on lists of files.
Example: find . -name "*.log" | xargs rm deletes all .log files in the current directory and its subdirectories.

5. find - Locating Files and Directories

Usage: find /path -type f -name "*.txt"
Explanation: find is invaluable for locating files based on criteria like name, type, size, or modification date.
Example: find /etc -type f -name "*.conf" finds all configuration files in /etc.

6. chmod and chown - Managing Permissions and Ownership

Usage: chmod 755 file, chown user:group file
Explanation: chmod and chown modify permissions and file ownership. Proper permissions are essential for security and functionality.
Example: chmod 644 document.txt sets read and write permissions for the owner and read-only for others.

7. curl and wget - Retrieving Web Content

Usage: curl -O url, wget url
Explanation: Both curland wgetfetch files from the internet, but curl is more versatile, supporting API requests.
Example: curl -O https://example.com/file.zip downloads a file to the current directory.

8. tar - File Archiving and Compression

Usage: tar -czvf archive.tar.gz /var/log/syslog
Explanation: tar is used for creating archives, combining multiple files into one. It’s often used with gzip for compression.
Example: tar -xzvf archive.tar.gz extracts a .tar.gz archive.

9. rsync - Syncing Files Across Directories and Systems

Usage: rsync -avz /source /destination
Explanation: rsync copies and synchronizes files, offering options for compression and partial transfers, making it ideal for backups.
Example: rsync -avz /home/user /backup/ backs up a user’s home directory to /backup.

10. netstat and ss - Network Monitoring

Usage: netstat -tuln, ss -tuln
Explanation: netstat and ss display network connections and listening ports. Use ss for more recent and faster network inspection.
Example: ss -tuln shows active TCP/UDP listening sockets.

Mastering BASH CLI in 3 Hours

11. `df`and `du`- Disk Usage Analysis

Usage: df -h, du -sh /path
Explanation: df provides an overview of disk space usage, while du gives a breakdown by directory.
Example: du -sh /home shows the size of the /home directory.

12. `ps` and `top`- Monitoring Running Processes

Usage: ps aux, top
Explanation: ps lists processes, while top provides real-time monitoring of CPU and memory usage.
Example: top helps identify resource-intensive processes.

13. kill and pkill - Terminating Processes

Usage: kill PID,pkill name
Explanation: kill sends signals to processes. Use kill -9 for forceful termination, and pkill to terminate by name.
Example: pkill firefox stops all Firefox processes.

14. `tail`and `head` - Viewing File Sections

Usage: tail -n 20 file.txt, head -n 20 file.txt
Explanation: taildisplays the last lines, and head shows the first lines of a file. Useful for log analysis.
Example: tail -f /var/log/syslog continuously shows new log entries.

15. `man`- Displaying Command Manuals

Usage: man service
Explanation: man displays manuals for commands, helping you understand usage and options.
Example: man rsync provides details on using rsync.

16. diff - File Comparison

Usage:diff file1 file2
Explanation: diff shows line-by-line differences between files, useful for version control.
Example: diff config.old config.new compares two versions of a configuration file.

17. `history`- Command Recall and Re-execution

Usage: history
Explanation: Lists past commands, allowing for easy re-execution.
Example: !100 re-executes command number 100.

18. alias - Creating Command Shortcuts

Usage: alias ll='ls -al'
Explanation: aliasallows you to create shortcuts for commonly used commands.
Example: alias rmf='rm -f' creates a shortcut for force deletion.

19. `crontab`- Scheduling Jobs

Usage: crontab -e
Explanation: Schedules recurring tasks using cron. Useful for automation.
Example: 0 2 * * * /path/to/script.sh runs a script daily at 2 AM.

20. zip and unzip - File Compression and Extraction

Usage: zip -r archive.zip folder, unzip archive.zip
Explanation: Creates and extracts ZIP archives, offering a quick way to manage compressed files.
Example: zip -r project.zip /project compresses /project into project.zip.

21. `ln` - Creating Links

Usage: ln -s /source /destination
Explanation: ln creates hard and symbolic links, linking files or directories.
Example: ln -s /opt/project /home/user/project_link creates a symbolic link.

22. `mount` and `umount` - Mounting File Systems

Usage: mount /dev/sdb1 /mnt, umount /mnt
Explanation: Mounts and unmounts storage devices, allowing access to external storage.
Example: mount -o loop disk.iso /mnt mounts an ISO file.

23. `echo` and `printf` - Displaying Text

Usage: echo "Hello", printf "User: %s\n" "$USER"
Explanation: Outputs text, useful in scripts for showing variable values or message formatting.
Example: printf "Current date: %(%Y-%m-%d)T\n" -1 shows the current date.

24. uptime - System Uptime and Load

Usage: uptime
Explanation: Shows system uptime and average load, helpful for monitoring performance.
Example: uptime displays the system’s active time since the last reboot.

25. iptables - Configuring Firewall Rules

Usage: iptables -A INPUT -p tcp --dport 22 -j ACCEPT
Explanation: iptables manages firewall settings, controlling access to ports and services.
Example: iptables -L lists current firewall rules.

These 25 commands are indispensable tools for any Linux user, helping manage, monitor, and automate tasks efficiently. Familiarizing yourself with them will deepen your command-line expertise and significantly enhance your productivity on Linux systems.

🔗 Support my Work

▶️ Support by Subscribing my YouTube
▶️ Explore more open-source tutorials on my website
▶️ Follow me on X
☕ Buy me a Coffee

DEV Community: Oliver Bennet

Automate Repetitive Tasks: Creating Bash Scripts for Everyday Use 🚀

Why Learn Bash Scripting?

What You’ll Learn in the Full Guide

Who Should Read This?

Ready to Master Automation?

Automate Repetitive Tasks: Creating Bash Scripts for Everyday Use | by Oliver Bennet | Dec, 2024 | Medium

Oliver Bennet ・ Dec 5, 2024 ・ graphpe.Medium

Bash Basics: A Beginner’s Guide to Shell Scripting

1. What is Bash Scripting?

2. Getting Started with Bash

3. Understanding Bash Basics

4. Practical Applications

5. Best Practices in Bash Scripting

6. Debugging and Troubleshooting

7. Expanding Your Skills

Conclusion

Mastering Bash Variables and Parameters | by Oliver Bennet | Dec, 2024 | Medium

Oliver Bennet ・ Dec 4, 2024 ・ graphpe.Medium

Automating Network Devices with Python and Netmiko: A Comprehensive Guide

Introduction: The Real-World Impact of Network Automation

Understanding Netmiko and Its Benefits

Key Features of Netmiko

Use Cases for Automating Network Devices

1. Bulk Configuration Changes

2. Backup Configurations

3. Compliance Auditing

4. Real-Time Device Monitoring

Advanced and Complex Use Cases

5. Multi-Vendor Network Orchestration

6. Firmware Upgrades

7. Zero-Touch Provisioning

Optimizing Automation with Advanced Techniques

1. Parallel Execution

2. Centralized Logging

Summary

🚀 Top 10 sed Command Use Cases + 5 Real-Life Scenarios!

Top 10 SED Use Cases Every Linux User Should Know | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 20, 2024 ・ graphpe.Medium

🔐 Why Your Team Needs Passbolt for Password Management

Why Passbolt?

Passbolt: A Comprehensive Guide to Secure Password Management for Teams | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 18, 2024 ・ graphpe.Medium

🎯 Master Wireshark: 15 Essential Filters for Cybersecurity Professionals

Wireshark Display Filters: Essential Filters Every Cybersecurity Analyst Should Master | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 19, 2024 ・ graphpe.Medium

Wireshark #Cybersecurity #NetworkTools #OpenSource

Essential Linux Text Manipulation Tools: cut, sort, uniq, tr 🛠️

4 Most Essential Text Manipulation Tools in Linux: cut, sort, uniq, tr | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 20, 2024 ・ graphpe.Medium

Linux #TextProcessing #CLI #OpenSource #DevOps #BashScripting

What's a Quickie Post?

Why Use Quickie Posts?

Linux Kernel 6.12: A Quantum Leap in Performance and Features

Real-time Performance Revolution

Expanded Hardware Ecosystem

Performance Tweaks and Optimizations

Developer Tools and Ecosystem

Other Notable Features

A Deeper Dive into Real-time Performance

Real-world Use Cases and Benefits

The Future of Linux Kernel

Recent Articles

Effortless File Transfers: A Guide to Using scp and rsync | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 15, 2024 ・ graphpe.Medium

25 Advanced Bash Commands Every Linux User Should Know | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 8, 2024 ・ graphpe.Medium

Top 10 SED Use Cases Every Linux User Should Know | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 5, 2024 ・ graphpe.Medium

Linux Aliases You Didn’t Know You Needed: Practical Shortcuts for Every User | by Oliver Bennet | Nov, 2024 | Medium

Oliver Bennet ・ Nov 15, 2024 ・ graphpe.Medium

Recent Videos

Prometheus 3.0: A Quantum Leap in Monitoring

Key Features and Enhancements

Native Histograms:

OpenTelemetry Support:

Enhanced PromQL:

Improved Alerting:

New Web UI:

Why Upgrade to Prometheus 3.0?

Oliver Bennet ・ Dec 5, 2024 ・
graphpe.Medium

Oliver Bennet ・ Dec 4, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 20, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 18, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 19, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 20, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 15, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 8, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 5, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 15, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 15, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 8, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 5, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 15, 2024 ・
graphpe.Medium

Oliver Bennet ・ Nov 12, 2024 ・
graphpe.Medium

11. `df`and `du`- Disk Usage Analysis

12. `ps` and `top`- Monitoring Running Processes

14. `tail`and `head` - Viewing File Sections

15. `man`- Displaying Command Manuals

17. `history`- Command Recall and Re-execution

19. `crontab`- Scheduling Jobs

21. `ln` - Creating Links

22. `mount` and `umount` - Mounting File Systems

23. `echo` and `printf` - Displaying Text