DEV Community: Ethan Vance

How to Detect a DDoS Attack on a Linux Server via CLI

Ethan Vance — Fri, 21 Aug 2026 05:00:35 +0000

When your Linux server load suddenly spikes, guessing the cause is not an option. In these critical moments, you need to know immediately whether you are dealing with a legitimate traffic surge, a misbehaving application, or a DDoS attack.

Here is a hands-on guide to diagnosing malicious traffic directly from your terminal using standard Linux command-line utilities.

1. Monitor Network Interfaces (Volumetric Attacks)

The most common DDoS attack is a volumetric flood. Before digging into logs, check the raw traffic hitting your network interfaces.

Real-Time Bandwidth with iftop:

sudo iftop -n

Note: The -n flag prevents DNS resolution, which is crucial during an attack because DNS lookups will slow down the tool.

Check Packets Per Second (PPS) with sar:

Bash
sar -n DEV 1

If your inbound traffic (RX) or PPS is pinned to its absolute limit while CPU usage remains normal, it strongly indicates a Layer 3 network-level flood.

2. Analyze Active TCP States (Protocol Attacks)

If legitimate users are failing to connect, the attacker is likely targeting your server's connection-handling capacity (Layer 4).

Count Total Connections by IP Address:

Bash
ss -H -tn | awk '{print $5}' \vert{} sed 's/:[^:]*$//' | sort | uniq -c | sort -nr | head -10

Detect a SYN Flood Attack:

A SYN flood repeatedly sends initial connection requests (SYN) but never completes the handshake. To count connections stuck in the SYN_RECV state:

Bash
ss -H -n -t state syn-recv | wc -l

Pro-Tip: For a rapid summary of your current TCP states without locking up your terminal, simply type ss -s.

3. Inspect Web Server Logs (Layer 7 HTTP Floods)

If your network bandwidth is fine but your server's CPU or memory is maxed out, you might be facing an HTTP flood.

Identify Top Attacking IPs via Web Logs (Nginx):

Bash
tail -n 10000 /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -nr | head -10

Find the Most Hammered URLs:

Bash
tail -n 10000 /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -10

4. Defense and Upstream Mitigation

Once the observed traffic patterns are consistent with a DDoS attack, you can begin mitigation.

Block specific IPs: sudo iptables -A INPUT -s ATTACKER_IP -j DROP
Enable TCP SYN Cookies: sudo sysctl -w net.ipv4.tcp_syncookies=1

Understand the Limits of Local Server Defense:

Local firewalls are useful for targeted attacks, but they cannot stop a massive volumetric flood. If an attacker sends 50Gbps of traffic to your 1Gbps interface, dropping packets locally at the OS level still means your pipe is clogged.

To survive large-scale volumetric or complex multi-vector DDoS attacks, malicious traffic must be filtered before it ever reaches your server through upstream traffic scrubbing or high-capacity network-level mitigation.

Why Upgrading Your Server's Hardware Won't Stop a DDoS Attack

Ethan Vance — Thu, 20 Aug 2026 04:44:50 +0000

When a web application starts slowing down or dropping connections under heavy load, the instinct for many developers and sysadmins is to scale up: add a faster CPU, double the RAM, or move to NVMe storage.

But what happens when the traffic isn't a viral product launch, but a coordinated Distributed Denial-of-Service (DDoS) attack?

The harsh reality is that a dedicated server can have top-tier hardware and a 20Gbps network port, yet still become entirely unreachable. Let’s dive into the technical mechanics of why this happens and where the actual bottlenecks occur.

The Traffic Path: Where Do Things Break?

When an attack is launched, the malicious data doesn't instantly appear on your server's processor or memory. The traffic follows a specific path:

Internet → Upstream Network → Mitigation Infrastructure → Server Network Interface → OS → Application

If incoming traffic exceeds the available capacity at any point in this path, the link becomes saturated. Legitimate traffic simply cannot reach your server.

How DDoS Attacks Choke Network Performance

Attacks generally fall into three categories (Volumetric, Protocol, and Application-layer), and they impact your network in the following ways:

1. Bandwidth Saturation (The Clogged Pipe)

Every network connection has a finite limit. In a volumetric attack (like a UDP flood), the attacker's goal is to consume all available bandwidth between the target and the wider internet. Even if you have a 10Gbps or 20Gbps port, a massive attack can fill that "pipe" completely. Your CPU might be sitting at 5% utilization, but your users still get a 502 Bad Gateway or connection timeout because their requests can't physically reach the server.

2. Connection State Exhaustion

Not all attacks rely on pure data volume. Protocol attacks (like SYN floods) target the connection-handling capacity of network infrastructure like firewalls or load balancers. By initiating massive numbers of incomplete TCP connection attempts, attackers can consume all available connection state tables. Once exhausted, the network device simply drops any new connections from legitimate users.

3. Packet Loss and Latency

Excessive traffic creates network congestion. When routers and switches are overwhelmed, packets spend more time waiting in queues (increasing latency). When the buffers are full, packets are dropped entirely (packet loss). This forces retransmissions, creating even more traffic and making interactive applications (like WebSockets or game servers) unusable.

The Host Firewall Misconception

A common misconception among developers is that iptables or a standard host-based firewall is enough to stop a network flood.

While a firewall is essential for controlling access (e.g., blocking unused ports), it only filters traffic after it has reached your server's network interface. If a volumetric attack has already saturated your upstream bandwidth, your local firewall dropping the packets won't magically free up the network pipe.

The Solution: Upstream Mitigation

To truly protect a server, you can't rely on the server to defend itself. The most effective protection happens upstream.

Upstream DDoS mitigation analyzes incoming data in real-time, passing it through traffic scrubbing centers. The core concept is straightforward:

Incoming traffic → Scrubbing Center → Malicious traffic dropped → Clean traffic forwarded

This preserves your network capacity and ensures your server's hardware is only processing legitimate application logic, not fighting off junk packets.

Configuring NVIDIA MIG: Technical Walkthrough for Bare-Metal GPU Partitioning published: true

Ethan Vance — Tue, 11 Aug 2026 11:36:06 +0000

High-capacity GPUs (like the A100, H100, or Blackwell) often sit underutilized in bare-metal environments. Software-based time slicing lacks hardware-level resource guarantees.

NVIDIA Multi-Instance GPU (MIG) solves this by partitioning a single physical card into independent instances at the hardware level, giving each partition dedicated compute, memory, and bandwidth.

Here is a quick step-by-step CLI walkthrough on Ubuntu 24.04:

1. Install Drivers and Verify Support

sudo ubuntu-drivers install
nvidia-smi -q

Look for the MIG Mode section in the output to confirm hardware support. (Install nvidia-container-toolkit if you plan to run Docker).

2. Enable MIG Mode

Ensure no active workloads are attached, then enable MIG mode on GPU 0:

sudo nvidia-smi -i 0 -mig 1

3. List Supported Profiles

Query the supported resource layouts for your GPU model:

nvidia-smi mig -lgip

4. Partition the GPU

Create GPU Instances (GI) and Compute Instances (CI) using your chosen profile string:

sudo nvidia-smi mig -cgi 2g.48gb,2g.48gb -C

Verify active partitions using nvidia-smi mig -lgi.

5. Target MIG Instances in Docker

List generated UUIDs:

nvidia-smi -L

Pass the specific MIG UUID directly to your Docker container:

docker run --rm -it --gpus '"device=MIG-af414487-fcaa-5f42-b210-6f614c9cf780"' nvcr.io/nvidia/pytorch: nvidia-smi

6. Reset Partitions

To revert to single-instance operation:

sudo nvidia-smi mig -dci
sudo nvidia-smi mig -dgi
sudo nvidia-smi -i 0 -mig 0

Managing bare-metal lifecycle, drivers, and hardware provisioning at scale takes operational bandwidth. At MIG servers, we provide pre-configured, dedicated GPU hardware optimized for partitioned AI workloads out of the box.

Deep Dive into NVIDIA Blackwell Architecture: Redefining GenAI Infrastructure

Ethan Vance — Thu, 06 Aug 2026 07:30:18 +0000

When we talk about the evolution of modern GPU servers, the NVIDIA Blackwell architecture represents a monumental leap forward. Purpose-built to handle the most demanding AI and cloud computing workloads, Blackwell is strictly an enterprise-grade system.

Unlike consumer gaming GPUs, Blackwell is completely optimized for processing massive datasets, complex neural networks, and generative AI systems. It directly succeeds the highly successful NVIDIA Hopper architecture, bringing a massive leap in compute performance, memory bandwidth, and multi-node scalability to the data center.

Let's dive into the hardware and see what makes this silicon so groundbreaking. 👇

The Hardware: An Entirely New Class of AI Superchip 🧠

To achieve unprecedented computing density, NVIDIA engineering broke through traditional manufacturing limits:

Unmatched Scale: These GPUs pack an astounding 208 billion transistors, providing the raw compute density needed for trillion-parameter models.
Custom Fabrication: The architecture is manufactured utilizing a custom-built TSMC 4NP process, balancing extreme performance with energy efficiency.
Unified Architecture: To overcome physical die limits, all Blackwell products feature two reticle-limited dies. Instead of acting as separate processors, they are seamlessly linked by a 10 terabytes per second (TB/s) chip-to-chip interconnect. This allows the dual-die setup to function flawlessly as a single, unified GPU.

Inside the Technological Breakthroughs ⚡

Blackwell is not just a faster chip; it is a fundamental redesign of how computing resources interact.

1. Second-Generation Transformer Engine
Training and running Large Language Models (LLMs) requires staggering amounts of computational power. Blackwell introduces its second-generation Transformer Engine, which pairs custom Tensor Cores with software like NVIDIA TensorRT™-LLM. What truly sets it apart is micro-tensor scaling, enabling FP4 (4-bit floating point) AI precision. This effectively doubles the performance and memory capacity for next-generation models while maintaining high accuracy.

2. 5th-Generation NVLink & NVLink Switch
Even the fastest GPUs will bottleneck if the network connecting them is slow. The 5th-generation NVLink interconnect solves this by scaling up to 576 GPUs. Within a single 72-GPU NVLink domain (NVL72), the NVLink Switch Chip enables a massive 130TB/s of GPU bandwidth.

3. Secure AI with Confidential Computing
Security is paramount for enterprise data. Blackwell is the industry’s first TEE-I/O capable GPU. NVIDIA Confidential Computing protects sensitive data and models from unauthorized access without any performance degradation.

4. Decompression Engine & RAS
The architecture features a dedicated Decompression Engine that accelerates the full pipeline of database queries. Additionally, intelligent resiliency is handled via a dedicated RAS (Reliability, Availability, and Serviceability) Engine, which uses AI-powered predictive management to minimize downtime.

Blackwell vs. Hopper: What’s the Real Difference? 📊

The Hopper architecture (H100) is an incredibly powerful foundation for today's workloads. However, Blackwell (B200) is purpose-built for the massive scale of tomorrow.

Feature	NVIDIA Hopper (H100)	NVIDIA Blackwell (B200)
Primary Focus	Mixed AI & Traditional HPC	Massive LLMs & Generative AI
Transformer Precision	1st Gen Tensor Cores with FP8 precision	2nd Gen Tensor Cores with FP4 micro-tensor scaling
Interconnect Technology	4th Gen NVLink	5th Gen NVLink (Scales up to 576 GPUs)
Domain Bandwidth	Scalable for standard GPU clusters	Up to 130 TB/s within a 72-GPU NVL72 domain
Confidential Computing	Standard hardware security	First TEE-I/O capable GPU with no performance overhead

The Infrastructure Reality Check 🏗️

While the performance gains are undeniable, deploying Blackwell in-house introduces severe infrastructure challenges. These are not plug-and-play GPUs:

⚠️ Extreme Power Draw: A single Blackwell GPU can consume up to ~1,000 watts, straining standard data center electrical limits.

💧 Mandatory Liquid Cooling: Traditional air-cooling systems are incapable of dissipating the heat. Liquid cooling infrastructure is now a strict requirement.

🏢 Incompatible with Standard Racks: You cannot slot a Blackwell GPU into a legacy server chassis. These require purpose-built AI systems like NVIDIA HGX or DGX platforms.

Key Developer Use Cases 💻

By matching hardware innovations to modern software demands, Blackwell unlocks new capabilities:

LLM Training & Fine-Tuning: Accelerate time-to-market for proprietary models using FP4 precision.

Large-Scale Inference: Handle high token throughput efficiently for real-time AI chatbots, driving down the compute cost-per-token.

Big Data Analytics & HPC: Rapidly process massive datasets and blend computing simulations with machine learning seamlessly.

Physical AI & Robotics: Train complex vision models and run high-fidelity Digital Twin simulations for autonomous logic.

Conclusion 🏁

The NVIDIA Blackwell architecture has definitively set the new standard for accelerated computing. However, as developers and engineers, we must also prepare for the massive physical constraints—navigating 1000W power limits and liquid cooling will be just as crucial as writing the algorithms themselves.

The future of AI infrastructure is incredibly exciting, but it demands a complete rethink of data center physics.

How to Build a High-Availability (HA) Cluster on Bare Metal

Ethan Vance — Thu, 23 Jul 2026 08:49:54 +0000

When deploying mission-critical applications, a Single Point of Failure (SPOF) is a disaster waiting to happen. High Availability (HA) is essential for achieving uptime targets like 99.99%.

In this tutorial, we outline a production-grade, 7-node High-Availability cluster built on bare-metal servers using a Private VLAN for maximum performance, security, and hardware control.

🏗️ 3-Tier Architecture Overview

Traffic flows through three distinct, isolated tiers:

Floating VIP (203.0.113.100): Single public IP entry point.
Tier 1 (Load Balancers - LB-01 & LB-02): Active/Passive HAProxy + Keepalived setup for automated 1–3 second VIP failover.
Tier 2 (Web Tier - WEB-01 & WEB-02): Nginx nodes isolated from public access.
Tier 3 (Database Tier - DB-01, DB-02, DB-03): MariaDB Galera Cluster with synchronous, certification-based replication.

📋 Server Topology & IP Scheme

+----------+-----------------+---------------+--------------------+
| Hostname | Role            | Public IP     | Private IP (VLAN)  |
+----------+-----------------+---------------+--------------------+
| VIP      | Floating IP     | 203.0.113.100 | -                  |
| LB-01    | Load Balancer 1 | 203.0.113.101 | 10.0.0.10          |
| LB-02    | Load Balancer 2 | 203.0.113.102 | 10.0.0.11          |
| WEB-01   | Web Node 1      | -             | 10.0.0.20          |
| WEB-02   | Web Node 2      | -             | 10.0.0.21          |
| DB-01    | DB Node 1       | -             | 10.0.0.30          |
| DB-02    | DB Node 2       | -             | 10.0.0.31          |
| DB-03    | DB Node 3       | -             | 10.0.0.32          |
+----------+-----------------+---------------+--------------------+

🔒 Security & Sysctl Tuning

Allow incoming traffic strictly from your Private VLAN (10.0.0.0/24) using ufw:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh

# Allow internal Web Traffic
sudo ufw allow from 10.0.0.0/24 to any port 80
sudo ufw allow from 10.0.0.0/24 to any port 443

# Allow internal Galera & MySQL Traffic
sudo ufw allow from 10.0.0.0/24 to any port 3306  # MySQL
sudo ufw allow from 10.0.0.0/24 to any port 4444  # Galera SST
sudo ufw allow from 10.0.0.0/24 to any port 4567  # Galera Cluster
sudo ufw allow from 10.0.0.0/24 to any port 4568  # Galera IST

sudo ufw enable

Apply kernel tweaks on all nodes to enable non-local binding and scale network queues:

1. MariaDB Galera Configuration

Edit the Galera configuration file:

/etc/mysql/mariadb.conf.d/60-galera.cnf

[galera]
bind-address = 0.0.0.0

binlog_format = row
default_storage_engine = InnoDB
innodb_autoinc_lock_mode = 2

wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_name = ha_production_cluster
wsrep_cluster_address = "gcomm://10.0.0.30,10.0.0.31,10.0.0.32"

wsrep_sst_method = mariabackup

Kernel Networking (Sysctl)

Create the sysctl configuration file:

/etc/sysctl.d/99-ha-cluster.conf

net.core.somaxconn = 65535
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.tcp_max_syn_backlog = 65535

Apply the changes:

sudo sysctl --system

💡 Note

Bootstrap the Galera cluster only on DB-01:
sudo galera_new_cluster
After the cluster is initialized, start MariaDB normally on DB-02 and DB-03:
sudo systemctl start mariadb

2. Keepalived Unicast VRRP

Modern bare-metal servers and many cloud providers block multicast traffic. To ensure reliable VIP failover, configure Unicast VRRP.

Edit the Keepalived configuration:

/etc/keepalived/keepalived.conf

LB-01 (MASTER)

vrrp_instance VI_1 {
    state MASTER
    interface ens18
    virtual_router_id 51
    priority 100

    unicast_src_ip 10.0.0.10

    unicast_peer {
        10.0.0.11
    }

    virtual_ipaddress {
        203.0.113.100/32
    }
}

LB-02 (BACKUP)

vrrp_instance VI_1 {
    state BACKUP
    interface ens18
    virtual_router_id 51
    priority 90

    unicast_src_ip 10.0.0.11

    unicast_peer {
        10.0.0.10
    }

    virtual_ipaddress {
        203.0.113.100/32
    }
}

🔗 Read the full step-by-step implementation guide with all HAProxy config files and failover testing steps:

Read Full Step-by-Step Tutorial Here

NVMe vs SATA SSD: Why Maxing Out Your CPU Won't Fix Your Database Stutters

Ethan Vance — Thu, 23 Jul 2026 06:11:32 +0000

Ever provisioned a brand new dedicated server, maxed out the RAM, and upgraded to the latest multi-core CPUs—only to watch your database queries stutter during peak traffic?

You check htop. Your CPU usage is low. Your RAM has plenty of headroom. So, what’s going wrong?

The answer is almost always the most overlooked component of server architecture: The Storage Bottleneck (High I/O Wait).

Today, we are breaking down the exact architectural differences between legacy SATA SSDs and NVMe, the PCIe advantage, and why throwing more compute power at a storage problem will never work. Let's dive in! 🚀

🏗️ The Architecture: AHCI vs. PCIe Bus

To understand the bottleneck, we have to look at the data pathway.

SATA and the AHCI Bottleneck
SATA SSDs use the AHCI protocol, built in the early 2000s for mechanical spinning hard drives. Because HDDs are slow, AHCI was designed with a single command queue that holds a maximum of 32 commands.

Even if the flash memory inside your SATA SSD is fast, it is forced through this legacy controller. When your PostgreSQL or MySQL database fires thousands of simultaneous read/write requests, that 32-command queue instantly fills up. Your CPU is forced to wait.

NVMe and the PCIe Advantage
NVMe (Non-Volatile Memory Express) was built from the ground up for flash storage. Instead of a legacy controller, NVMe connects directly to the motherboard’s PCI Express (PCIe) bus.

The parallel processing difference is insane:

SATA (AHCI): 1 queue, 32 commands per queue.
NVMe: Up to 64,000 queues, with 64,000 commands per queue! 🤯

📊 The Ultimate Performance Showdown

Here is how that architectural difference translates into real-world benchmarks:

--Throughput (Seq)--
Standard SATA SSD - ~550 MB/s
Enterprise NVMe (PCIe Gen4) - 7,000+ MB/s
The Gap - ~12x Faster

--Concurrency (IOPS)--
Standard SATA SSD - ~80,000 IOPS
Enterprise NVMe (PCIe Gen4) - 500,000+ IOPS
The Gap - 6x Higher

--Latency--
Standard SATA SSD - ~100 µs
Enterprise NVMe (PCIe Gen4) - Sub-20 µs
The Gap - ~5x Lower Wait

Why IOPS matters: If a database hits the 80k IOPS ceiling of a SATA drive, new queries queue up. This spikes your CPU I/O wait. NVMe’s 500k+ IOPS ensures instant execution, even during traffic spikes.

🛠️ The DevOps / SRE Perspective: Skip Hardware RAID

If you are deploying NVMe drives, here is a golden rule: Do NOT use a legacy hardware RAID controller.

Most enterprise hardware RAID controllers connect via a PCIe x8 slot, which physically caps bandwidth to around 14,000 MB/s. If you have four PCIe Gen4 NVMe drives capable of 28,000 MB/s combined, the RAID card will instantly bottleneck your throughput by 50%.

The Solution: Stick to Advanced Software RAID (mdadm), Intel VROC, or ZFS mirroring. Software RAID allows drives to connect directly to the PCIe lanes while using a microscopic fraction of modern CPU power to calculate parity.

🤔 When is an NVMe Upgrade Actually Mandatory?

SATA is still fine for basic static web hosting, cold backups, and fully in-memory databases (where data fits entirely in RAM).

However, you must upgrade to an NVMe architecture if you run:

Heavy Relational Databases (MySQL/PostgreSQL): Where Write-Ahead Logging (WAL) and synchronous commits easily overwhelm SATA.
High-Traffic E-commerce: Where uncacheable, dynamic queries dictate page load speeds (and revenue).
Proxmox/Virtualization Clusters: To combat the "I/O blender effect" of multiple VMs sharing the same storage.

Wrapping Up
Scaling a modern application requires an end-to-end data path that matches your compute power. If your infrastructure handles intensive read/write workloads, sticking to SATA will continuously force your system into I/O wait.

If your workloads demand zero-bottleneck architecture, you can check out our highly customizable NVMe dedicated server deployments, engineered specifically with enterprise PCIe Gen4 drives and ZFS/Software RAID configurations.

How to Add a Linux Target Node to Prometheus (Step-by-Step)

Ethan Vance — Thu, 11 Jun 2026 06:48:26 +0000

Hey everyone! 👋

Monitoring your infrastructure is super important for maintaining system health. If you already have a Prometheus server running, the next logical step is adding your servers to it.

In this quick guide, we will look at how to add a new Linux server (Target Node) to your existing Prometheus monitoring system using Node Exporter. Node Exporter collects system metrics such as CPU, memory, and disk usage, which Prometheus then scrapes.

📌 Prerequisites

A running Prometheus Server.
A new Target Node (the Linux server you want to monitor).
Root or sudo privileges on both servers.

Step 1: Install Node Exporter on the Target Node

To monitor the new Target Node, Node Exporter must be installed and running. On most RHEL-based distributions (like AlmaLinux, Rocky Linux, or CentOS), you can install it using:

dnf install -y prometheus-node-exporter

Note: If the package is unavailable, you may need to enable the EPEL or CRB repository first.

Step 2: Verify Node Exporter is Running

To monitor the new Target Node, Node Exporter must be installed and running. On most RHEL-based distributions (like AlmaLinux, Rocky Linux, or CentOS), you can install it using:

On your Target Node, run:

curl http://localhost:9100/metrics

If it's working correctly, you will immediately see a long list of system metrics printed on your terminal.

🚀 Want to complete the setup?
We have successfully installed Node Exporter, but to finish the setup we still need to:

Open Port 9100 in your Firewall
Configure the Main Prometheus Server (prometheus.yml)
Verify the Target in the Web UI
Test metrics with PromQL Queries

View Full Tutorial:

How to Add Linux Target Nodes to Prometheus Monitoring

👇 Let me know in the comments if you face any issues while setting this up!

The SysAdmin Way to Install Prometheus & Node Exporter on CentOS 9 (Without Messy RPMs)

Ethan Vance — Thu, 11 Jun 2026 05:11:24 +0000

If you manage enterprise hardware or dedicated servers, you already know that real-time visibility into your infrastructure is non-negotiable.

Prometheus paired with Node Exporter is the industry standard for this. But here is the problem I keep seeing: too many people rely on outdated, third-party RPMs to install them.

When you use unofficial repositories, you run into version mismatches, missing features, and potential security risks.

While building out the infrastructure at MIG servers, I realized we needed a cleaner, SysAdmin-approved approach. The best way? Using the official upstream binaries. It’s easier to audit, highly secure, and simple to keep updated.

I just published a complete, step-by-step guide on exactly how to set this up on CentOS Stream 9.

What the full guide covers:

✅ Creating dedicated service users for security (never run as root!)
✅ Verifying SHA256 checksums of the official binaries
✅ Writing custom systemd unit files for automated background processing
✅ Configuring scrape_configs to monitor local and remote targets
✅ Essential Firewall and security warnings (Don't expose port 9090!)

If you want to set up your monitoring stack the right way, I’ve documented every single command and configuration file you need.

📖 Read my full step-by-step tutorial here:
👉 How to Install Prometheus and Node Exporter on CentOS Stream 9

Let me know in the comments: What does your current monitoring stack look like? Are you team Prometheus or do you prefer something else?

How to Install Prometheus and Node Exporter on CentOS Stream 9 (The Upstream Way)

Ethan Vance — Fri, 10 Apr 2026 09:55:48 +0000

If you are managing Linux infrastructure, having real-time visibility into your servers is non-negotiable. Prometheus is the industry-standard, open-source monitoring and alerting toolkit. Paired with Node Exporter, it becomes a powerhouse for collecting host metrics like CPU usage, memory consumption, load averages, and network statistics.

In this guide, we'll look at the SysAdmin-approved way to install Prometheus and Node Exporter on CentOS Stream 9.

Instead of relying on outdated third-party RPMs, we will use the official upstream binaries. This approach is cleaner, easy to audit, and simple to keep updated.

Why the Upstream Approach?

Relying on old third-party repositories can introduce version mismatches, missing features, and security issues. By downloading directly from the official Prometheus releases and verifying the SHA256 checksums, you guarantee your binaries are authentic and uncorrupted.

Key Best Practices for Deployment

If you are setting this up in a production environment, here are the critical steps you need to follow:

1. Create Dedicated Service Users
For security purposes, services should never run as root. Create dedicated system users (prometheus and node_exporter) with no login shell to isolate the services.

2. Verify Official Binaries
Always download the sha256sums.txt alongside your tarballs and verify them using sha256sum -c. Only proceed if the output says OK.

3. Configure Systemd Services
Create custom systemd unit files for both Prometheus and Node Exporter. This ensures they run reliably in the background, start automatically on boot, and manage data retention properly (e.g., by setting the --storage.tsdb.retention.time=15d flag).

4. Lock Down the Firewall
CRITICAL SECURITY WARNING: Exposing port 9090 (Prometheus UI) or 9100 (Node Exporter) directly to the public internet is highly discouraged.

Bind Node Exporter to 127.0.0.1 for local single-server setups.
For remote scraping, use strict firewalld source IP restrictions, VPNs (like WireGuard/Tailscale), or Reverse Proxies.

The Complete Step-by-Step Guide

We have documented the entire process from start to finish. If you want the complete, copy-paste friendly commands, we have put together the full SysAdmin guide on our blog.

It includes:

The exact Bash commands to download, verify, and extract the binaries.
The prometheus.yml scrape configurations.
The complete systemd unit files for both services.
Initial PromQL queries to test your new monitoring stack.

📖 How to Install Prometheus and Node Exporter on CentOS Stream 9

3 Common Mistakes to Avoid

Skipping config validation: Always run promtool check config /etc/prometheus/prometheus.yml before restarting systemd. A simple YAML indentation typo will prevent Prometheus from starting.
Assuming up == 1 means perfect health: This only confirms that Prometheus can reach and scrape the target. It does not guarantee that all expected metrics are actually present.
Forgetting time synchronization: If your Prometheus server and the monitored nodes are out of sync, your graphs, rate calculations, and alerts will be inaccurate.

Happy monitoring! Let me know in the comments if you have any questions about configuring your scrape jobs or writing PromQL queries.

What is CUDA? Understanding the Technology Behind AI and GPU Computing

Ethan Vance — Fri, 06 Mar 2026 05:01:03 +0000

If you're building infrastructure for Artificial Intelligence (AI), Machine Learning (ML), or High-Performance Computing (HPC), powerful hardware alone isn't enough. The real performance advantage comes from the software layer that drives the GPU. In NVIDIA's ecosystem, that layer is CUDA.

In this article, we'll break down what CUDA actually is, how its architecture works, and why it has become the industry standard for accelerating compute-intensive workloads.

What Exactly is CUDA?

Many developers assume CUDA is a programming language or even an operating system. That is not accurate.

CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by NVIDIA. It allows developers to use the massive parallel processing power of GPUs for general-purpose computing.

Instead of relying only on CPUs for heavy computations, CUDA enables workloads like deep learning, scientific simulations, and matrix operations to run thousands of operations simultaneously on GPU cores.

Simple analogy

GPU → Raw compute engine
CUDA → Software layer that unlocks GPU parallelism

CUDA provides:

APIs
Compilers
Development tools
Optimized libraries

These tools allow developers to utilize GPU acceleration without writing low-level assembly code.

CPU vs GPU Architecture

Understanding CUDA requires understanding the fundamental difference between CPUs and GPUs.

Feature	CPU	GPU
Core Count	Dozens of powerful cores	Thousands of smaller cores
Execution Model	Sequential tasks	Massively parallel execution
Transistor Focus	Cache and control logic	Data processing throughput
Best Use Case	Complex control logic	Matrix operations and AI workloads

GPUs are specifically designed for data-parallel workloads, which is why they are ideal for deep learning and scientific computing.

The CUDA Software Stack

CUDA is not a single tool. It is a full ecosystem for GPU development.

nvcc – CUDA Compiler

The NVIDIA CUDA Compiler Driver (nvcc) separates:

Host code (runs on the CPU)

Device code (runs on the GPU)

This allows developers to write heterogeneous programs where CPU and GPU work together.

CUDA APIs

CUDA provides two major APIs:

CUDA Runtime API

High-level interface used in most CUDA applications.

CUDA Driver API

Low-level interface for more granular control of GPU execution.

CUDA Libraries

CUDA also provides highly optimized libraries used across AI and HPC applications.

cuBLAS

Optimized linear algebra operations for GPUs.

cuDNN

Deep neural network primitives such as convolution, pooling, softmax, and attention.

These libraries power frameworks like:

PyTorch
TensorFlow
JAX

CUDA Programming Model

CUDA assumes a heterogeneous system consisting of:

Host

CPU
Host memory

Device

GPU
Device memory

Execution typically follows this workflow.

Data Transfer

Data is copied from host memory (CPU) to device memory (GPU).

Kernel Execution

A CUDA function called a Kernel is executed on the GPU.

Execution hierarchy:

Threads
Blocks
Grids

Threads are the smallest execution units, while blocks allow threads to cooperate using shared memory.

Result Retrieval

Once the computation is complete, results are copied back from GPU memory to CPU memory.

Performance depends heavily on memory access patterns. Efficient CUDA programs maximize the use of:

Registers
Shared memory

while minimizing slower global memory access.

Why CUDA Dominates AI Infrastructure

NVIDIA’s leadership in AI infrastructure is largely due to the CUDA ecosystem.

Reasons include:

Mature development platform
Highly optimized performance libraries
Deep integration with AI frameworks
Strong developer ecosystem

Major frameworks like PyTorch and TensorFlow rely heavily on CUDA for GPU acceleration.

Because CUDA applications are built specifically for NVIDIA GPUs, it has also created a strong ecosystem around NVIDIA hardware.

Final Thoughts

CUDA has become a foundational technology for modern GPU computing. By enabling developers to harness massive parallelism inside GPUs, CUDA allows AI systems, machine learning models, and scientific computing workloads to run dramatically faster.

For developers working with AI, HPC, or GPU-accelerated computing, understanding CUDA is essential.

Original article:

Understanding NVIDIA CUDA: The Core of GPU Parallel Computing

Finally found a way to rent H100s without selling a kidney (MIG Tech)

Ethan Vance — Tue, 20 Jan 2026 12:17:28 +0000

Is it just me, or is trying to rent a dedicated H100 or A100 right now an absolute nightmare?

I've been working on some LLM fine-tuning recently, and I kept running into the same problem: Overkill.

I needed the architecture of the H100 (for the transformer engine), but I didn't need the entire card 24/7. Paying $4/hr+ for a GPU that sits idle 80% of the time just burns through the budget.

The "Aha" Moment: Splitting the Hardware

I did some digging and realized I should be looking for MIG (Multi-Instance GPU) capable servers.

If you aren't familiar with it, MIG basically lets you slice a physical GPU (like an A100 or H100) into up to 7 completely isolated instances. It’s not just software partitioning; it’s hardware-level isolation. So you get your own dedicated memory and cache.

The Resource: MIG servers

I came across a provider called MIG servers that specializes exactly in this. I wanted to share it here because their inventory is actually pretty impressive compared to the "Sold Out" signs I see everywhere else.

They seem to have bare metal stock in:

USA: Dallas, LA, Chicago
Europe: Luxembourg, London, Amsterdam
Asia: Incheon, Tokyo

What stood out to me was the flexibility. You can grab a massive 8x H100 cluster if you are training, or just slice up an A100 if you are doing inference.

Why it matters

If you are a DevOps engineer or working in AI, you know that "Time-Slicing" is usually laggy and insecure. MIG solves that.

I wrote a deeper breakdown on my personal blog about the technical specs and pricing comparisons, but I just wanted to drop this here for anyone struggling to find hardware.

To give you an idea of what MIG-ready hardware looks like, here are the specs we typically deploy for these workloads at MIG Servers:

Location	CPU	GPU Configuration	Max MIG Instances
Luxembourg	2x Xeon Platinum 8480+	8x NVIDIA H100 (200Gbps)	56 Instances
Dallas, USA	2x EPYC 9354	8x NVIDIA H100 NVLink	56 Instances
London, UK	2x Xeon Gold 6210U	NVIDIA A30	4 Instances

👉 Check out full breakdown and the server list here

Let me know if you guys have tried partitioning H100s yet!

[Boost]

Ethan Vance — Mon, 17 Nov 2025 06:40:34 +0000

Architecture for APAC: The Engineering Case for Singapore Bare Metal Infrastructure

Ethan Vance ・ Nov 17

#linux #webdev #dedicatedservers