DEV Community: Ethan Vance

How to Install Prometheus and Node Exporter on CentOS Stream 9 (The Upstream Way)

Ethan Vance — Fri, 10 Apr 2026 09:55:48 +0000

If you are managing Linux infrastructure, having real-time visibility into your servers is non-negotiable. Prometheus is the industry-standard, open-source monitoring and alerting toolkit. Paired with Node Exporter, it becomes a powerhouse for collecting host metrics like CPU usage, memory consumption, load averages, and network statistics.

In this guide, we'll look at the SysAdmin-approved way to install Prometheus and Node Exporter on CentOS Stream 9.

Instead of relying on outdated third-party RPMs, we will use the official upstream binaries. This approach is cleaner, easy to audit, and simple to keep updated.

Why the Upstream Approach?

Relying on old third-party repositories can introduce version mismatches, missing features, and security issues. By downloading directly from the official Prometheus releases and verifying the SHA256 checksums, you guarantee your binaries are authentic and uncorrupted.

Key Best Practices for Deployment

If you are setting this up in a production environment, here are the critical steps you need to follow:

1. Create Dedicated Service Users
For security purposes, services should never run as root. Create dedicated system users (prometheus and node_exporter) with no login shell to isolate the services.

2. Verify Official Binaries
Always download the sha256sums.txt alongside your tarballs and verify them using sha256sum -c. Only proceed if the output says OK.

3. Configure Systemd Services
Create custom systemd unit files for both Prometheus and Node Exporter. This ensures they run reliably in the background, start automatically on boot, and manage data retention properly (e.g., by setting the --storage.tsdb.retention.time=15d flag).

4. Lock Down the Firewall
CRITICAL SECURITY WARNING: Exposing port 9090 (Prometheus UI) or 9100 (Node Exporter) directly to the public internet is highly discouraged.

Bind Node Exporter to 127.0.0.1 for local single-server setups.
For remote scraping, use strict firewalld source IP restrictions, VPNs (like WireGuard/Tailscale), or Reverse Proxies.

The Complete Step-by-Step Guide

We have documented the entire process from start to finish. If you want the complete, copy-paste friendly commands, we have put together the full SysAdmin guide on our blog.

It includes:

The exact Bash commands to download, verify, and extract the binaries.
The prometheus.yml scrape configurations.
The complete systemd unit files for both services.
Initial PromQL queries to test your new monitoring stack.

📖 How to Install Prometheus and Node Exporter on CentOS Stream 9

3 Common Mistakes to Avoid

Skipping config validation: Always run promtool check config /etc/prometheus/prometheus.yml before restarting systemd. A simple YAML indentation typo will prevent Prometheus from starting.
Assuming up == 1 means perfect health: This only confirms that Prometheus can reach and scrape the target. It does not guarantee that all expected metrics are actually present.
Forgetting time synchronization: If your Prometheus server and the monitored nodes are out of sync, your graphs, rate calculations, and alerts will be inaccurate.

Happy monitoring! Let me know in the comments if you have any questions about configuring your scrape jobs or writing PromQL queries.

What is CUDA? Understanding the Technology Behind AI and GPU Computing

Ethan Vance — Fri, 06 Mar 2026 05:01:03 +0000

If you're building infrastructure for Artificial Intelligence (AI), Machine Learning (ML), or High-Performance Computing (HPC), powerful hardware alone isn't enough. The real performance advantage comes from the software layer that drives the GPU. In NVIDIA's ecosystem, that layer is CUDA.

In this article, we'll break down what CUDA actually is, how its architecture works, and why it has become the industry standard for accelerating compute-intensive workloads.

What Exactly is CUDA?

Many developers assume CUDA is a programming language or even an operating system. That is not accurate.

CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by NVIDIA. It allows developers to use the massive parallel processing power of GPUs for general-purpose computing.

Instead of relying only on CPUs for heavy computations, CUDA enables workloads like deep learning, scientific simulations, and matrix operations to run thousands of operations simultaneously on GPU cores.

Simple analogy

GPU → Raw compute engine
CUDA → Software layer that unlocks GPU parallelism

CUDA provides:

APIs
Compilers
Development tools
Optimized libraries

These tools allow developers to utilize GPU acceleration without writing low-level assembly code.

CPU vs GPU Architecture

Understanding CUDA requires understanding the fundamental difference between CPUs and GPUs.

Feature	CPU	GPU
Core Count	Dozens of powerful cores	Thousands of smaller cores
Execution Model	Sequential tasks	Massively parallel execution
Transistor Focus	Cache and control logic	Data processing throughput
Best Use Case	Complex control logic	Matrix operations and AI workloads

GPUs are specifically designed for data-parallel workloads, which is why they are ideal for deep learning and scientific computing.

The CUDA Software Stack

CUDA is not a single tool. It is a full ecosystem for GPU development.

nvcc – CUDA Compiler

The NVIDIA CUDA Compiler Driver (nvcc) separates:

Host code (runs on the CPU)

Device code (runs on the GPU)

This allows developers to write heterogeneous programs where CPU and GPU work together.

CUDA APIs

CUDA provides two major APIs:

CUDA Runtime API

High-level interface used in most CUDA applications.

CUDA Driver API

Low-level interface for more granular control of GPU execution.

CUDA Libraries

CUDA also provides highly optimized libraries used across AI and HPC applications.

cuBLAS

Optimized linear algebra operations for GPUs.

cuDNN

Deep neural network primitives such as convolution, pooling, softmax, and attention.

These libraries power frameworks like:

PyTorch
TensorFlow
JAX

CUDA Programming Model

CUDA assumes a heterogeneous system consisting of:

Host

CPU
Host memory

Device

GPU
Device memory

Execution typically follows this workflow.

Data Transfer

Data is copied from host memory (CPU) to device memory (GPU).

Kernel Execution

A CUDA function called a Kernel is executed on the GPU.

Execution hierarchy:

Threads
Blocks
Grids

Threads are the smallest execution units, while blocks allow threads to cooperate using shared memory.

Result Retrieval

Once the computation is complete, results are copied back from GPU memory to CPU memory.

Performance depends heavily on memory access patterns. Efficient CUDA programs maximize the use of:

Registers
Shared memory

while minimizing slower global memory access.

Why CUDA Dominates AI Infrastructure

NVIDIA’s leadership in AI infrastructure is largely due to the CUDA ecosystem.

Reasons include:

Mature development platform
Highly optimized performance libraries
Deep integration with AI frameworks
Strong developer ecosystem

Major frameworks like PyTorch and TensorFlow rely heavily on CUDA for GPU acceleration.

Because CUDA applications are built specifically for NVIDIA GPUs, it has also created a strong ecosystem around NVIDIA hardware.

Final Thoughts

CUDA has become a foundational technology for modern GPU computing. By enabling developers to harness massive parallelism inside GPUs, CUDA allows AI systems, machine learning models, and scientific computing workloads to run dramatically faster.

For developers working with AI, HPC, or GPU-accelerated computing, understanding CUDA is essential.

Original article:

Understanding NVIDIA CUDA: The Core of GPU Parallel Computing

Finally found a way to rent H100s without selling a kidney (MIG Tech)

Ethan Vance — Tue, 20 Jan 2026 12:17:28 +0000

Is it just me, or is trying to rent a dedicated H100 or A100 right now an absolute nightmare?

I've been working on some LLM fine-tuning recently, and I kept running into the same problem: Overkill.

I needed the architecture of the H100 (for the transformer engine), but I didn't need the entire card 24/7. Paying $4/hr+ for a GPU that sits idle 80% of the time just burns through the budget.

The "Aha" Moment: Splitting the Hardware

I did some digging and realized I should be looking for MIG (Multi-Instance GPU) capable servers.

If you aren't familiar with it, MIG basically lets you slice a physical GPU (like an A100 or H100) into up to 7 completely isolated instances. It’s not just software partitioning; it’s hardware-level isolation. So you get your own dedicated memory and cache.

The Resource: MIG servers

I came across a provider called MIG servers that specializes exactly in this. I wanted to share it here because their inventory is actually pretty impressive compared to the "Sold Out" signs I see everywhere else.

They seem to have bare metal stock in:

USA: Dallas, LA, Chicago
Europe: Luxembourg, London, Amsterdam
Asia: Incheon, Tokyo

What stood out to me was the flexibility. You can grab a massive 8x H100 cluster if you are training, or just slice up an A100 if you are doing inference.

Why it matters

If you are a DevOps engineer or working in AI, you know that "Time-Slicing" is usually laggy and insecure. MIG solves that.

I wrote a deeper breakdown on my personal blog about the technical specs and pricing comparisons, but I just wanted to drop this here for anyone struggling to find hardware.

To give you an idea of what MIG-ready hardware looks like, here are the specs we typically deploy for these workloads at MIG Servers:

Location	CPU	GPU Configuration	Max MIG Instances
Luxembourg	2x Xeon Platinum 8480+	8x NVIDIA H100 (200Gbps)	56 Instances
Dallas, USA	2x EPYC 9354	8x NVIDIA H100 NVLink	56 Instances
London, UK	2x Xeon Gold 6210U	NVIDIA A30	4 Instances

👉 Check out full breakdown and the server list here

Let me know if you guys have tried partitioning H100s yet!

[Boost]

Ethan Vance — Mon, 17 Nov 2025 06:40:34 +0000

Architecture for APAC: The Engineering Case for Singapore Bare Metal Infrastructure

Ethan Vance ・ Nov 17

#linux #webdev #dedicatedservers

Architecture for APAC: The Engineering Case for Singapore Bare Metal Infrastructure

Ethan Vance — Mon, 17 Nov 2025 06:28:16 +0000

For DevOps engineers and solutions architects deploying in the Asia-Pacific (APAC) region, the challenge isn't just distance; it is network topology. With a user base exceeding 3 billion, the difference between a 50ms and a 200ms Round Trip Time (RTT) dictates application viability.

While cloud virtualization offers flexibility, high-performance workloads—specifically gaming, real-time analytics, and LLM training—often hit the "noisy neighbor" wall. This article analyzes the technical infrastructure of Singapore as a hosting hub, examining connectivity ecosystem, hardware proximity, and the efficiency of bare metal over virtualized environments.

1. The Physics of Latency: Why Topology Matters

Singapore isn't just a geographical location; it is a primary peering exchange point. The island acts as a landing site for over 30 major submarine cable systems (including AAG, SJC2, and FASTER).

For a developer, this density translates to fewer hops. When you host in Singapore, you aren't routing through Japan or the US to reach Indonesia or India. You are utilizing direct peering links.

Typical RTT Metrics from Singapore (SG1):

Jakarta/Manila: < 20ms
Tokyo/Mumbai: < 50ms
Sydney: < 95ms

Technical Note: Achieving these speeds requires a provider using multi-homed BGP (Border Gateway Protocol) sessions. BGP automation ensures that if a specific carrier (e.g., NTT) experiences packet loss, the route automatically fails over to an alternative path (e.g., Tata or Singtel) without manual intervention.

2. Bare Metal vs. Virtualization Overhead

The convenience of VPS (Virtual Private Servers) comes with a performance tax known as "Hypervisor Overhead." In a virtualized environment, the physical CPU must translate instructions from the guest OS to the host hardware.

For I/O-heavy applications, this results in:

CPU Steal Time: Waiting for the physical scheduler to allocate cycles.
I/O Wait: Latency introduced by sharing disk controllers with other tenants.

The Bare Metal Advantage: Deploying on dedicated hardware (e.g., AMD EPYC 9754 or Intel Xeon Platinum) provides raw access to the kernel. There is no abstraction layer.

PCIe 5.0 & NVMe: You get full throughput (up to 14 GB/s read speeds) without virtualization throttling.

Deterministic Performance: Unlike a VPS where performance fluctuates based on neighbors, dedicated resources provide a flat-line performance graph essential for predictable SLAs.

3. The GPU Sovereignty Factor

For AI engineers working with Large Language Models (LLMs) or CUDA-accelerated rendering, hardware availability is a critical bottleneck.

Singapore offers a unique distinct advantage regarding high-performance compute (HPC) availability. Data centers here frequently stock enterprise-grade clusters (NVIDIA H100, A100, L40S) that are often supply-constrained in Western availability zones. Accessing these via bare metal allows for:

Direct Passthrough: No vGPU licensing costs or performance loss.
Cluster Scaling: Low-latency cross-connects allow for efficient multi-node training setups.

4. Data Center Efficiency (PUE) and Resilience

**
Modern infrastructure in Singapore is dictated by land scarcity, driving vertical innovation. The standard for new facilities involves strictly regulated power efficiency.

Key Specs for System Architects:

PUE (Power Usage Effectiveness): < 1.3. This is achieved via Direct-to-Chip liquid cooling, essential for sustaining the thermal design power (TDP) of modern high-density racks.
Compliance Stack: Look for SOC 2, ISO 27001, and TVRA (Threat Vulnerability Risk Assessment) certification.
N+1 Redundancy: Ensure the facility utilizes independent dual power feeds to the rack, backed by redundant UPS and generator systems.

Summary: When to Switch to Dedicated

**
While containerization (Kubernetes) on cloud instances serves microservices well, monolithic databases, high-frequency trading platforms, and game servers require the raw clock speed of dedicated hardware.

If your traceroute shows excessive hops or your database IOPS are inconsistent, moving the workload to a Singapore-based dedicated environment is the logical architectural step for stabilizing APAC performance.