DEV Community: Vultr

Configuring Smart Routing in NVIDIA Dynamo

Sanskriti Harmukh — Wed, 14 Oct 2026 21:07:15 +0000

NVIDIA Dynamo is an open-source, high-throughput, low-latency inference framework designed to deploy large-scale generative AI and reasoning models across multi-node, multi-GPU environments. Its KV Router intelligently directs inference requests by evaluating computational costs across workers, considering both prefill and decode phases. By maximizing KV cache reuse and minimizing redundant computation, smart routing reduces time-to-first-token (TTFT) by up to 40% and increases overall throughput by 25-30% compared to round-robin load balancing. This guide covers infrastructure setup, router enablement for both aggregated and disaggregated serving patterns, parameter tuning, and monitoring for production deployments. By the end, you'll have KV-aware smart routing configured and verified across both serving patterns, with the parameters tuned for your workload.

Prerequisites

Before you begin, ensure you:

Have access to a GPU-enabled server with NVIDIA GPUs installed and the NVIDIA Container Toolkit configured, using a non-root user with sudo privileges.
- 2 GPUs for aggregated serving with smart routing.
- 4 GPUs for disaggregated serving with smart routing.
Install Docker Engine and Docker Compose.
Create a Hugging Face account and generate an access token for gated models like Llama.
Understand basic LLM inference concepts (prefill vs decode phases, KV cache).

Understanding KV-Aware Routing

Traditional load balancing strategies like round-robin or random routing distribute requests evenly across workers without considering their cached state. KV-aware routing optimizes performance by directing requests to workers that already have relevant KV cache blocks, reducing redundant computation during the prefill phase.

How KV Cache Routing Works

The KV Router makes routing decisions based on a cost function that balances two factors.

1. Prefill Cost — the router calculates prefill cost by determining how many tokens need processing from scratch. When a worker has cached KV blocks matching the request's input prefix, those blocks can be reused, reducing prefill computation:

Prefill blocks = (Total input tokens - Cached tokens) / Block size

2. Decode Cost — the router tracks active blocks on each worker to estimate decode capacity. Workers with fewer active sequences have more capacity for new requests.

3. Combined Cost Formula:

cost = overlap_score_weight × prefill_blocks + decode_blocks

Where:

overlap_score_weight balances TTFT optimization (higher values) vs load distribution (lower values).
Lower cost indicates a better routing choice.
Default overlap_score_weight is 1.0.

Worker Selection Process

The router evaluates all available workers and selects the one with the lowest cost. Consider this example with 3 workers and overlap_score_weight = 1.0:

Worker	Cached Blocks	Prefill Blocks	Decode Blocks	Total Cost	Selected
Worker 1	2	8	10	18 (1.0 × 8 + 10)	No
Worker 2	5	5	5	10 (1.0 × 5 + 5)	Yes
Worker 3	8	2	9	11 (1.0 × 2 + 9)	No

Worker 2 provides the best balance of cache reuse and available capacity, minimizing both prefill computation and decode load.

Performance Benefits

KV-aware routing provides measurable improvements over basic load balancing:

TTFT Reduction: 30-40% lower time-to-first-token by maximizing cache hits.
Throughput Increase: 25-30% higher tokens/second by reducing redundant prefill computation.
Better GPU Utilization: More balanced load distribution across workers.
Cache Efficiency: Higher cache hit rates through intelligent request placement.

These benefits are most significant for workloads with long input prompts (>2000 tokens) where prefill cost dominates, similar or repetitive input prefixes across requests, and multiple inference workers with independent KV caches.

Clone the Dynamo Repository

NVIDIA Dynamo provides deployment scripts, launch utilities, and runtime modules for inference workloads.

1. Clone the repository:

$ git clone https://github.com/ai-dynamo/dynamo.git

2. Navigate to the repository directory:

$ cd dynamo

3. Switch to the latest stable release:

$ git checkout release/0.9.0

Visit the Dynamo releases page to find the latest stable version.

Verify CUDA Version and Pull the Container Image

The vLLM container requires a CUDA version match between the host driver and container runtime to prevent GPU kernel incompatibilities.

1. Check the installed CUDA version:

$ nvidia-smi

The output displays the CUDA version in the top-right corner of the table:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
....

2. Pull the vLLM container image from NGC, matching the image tag to your system's CUDA version:

For CUDA 13.x:

$ docker pull nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0-cuda13

For CUDA 12.x:

$ docker pull nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0

Visit the NVIDIA NGC Catalog to view all available image tags and CUDA versions.

3. (Optional) Build the container from source instead of pulling the pre-built image:

$ ./container/build.sh --framework VLLM

This creates an image named dynamo:latest-vllm. If you use this locally built image, replace nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0-cuda13 with dynamo:latest-vllm in all subsequent commands.

Configure Hugging Face Cache Permissions

The container runs as UID 1000 and requires write access to the Hugging Face cache directory for model downloads. Incorrect permissions prevent the container from accessing cached model weights, causing worker initialization failures.

1. Create the cache directory if it does not exist:

$ mkdir -p container/.cache/huggingface

2. Set ownership to the container user (UID 1000):

$ sudo chown -R 1000:1000 container/.cache/huggingface

3. Set appropriate permissions:

$ sudo chmod -R 775 container/.cache/huggingface

Start Infrastructure Services

Dynamo's KV Router requires etcd for worker discovery and NATS for KV cache event propagation. The etcd service maintains a registry of active workers and their capabilities, while NATS handles real-time KV block events (creation and eviction) that let the router track cached state across workers.

1. Start the infrastructure services using Docker Compose:

$ docker compose -f deploy/docker-compose.yml up -d

2. Verify the services are running:

$ docker compose -f deploy/docker-compose.yml ps

The output displays the running etcd and NATS containers:

NAME                   IMAGE                      COMMAND                  SERVICE       CREATED          STATUS          PORTS
deploy-etcd-server-1   bitnamilegacy/etcd:3.6.1   "/opt/bitnami/script…"   etcd-server   18 seconds ago   Up 18 seconds   0.0.0.0:2379-2380->2379-2380/tcp, [::]:2379-2380->2379-2380/tcp
deploy-nats-server-1   nats:2.11.4                "/nats-server -c /et…"   nats-server   18 seconds ago   Up 18 seconds   0.0.0.0:4222->4222/tcp, [::]:4222->4222/tcp, 0.0.0.0:6222->6222/tcp, [::]:6222->6222/tcp, 0.0.0.0:8222->8222/tcp, [::]:8222->8222/tcp

KV Router Configuration Parameters

The KV Router uses environment variables to control routing behavior, cache tracking, and worker selection logic. These parameters determine how the router balances Time To First Token (TTFT) optimization through cache reuse against Inter-Token Latency (ITL) through load distribution. All parameters can be set via environment variables or command-line arguments on the Dynamo frontend.

Core Router Parameters

Variable	Description	Default	Recommended	Notes
`DYN_ROUTER_MODE`	Enables KV cache-aware routing strategy	`round_robin`	`kv`	Set to `kv` to enable cache-aware routing. Other modes: `round_robin`, `random`
`DYN_ROUTER_TEMPERATURE`	Controls routing randomness via softmax sampling	`0.0`	`0.1`	`0.0` = deterministic (always lowest cost). Higher values add randomness to prevent saturation
`DYN_KV_OVERLAP_SCORE_WEIGHT`	Weight for prefix cache overlaps in cost calculation	`1.0`	`1.5`	Formula: `cost = weight × prefill_blocks + decode_blocks`. Higher values prioritize TTFT over ITL
`DYN_KV_CACHE_BLOCK_SIZE`	KV cache block size in tokens (must match backend)	Backend-specific	`16`	Must match vLLM's block size. Used for prefill block calculation
`DYN_KV_EVENTS`	Enables real-time KV event tracking from workers	`true`	`true`	When disabled, router predicts cache state with TTL-based expiration

Worker Capacity Thresholds

These parameters define when a worker is considered "busy" and should be excluded from routing decisions.

Variable	Description	Default	Recommended	Notes
`DYN_ACTIVE_DECODE_BLOCKS_THRESHOLD`	Busy threshold based on KV cache block utilization	Not set	`0.85`	Value between 0.0-1.0. Workers exceeding this percentage are excluded. Can update via `/busy_threshold` API
`DYN_ACTIVE_PREFILL_TOKENS_THRESHOLD`	Busy threshold based on prefill token count	Not set	`10000`	Absolute token count. Workers exceeding this are excluded. Uses OR logic with threshold fraction

Infrastructure Configuration

Variable	Description	Default	Notes
`ETCD_ENDPOINTS`	etcd server URL for worker discovery	N/A	Format: `http://localhost:2379`. Multiple endpoints separated by comma
`NATS_SERVER`	NATS server URL for KV event propagation	N/A	Format: `nats://localhost:4222`. Required for KV event streaming
`DYN_EVENT_PLANE`	Event plane backend for KV cache events	N/A	Set to `nats` to enable NATS-based event publishing
`PYTHONHASHSEED`	Deterministic hash seed for KV event IDs	N/A	Set to `0` for reproducible KV event ID generation
`DYN_SYSTEM_PORT`	Port for worker system status server	System-dependent	Used for health checks and metrics. Set per-worker to avoid conflicts (e.g., `8081`, `8082`)

Parameter Tuning Guidelines

For TTFT Optimization (Low Latency):

Increase DYN_KV_OVERLAP_SCORE_WEIGHT to 1.5-2.0 (prioritize cache hits).
Lower busy thresholds: DYN_ACTIVE_DECODE_BLOCKS_THRESHOLD=0.70, DYN_ACTIVE_PREFILL_TOKENS_THRESHOLD=500.
Keep DYN_ROUTER_TEMPERATURE low (0.0-0.1) for deterministic selection.

For Throughput Optimization (High Load):

Lower DYN_KV_OVERLAP_SCORE_WEIGHT to 0.5-1.0 (balance cache hits with load).
Higher busy thresholds: DYN_ACTIVE_DECODE_BLOCKS_THRESHOLD=0.95, DYN_ACTIVE_PREFILL_TOKENS_THRESHOLD=2000.
Increase DYN_ROUTER_TEMPERATURE to 0.3-0.5 for better load distribution.

When KV Events Are Unavailable:

Set DYN_KV_EVENTS=false to use prediction-based routing.
Router tracks state via routing decisions with TTL-based expiration (default 120s).
Less accurate than real-time events but works without backend KV event support.

Set Up the Deployment Environment

Both aggregated and disaggregated serving deployments use the same KV routing configuration. This section prepares the container environment and creates a shared configuration file for all deployment scripts.

1. Export your Hugging Face token (replace YOUR_HF_TOKEN with your actual token):

$ export HF_TOKEN=YOUR_HF_TOKEN

2. Run the vLLM container with GPU access and workspace mounting, using the image tag that matches your CUDA version:

$ ./container/run.sh -it --framework VLLM --mount-workspace --image nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0-cuda13 -e HF_TOKEN=$HF_TOKEN

This starts an interactive container session with GPU support and passes the Hugging Face token to the container.

3. Inside the container, create the common environment configuration file:

$ cat << 'EOF' > ~/kv_routing.env
# Infrastructure endpoints
export ETCD_ENDPOINTS=http://localhost:2379
export NATS_SERVER=nats://localhost:4222
export DYN_EVENT_PLANE=nats

# Set deterministic hash for KV event IDs
export PYTHONHASHSEED=0

# KV Router Configuration
export DYN_ROUTER_MODE=kv
export DYN_ROUTER_TEMPERATURE=0.1
export DYN_KV_OVERLAP_SCORE_WEIGHT=1.5
export DYN_KV_CACHE_BLOCK_SIZE=16
export DYN_KV_EVENTS=true

# Optional: Advanced routing configuration
export DYN_ACTIVE_DECODE_BLOCKS_THRESHOLD=0.85
export DYN_ACTIVE_PREFILL_TOKENS_THRESHOLD=10000

echo "Environment configuration loaded:"
echo "  etcd: $ETCD_ENDPOINTS"
echo "  NATS: $NATS_SERVER"
echo "  Router Mode: $DYN_ROUTER_MODE"
EOF

This file contains shared configuration variables for all deployment scripts, including etcd endpoints, NATS server address, and KV routing parameters.

Configure KV Routing for Aggregated Serving

Aggregated serving combines prefill and decode phases on a single worker. KV-aware routing optimizes which worker receives each request based on cached KV blocks, improving performance even when workers handle both phases.

Deploy Aggregated Serving with KV Routing

The deployment script launches multiple vLLM workers with KV event publishing enabled and a frontend configured for KV-aware routing, handling process cleanup and providing routing configuration feedback.

1. Inside the container, create the aggregated serving deployment script:

$ cat << 'EOF' > ~/nemotron_agg_kv_routing.sh
#!/bin/bash
source ~/kv_routing.env

MODEL="${1:-nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1}"

pkill -f "dynamo.frontend"; pkill -f "dynamo.vllm"; sleep 2
set -e
trap 'kill 0' EXIT

python -m dynamo.frontend --router-reset-states &
sleep 10

# Worker 1 - GPU 0 (ZMQ port 5557)
DYN_SYSTEM_PORT=${DYN_SYSTEM_PORT1:-8081} \
CUDA_VISIBLE_DEVICES=0 python3 -m dynamo.vllm \
    --model "$MODEL" \
    --trust-remote-code \
    --connector none \
    --enforce-eager \
    --enable-prefix-caching \
    --kv-events-config '{"enable_kv_cache_events":true,"endpoint":"tcp://*:5557"}' &

# Worker 2 - GPU 1 (ZMQ port 5558)
DYN_SYSTEM_PORT=${DYN_SYSTEM_PORT2:-8082} \
CUDA_VISIBLE_DEVICES=1 python3 -m dynamo.vllm \
    --model "$MODEL" \
    --trust-remote-code \
    --connector none \
    --enforce-eager \
    --enable-prefix-caching \
    --kv-events-config '{"enable_kv_cache_events":true,"endpoint":"tcp://*:5558"}' &

sleep 30
echo "Aggregated serving ready at http://localhost:8000 (Model: $MODEL)"
wait
EOF

This script configures aggregated serving with KV-aware routing:

Accepts the model name as a command-line argument (defaults to NVIDIA Nemotron Nano 4B).
Sources the environment configuration created earlier (includes DYN_EVENT_PLANE=nats).
Uses --connector none for aggregated serving (prefill+decode on same worker).
Uses --enforce-eager for fast deployment without compilation.
Enables prefix caching with --enable-prefix-caching for KV cache reuse.
Enables KV event publishing with ZMQ endpoints (ports 5557, 5558) on both workers.
Configures system ports with DYN_SYSTEM_PORT to avoid conflicts (8081, 8082).
Uses --trust-remote-code to support NVIDIA Nemotron models.

KV Event Flow Architecture: vLLM natively publishes KV cache events via ZMQ. Dynamo's bridge component automatically translates these events to NATS for router consumption:

┌─────────────────────┐
│ vLLM Worker 1       │
│ (ZMQ: tcp://*:5557) │───┐
└─────────────────────┘   │                                           ┌─────────────────────┐
                          │    ┌──────────────┐    ┌──────────────┐   │ KV Router           │
                          ├───>│ Dynamo Bridge├───>│ NATS Server  ├──>│ (Builds Radix Tree) │
                          │    └──────────────┘    └──────────────┘   │ Global Cache View   │
┌─────────────────────┐   │                                           └─────────────────────┘
│ vLLM Worker 2       │   │
│ (ZMQ: tcp://*:5558) │───┘
└─────────────────────┘

This architecture provides the router with a unified view of cache state across all workers while allowing vLLM to use its native ZMQ event publishing. You'll see ZMQ endpoints in worker logs, but events flow through the Dynamo bridge to NATS for router consumption.

2. Make the script executable:

$ chmod +x ~/nemotron_agg_kv_routing.sh

3. Run the aggregated serving script:

$ ~/nemotron_agg_kv_routing.sh

The script starts the frontend with KV routing enabled and launches two vLLM workers on GPUs 0 and 1. The frontend connects to NATS and begins consuming KV events from all registered workers, building a prefix (radix) tree to track cached blocks across workers.

To deploy a different model, pass the model name as an argument:

$ ~/nemotron_agg_kv_routing.sh "nvidia/Llama-3_3-Nemotron-Super-49B-v1_5"

Verify KV Routing

The router logs provide insights into worker selection and cost calculations.

1. Open a new terminal session and send a test request to the frontend:

$ curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1",
    "messages": [
      {"role": "system", "content": "You are a helpful AI assistant."},
      {"role": "user", "content": "Explain the concept of KV cache in LLM inference."}
    ],
    "max_tokens": 100,
    "temperature": 0.7
  }'

The output displays the model's response in JSON format with streaming tokens.

2. Check the frontend logs for routing decision details, which show cost calculations for each worker:

Formula for worker_id=7587893399646425890 dp_rank=0 with 0 cached blocks: 5.188 = 1.5 * prefill_blocks + decode_blocks = 1.5 * 2.125 + 2.000
Formula for worker_id=7587893399646425892 dp_rank=0 with 0 cached blocks: 5.188 = 1.5 * prefill_blocks + decode_blocks = 1.5 * 2.125 + 2.000
Selected worker: worker_id=7587893399646425892 dp_rank=0, logit: 5.188, cached blocks: 0, tree size: 0, total blocks: 75379

The router calculates cost per worker with cost = overlap_weight × prefill_blocks + decode_blocks. In this example, both workers have 0 cached blocks (cold start), equal prefill costs (2.125 blocks weighted by 1.5), and equal decode loads (2.000). With identical costs (5.188), the router selects a worker based on temperature sampling.

3. Send a second request with a similar prompt prefix:

$ curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1",
    "messages": [
      {"role": "system", "content": "You are a helpful AI assistant."},
      {"role": "user", "content": "Explain the concept of KV cache reuse in distributed LLM serving."}
    ],
    "max_tokens": 100,
    "temperature": 0.7
  }'

4. Check the routing decision for the second request in the logs. If the router detects cached blocks from the previous request with a matching prompt prefix, the logs show a reduced prefill cost:

Formula for worker_id=7587893399646425890 dp_rank=0 with 0 cached blocks: 5.375 = 1.5 * prefill_blocks + decode_blocks = 1.5 * 2.250 + 2.000
Formula for worker_id=7587893399646425892 dp_rank=0 with 1 cached blocks: 3.875 = 1.5 * prefill_blocks + decode_blocks = 1.5 * 1.250 + 2.000
Selected worker: worker_id=7587893399646425892 dp_rank=0, logit: 3.875, cached blocks: 1, tree size: 8, total blocks: 75379

The router prioritizes the worker with 1 cached block, reducing its cost from 5.375 to 3.875 — the cached block reduces prefill computation needed (1.250 blocks vs 2.250), demonstrating KV cache reuse resulting in faster TTFT for requests with overlapping prompt prefixes.

5. Open a new terminal session on the host (outside the container) and monitor the KV event stream to verify event publishing:

$ docker run --rm -it --network container:deploy-nats-server-1 \
  natsio/nats-box nats stream info namespace-dynamo-component-backend-kv-events

The output displays the JetStream statistics including message count and consumer information:

Information for Stream namespace-dynamo-component-backend-kv-events
......
                Subjects: namespace-dynamo-component-backend-kv-events.*
                Replicas: 1
                Storage: File
......
State:
......
                Messages: 8
                Bytes: 2.3 KiB
        Active Consumers: 1
......

The message count increases as workers publish KV cache events — each message represents a KV cache state change, such as block creation or eviction.

Configure KV Routing for Disaggregated Serving

Disaggregated serving separates prefill and decode phases across different worker pools. The KV Router optimizes both prefill worker selection (based on cache hits) and decode worker assignment (based on load distribution), while coordinating KV cache transfer between phases via NIXL.

Deploy Disaggregated Serving with KV Routing

The deployment script configures the frontend with increased overlap weight for prefill optimization, launches prefill workers with low batch sizes for TTFT optimization, and decode workers with high batch sizes for throughput maximization. The NIXL connector enables efficient KV cache transfer between prefill and decode workers.

1. If the aggregated serving script is still running from the previous section, stop it by pressing Ctrl+C in the container terminal.

2. Inside the same container, create the disaggregated serving deployment script:

$ cat << 'EOF' > ~/nemotron_disagg_kv_routing.sh
#!/bin/bash
source ~/kv_routing.env

MODEL="${1:-nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1}"

pkill -f "dynamo.frontend"; pkill -f "dynamo.vllm"; sleep 2
set -e
trap 'kill 0' EXIT

python -m dynamo.frontend --router-reset-states &
sleep 10

# Decode Worker 1 - GPU 0
CUDA_VISIBLE_DEVICES=0 python3 -m dynamo.vllm \
    --model "$MODEL" \
    --connector nixl \
    --is-decode-worker \
    --max-model-len 2048 \
    --enable-prefix-caching \
    --trust-remote-code &

# Decode Worker 2 - GPU 1
VLLM_NIXL_SIDE_CHANNEL_PORT=20097 \
CUDA_VISIBLE_DEVICES=1 python3 -m dynamo.vllm \
    --model "$MODEL" \
    --connector nixl \
    --is-decode-worker \
    --max-model-len 2048 \
    --enable-prefix-caching \
    --trust-remote-code &

# Prefill Worker 1 - GPU 2 (ZMQ port 5559)
CUDA_VISIBLE_DEVICES=2 python3 -m dynamo.vllm \
    --model "$MODEL" \
    --connector nixl \
    --is-prefill-worker \
    --max-model-len 2048 \
    --enable-prefix-caching \
    --trust-remote-code \
    --kv-events-config '{"enable_kv_cache_events":true,"endpoint":"tcp://*:5559"}' &

# Prefill Worker 2 - GPU 3 (ZMQ port 5560)
VLLM_NIXL_SIDE_CHANNEL_PORT=20099 \
CUDA_VISIBLE_DEVICES=3 python3 -m dynamo.vllm \
    --model "$MODEL" \
    --connector nixl \
    --is-prefill-worker \
    --max-model-len 2048 \
    --enable-prefix-caching \
    --trust-remote-code \
    --kv-events-config '{"enable_kv_cache_events":true,"endpoint":"tcp://*:5560"}' &

sleep 30
echo "Disaggregated serving ready at http://localhost:8000 (Model: $MODEL)"
echo "Architecture: 2 Decode Workers (GPU 0-1) + 2 Prefill Workers (GPU 2-3)"
wait
EOF

This script configures disaggregated serving with KV-aware routing and NIXL for KV cache transfer:

Accepts the model name as a command-line argument (defaults to NVIDIA Nemotron Nano 4B).
Sources the environment configuration created earlier (includes DYN_EVENT_PLANE=nats).
Uses --connector nixl to enable NIXL for KV cache transfer between prefill and decode workers.
Sets unique NIXL ports via VLLM_NIXL_SIDE_CHANNEL_PORT for workers 2 and 4 (20097, 20099).
Enables prefix caching on all workers with --enable-prefix-caching for KV cache reuse.
Configures decode workers on GPUs 0-1 to receive KV cache from prefill workers via NIXL.
Configures prefill workers on GPUs 2-3 with ZMQ endpoints (ports 5559, 5560) for KV event publishing.
Uses --trust-remote-code to support NVIDIA Nemotron models.
Uses --max-model-len 2048 for efficient memory usage across all workers.
Frontend automatically detects prefill and decode workers to enable disaggregated routing.

3. Make the script executable:

$ chmod +x ~/nemotron_disagg_kv_routing.sh

4. Run the disaggregated serving script:

$ ~/nemotron_disagg_kv_routing.sh

The script starts the frontend, two decode workers on GPUs 0-1, and two prefill workers on GPUs 2-3. The frontend automatically detects prefill workers and creates an internal prefill router for cache-aware prefill worker selection, while the main router handles decode worker assignment based on load distribution.

To deploy a different model, pass the model name as an argument:

$ ~/nemotron_disagg_kv_routing.sh "nvidia/Llama-3_3-Nemotron-Super-49B-v1_5"

Verify Disaggregated KV Routing

Disaggregated routing involves two routing decisions: prefill worker selection based on cache overlap, and decode worker assignment based on load and available capacity.

1. Open a new terminal session and send a test request to verify the disaggregated flow:

$ curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1",
    "messages": [
      {"role": "system", "content": "You are a helpful AI assistant specialized in distributed systems."},
      {"role": "user", "content": "Explain how KV cache transfer works in disaggregated LLM serving using NIXL."}
    ],
    "max_tokens": 150,
    "temperature": 0.7
  }'

2. Check the frontend logs for prefill routing decisions, which show prefill worker cost calculations with the higher overlap weight (1.5):

INFO dynamo_llm::kv_router::scheduler: Formula for worker_id=7587893399646425930 dp_rank=0 with 0 cached blocks: 4.125 = 1.5 * prefill_blocks + decode_blocks = 1.5 * 2.750 + 0.000
INFO dynamo_llm::kv_router::scheduler: Formula for worker_id=7587893399646425934 dp_rank=0 with 0 cached blocks: 4.125 = 1.5 * prefill_blocks + decode_blocks = 1.5 * 2.750 + 0.000
INFO dynamo_llm::kv_router::scheduler: Selected worker: worker_id=7587893399646425930 dp_rank=0, logit: 4.125, cached blocks: 0, tree size: 0, total blocks: 75379

On the first (cold-start) request, both prefill workers have 0 cached blocks, resulting in equal costs (4.125). The decode blocks component is 0.0 because prefill workers only handle the prefill phase.

3. Check the decode routing decisions in the script output. After prefill completion, the decode router assigns the request to a decode worker:

INFO dynamo_llm::kv_router::scheduler: Formula for worker_id=7587893399646425936 dp_rank=0 with 0 cached blocks: 2.000 = 0.0 * prefill_blocks + decode_blocks = 0.0 * 2.750 + 2.000
INFO dynamo_llm::kv_router::scheduler: Selected worker: worker_id=7587893399646425936 dp_rank=0, logit: 2.000, cached blocks: 0, tree size: 0, total blocks: 75379

The prefill blocks component is 0.0 in this formula because decode workers don't compute prefills — they receive KV cache from prefill workers via NIXL transfer.

4. Monitor NIXL transfer success in the worker logs displayed by the script:

INFO nixl_connector._nixl_handshake: NIXL compatibility check passed (hash: cae818f9461cb277c305be4b8cd4aca3273e21a6ea98f7399d1a339560e1b3ea)

NIXL successfully transfers KV cache blocks from prefill to decode workers using RDMA for low-latency, high-bandwidth data transfer.

5. Send a second request with a similar prompt prefix to verify cache reuse:

$ curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1",
    "messages": [
      {"role": "system", "content": "You are a helpful AI assistant specialized in distributed systems."},
      {"role": "user", "content": "Explain how KV cache reuse works in disaggregated LLM serving."}
    ],
    "max_tokens": 150,
    "temperature": 0.7
  }'

6. Check the prefill routing logs for the second request, which show improved cache hits:

INFO dynamo_llm::kv_router::scheduler: Formula for worker_id=7587893399646425930 dp_rank=0 with 1 cached blocks: 2.250 = 1.5 * prefill_blocks + decode_blocks = 1.5 * 1.500 + 0.000
INFO dynamo_llm::kv_router::scheduler: Formula for worker_id=7587893399646425934 dp_rank=0 with 0 cached blocks: 3.750 = 1.5 * prefill_blocks + decode_blocks = 1.5 * 2.500 + 0.000
INFO dynamo_llm::kv_router::scheduler: Selected worker: worker_id=7587893399646425930 dp_rank=0, logit: 2.250, cached blocks: 1, tree size: 2, total blocks: 75379

The prefill router now sees 1 cached block on the selected worker, reducing its cost from 3.750 to 2.250 — demonstrating KV-aware routing's ability to optimize TTFT by leveraging cached prompt prefixes.

Configure KV Routing with SGLang and TensorRT-LLM

The KV Router environment configuration is universal across all inference backends. The same environment variables used for vLLM deployments work identically with SGLang and TensorRT-LLM, requiring only backend-specific adjustments for block size and KV event publishing.

Universal Environment Variables

These environment variables work across all backends (vLLM, SGLang, TensorRT-LLM) without modification:


# Infrastructure endpoints
export ETCD_ENDPOINTS=http://localhost:2379
export NATS_SERVER=nats://localhost:4222
export DYN_EVENT_PLANE=nats
export PYTHONHASHSEED=0

# Router configuration
export DYN_ROUTER_MODE=kv
export DYN_ROUTER_TEMPERATURE=0.1
export DYN_KV_OVERLAP_SCORE_WEIGHT=1.5
export DYN_KV_EVENTS=true

# Load balancing thresholds
export DYN_ACTIVE_DECODE_BLOCKS_THRESHOLD=0.85
export DYN_ACTIVE_PREFILL_TOKENS_THRESHOLD=10000

Backend-Specific Configuration

While the router environment variables are universal, two aspects require backend-specific configuration.

Block Size Configuration — DYN_KV_CACHE_BLOCK_SIZE must match your backend's actual block size:

Backend	Typical Block Size	Configuration
SGLang	16	`export DYN_KV_CACHE_BLOCK_SIZE=16`
TensorRT-LLM	64 or 128 (varies by model)	`export DYN_KV_CACHE_BLOCK_SIZE=64`
vLLM	16	`export DYN_KV_CACHE_BLOCK_SIZE=16`

Mismatched block sizes between the router and workers cause incorrect cost calculations and routing decisions. Always verify your backend's block size configuration.

KV Event Publishing Configuration — each backend uses different command-line parameters to enable KV event publishing to the router:

Backend	Parameter	Example Value
SGLang	`--kv-events-config`	`nats://localhost:4222`
TensorRT-LLM	`--publish-events-and-metrics`	No value (flag only)
vLLM	`--kv-events-config`	`'{"enable_kv_cache_events":true,"endpoint":"tcp://*:5557"}'`

SGLang publishes events using a NATS URL format:

--kv-events-config nats://localhost:4222

TensorRT-LLM uses a simple flag to enable event publishing:

--publish-events-and-metrics

vLLM uses a JSON configuration format with a ZMQ endpoint:

--kv-events-config '{"enable_kv_cache_events":true,"endpoint":"tcp://*:5557"}'

Tuning KV Router Parameters

The KV Router provides several parameters to optimize routing behavior for different workload characteristics. Proper tuning balances cache reuse (TTFT optimization) with load distribution (ITL optimization).

Overlap Score Weight

The --kv-overlap-score-weight parameter controls the router's preference for cache hits versus load balancing. Default value: 1.0.

Increase (1.5-2.0) for prefill-heavy workloads — long input prompts (>2000 tokens), short output generation (<200 tokens), and where minimizing TTFT is critical:

$ python3 -m dynamo.frontend \
  --router-mode kv \
  --kv-overlap-score-weight 1.5

Decrease (0.5-0.8) for decode-heavy workloads — short input prompts (<500 tokens), long output generation (>1000 tokens), and where maximizing throughput is critical:

$ python3 -m dynamo.frontend \
  --router-mode kv \
  --kv-overlap-score-weight 0.7

Set to 0.0 to disable cache-aware routing and use pure load balancing:

$ python3 -m dynamo.frontend \
  --router-mode kv \
  --kv-overlap-score-weight 0.0

This mode ignores cached blocks and routes based solely on worker load — useful for workloads with no prefix overlap or for baseline performance comparisons.

Router Temperature

The --router-temperature parameter introduces randomness in worker selection to prevent saturation and improve load distribution. Default value: 0.0 (deterministic).

Temperature = 0.0 (default): Always select the lowest-cost worker. Best for maximizing cache reuse in stable workloads, with the risk of oversaturating workers with the best cache state.
Temperature = 0.2-0.5: Add slight randomness while still favoring low-cost workers. Best for production workloads with variable request patterns, giving better load distribution without sacrificing much cache efficiency:

$ python3 -m dynamo.frontend \
  --router-mode kv \
  --router-temperature 0.3

Temperature = 1.0+: More uniform distribution across workers. Best for debugging, testing, or when cache reuse is less important — approaches random selection at very high temperatures.

KV Events Configuration

The --no-kv-events flag disables real-time KV event tracking and switches to prediction-based routing. Default: KV events enabled.

When enabled, the router predicts cache state based on its own routing decisions instead of receiving events from workers:

$ python3 -m dynamo.frontend \
  --router-mode kv \
  --no-kv-events \
  --router-ttl 120 \
  --router-max-tree-size 1048576 \
  --router-prune-target-ratio 0.8

Additional parameters for prediction mode:

--router-ttl 120: Expire cached blocks after 120 seconds (default: 120).
--router-max-tree-size 1048576: Maximum blocks before pruning (default: 2^20).
--router-prune-target-ratio 0.8: Prune to 80% of max size when threshold exceeded.

Use prediction mode when your backend doesn't support KV event publishing, you want lower infrastructure overhead (no NATS dependency), or your deployment is simple enough that prediction accuracy is acceptable.

When using --no-kv-events, configure backends to disable event publishing:

vLLM: Use --kv-events-config '{"enable_kv_cache_events": false}'.
SGLang: Do not use --kv-events-config.
TRT-LLM: Do not use --publish-events-and-metrics.

Active Block Busy Detection

The --active-decode-blocks-threshold parameter marks workers as busy when their KV cache utilization exceeds a threshold, preventing overload. Default: disabled (no threshold).

$ python3 -m dynamo.frontend \
  --router-mode kv \
  --active-decode-blocks-threshold 0.85 \
  --active-prefill-tokens-threshold 10000

--active-decode-blocks-threshold 0.85: Mark worker busy at 85% KV cache utilization.
--active-prefill-tokens-threshold 10000: Mark worker busy when prefill tokens exceed 10,000.

Busy workers are excluded from routing decisions until their load drops below the threshold, preventing request queueing and maintaining consistent latency. Thresholds can be updated at runtime without restarting the frontend:

$ curl -X POST http://localhost:8000/busy_threshold \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1",
    "active_decode_blocks_threshold": 0.90,
    "active_prefill_tokens_threshold": 12000
  }'

The response confirms the updated thresholds:

{
  "model": "nvidia/Llama-3.1-Nemotron-Nano-4B-v1.1",
  "active_decode_blocks_threshold": 0.90,
  "active_prefill_tokens_threshold": 12000
}

Router Replica Synchronization

The --router-replica-sync flag enables active block state synchronization between multiple router replicas for consistent load balancing. Default: disabled.

Enable it when launching multiple frontend replicas for fault tolerance and horizontal scaling:


# Router replica 1
$ python3 -m dynamo.frontend \
  --router-mode kv \
  --http-port 8000 \
  --router-replica-sync

# Router replica 2
$ python3 -m dynamo.frontend \
  --router-mode kv \
  --http-port 8001 \
  --router-replica-sync

Each replica maintains its own view of active blocks (ephemeral state); replicas exchange routing decisions via NATS core messaging, and cached block state (persistent) syncs automatically via JetStream. New replicas start with zero active blocks but converge quickly through sync, giving consistent routing decisions and better load distribution across replicas.

Monitor KV Router Performance

To monitor KV Router performance metrics including routing decisions, cache efficiency, and worker utilization, deploy an observability stack (Grafana and Prometheus) alongside your Dynamo deployment. Add the KV Router environment variables from this guide to your deployment configuration, then use Grafana dashboards to visualize KV cache routing performance metrics and Prometheus to track TTFT improvements, cache hit rates, and worker selection patterns.

Next Steps

Tune DYN_KV_OVERLAP_SCORE_WEIGHT and DYN_ROUTER_TEMPERATURE against your real traffic patterns rather than the defaults
Stand up Grafana and Prometheus dashboards to track TTFT, cache hit rate, and worker load over time
Test SGLang or TensorRT-LLM backends with the same router configuration to compare cache efficiency
Enable --router-replica-sync before scaling to multiple frontend replicas in production

For the full guide with additional tips, visit the original article on Vultr Docs.

Automating Bash Scripts on Linux Using Cron and Systemd

Sanskriti Harmukh — Wed, 14 Oct 2026 21:06:31 +0000

Automating Bash scripts on Linux is essential for system administration tasks, such as installing applications and monitoring system logs. Cron is a job scheduler that runs tasks at specific intervals, while systemd is a service manager that handles applications through service profiles. This guide walks through writing two Bash scripts — one for log monitoring and one for backups — then automating them with a cron job and running one as a persistent systemd service. By the end, you'll have both a scheduled cron job and a managed systemd service running your own Bash scripts automatically.

Prerequisites

Before you begin:

Have access to a Ubuntu server as your management workstation.
Access the server using SSH as a non-root sudo user.
Install a LAMP stack (Apache, MySQL, PHP).

Create Bash Scripts

Bash scripts support multiple commands and functions that define how tasks run in a system. Follow the steps below to create two Bash scripts: one to back up critical web application files, and one to monitor system logs for errors.

1. Switch to your user's home directory:

$ cd

2. Create a new log-monitor.sh script:

$ nano log-monitor.sh

3. Add the following contents to the log-monitor.sh file:

#!/bin/bash

LOG_FILE="/var/log/syslog" 
ERROR_LOG="/opt/data/error_log.txt"
WARNING_LOG="/opt/data/warning_log.txt"

tail -F "$LOG_FILE" | while read line; do
    if echo "$line" | grep -q "error"; then
        echo "$line" >> "$ERROR_LOG"
    elif echo "$line" | grep -q "warning"; then
        echo "$line" >> "$WARNING_LOG"
    fi
done

Save and close the file. This script creates error_log.txt and warning_log.txt files in the /opt/data directory, filtering results from /var/log/syslog using a while loop. It writes each new line to the respective error or warning file as it appears.

4. Create another script named backup.sh:

$ nano backup.sh

5. Add the following contents to the file:

#!/bin/bash

DB_USER="user"
DB_PASSWORD="new-password"
DB_NAME="mysql"

DB_BACKUP_DIR="mysql_backup"
WEB_FILES_DIR="web_backup"

mkdir -p "$DB_BACKUP_DIR"
mkdir -p "$WEB_FILES_DIR"

DATE=$(date +"%Y%m%d_%H%M%S")

mysqldump -u"$DB_USER" -p"$DB_PASSWORD" "$DB_NAME" > "$DB_BACKUP_DIR/db_backup_$DATE.sql"
echo "MySQL backup completed: $DB_BACKUP_DIR/db_backup_$DATE.sql"

tar -czf "$WEB_FILES_DIR/web_backup_$DATE.tar.gz" /var/www/html
echo "Web application backup completed: $WEB_FILES_DIR/web_backup_$DATE.tar.gz"

Save and close the file. This script connects to the MySQL database server using the DB_... variables, then creates mysql_backup and web_backup directories to store the database and web application backups.

6. Move the Bash scripts to a system-wide directory like /opt:

$ sudo cp backup.sh log-monitor.sh /opt/

7. Create a new data directory under the system-wide directory:

$ sudo mkdir -p /opt/data

8. Change the directory permissions to 777 to allow all users to write files:

$ sudo chmod 777 /opt/data

Automate the Bash Script Using Cron

Cron creates jobs you can use to schedule and automatically run a Bash script at specific intervals. Follow the steps below to automate the backup.sh script to back up the MySQL database and web application files daily.

1. Open the crontab editor:

$ crontab -e

Select your desired text editor when prompted — for example, select 1 to choose the nano text editor.

2. Add the following directive at the end of the file:

0 2 * * * /opt/backup.sh

Save and close the file. This cron job runs backup.sh daily at 2:00 A.M. In the job:

0: Runs the script at the start of a minute. Set a value between 0 and 59.
2: Runs the script at 2:00 A.M. Specify an hour between 0 and 23.
*: Runs the script every day of the month. Specify a day between 1 and 31.
*: Runs the script every month. Specify a month between 1 and 12.
*: Runs the script daily. Specify a day of the week between 0 and 7.

3. List all cron jobs and verify the new task is available:

$ crontab -l

Output:

# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command

0 2 * * * ~/backup.sh

Automate the Bash Script as a System Service Using Systemd

Systemd starts system services automatically at boot and offers options for stopping and restarting them. Follow the steps below to set up the log-monitoring script as a system service.

1. Create a new system service file:

$ sudo nano /etc/systemd/system/logmonitor.service

2. Add the following contents to the logmonitor.service file:

[Unit]
Description= Automatic System Logs Monitoring and Filtering
After=network.target

[Service]
ExecStart=bash /opt/log-monitor.sh
Restart=always

[Install]
WantedBy=multi-user.target

Save and close the file. This service configuration runs the log-monitor.sh script from /opt to monitor and filter log entries in /var/log/syslog.

3. Reload systemd to apply the service file changes:

$ sudo systemctl daemon-reload

4. Enable the new system service to automatically start at boot:

$ sudo systemctl enable logmonitor.service

Output:

Created symlink /etc/systemd/system/multi-user.target.wants/logmonitor.service → /etc/systemd/system/logmonitor.service.

5. Start the system service:

$ sudo systemctl start logmonitor

6. View the system service status and verify it's running:

$ sudo systemctl status logmonitor

Output:

● logmonitor.service - Automatic System Logs Monitoring and Filtering
 Loaded: loaded (/etc/systemd/system/logmonitor.service; enabled; preset: enabled)
 Active: active (running) since Wed 2024-11-06 13:20:07 UTC; 11s ago
Main PID: 14293 (bash)
   Tasks: 3 (limit: 1062)
  Memory: 784.0K (peak: 1.5M)
     CPU: 40ms
  CGroup: /system.slice/logmonitor.service
          ├─14293 bash /opt/log-monitor.sh
          ├─14308 tail -F /var/log/syslog
          └─14309 bash /opt/log-monitor.sh

Nov 06 13:20:07 Linux-server systemd[1]: Started logmonitor.service - Automatic System Logs Monitoring and Filtering.

7. Run the following commands to simulate new error and warning entries in /var/log/syslog:

$ logger -p user.err "This is a simulated error log message from MySQL"
$ logger -p user.err "This is a simulated warning log message from Apache"
$ logger -p user.err "This is a simulated error log message from PHP"
$ logger -p user.err "This is a simulated Warning log message from MySQL"

8. List all files in /opt/data and verify that error_log.txt and warning_log.txt are available:

$ ls /opt/data

Output:

error_log.txt warning_log.txt

9. View the error_log.txt file and verify that only error entries are present:

$ cat error.log

Output:

2024-11-06T13:33:48.010096+00:00 Server linuxuser: This is a simulated error log message from MySQL
2024-11-06T13:33:48.010096+00:00 Server linuxuser: This is a simulated error log message from PHP

Next Steps

Add more granular filters (by service name or severity) to log-monitor.sh
Extend backup.sh with retention logic to prune old backups automatically
Ship backups to remote or off-site storage so they survive a full server loss
Add alerting (email or webhook) when new entries land in error_log.txt

For the full guide with additional tips, visit the original article on Vultr Docs.

Managing File and Directory Permissions in Linux

Sanskriti Harmukh — Wed, 14 Oct 2026 21:05:56 +0000

Linux permissions control how users access, modify, and execute files or directories on a system. Efficient management of user permissions improves system security and access control by preventing unauthorized access to sensitive data, files, and directories. This guide explains how to read and interpret Linux permissions, then use the chmod and chown commands to change them. By the end, you'll be able to view, calculate, and modify file and directory permissions — including special permissions like the sticky bit, SUID, and SGID — on your Linux workstation.

Linux Permissions

Linux permissions are categorized into read, write, and execute (RWX), which define user or group privileges on a file or directory:

Read (r): View the contents of a file or list the contents of a directory.
Write (w): Modify, add, remove, or rename files in a directory.
Execute (x): Run a file or access the files in a directory.

Permissions can be assigned to the following user classes:

Owner: The user who owns the file or directory.
Group: The group associated with the file or directory.
Others: All other system users.

If a user owns a file and also belongs to its group, the owner's permissions override the group's permissions. If the owner has read permission (r) but no write or execute permissions, they cannot write or execute the file even if the group has full permissions (rwx). You can add users to a group with the usermod or gpasswd command to grant them specific group permissions.

String Representation of Linux Permissions

Files and directories use a 10-character format to represent permissions, visible by running a command such as ls -l:

The first character represents the file type:
- -: Regular file.
- d: Directory.
- l: Symbolic link.
- b: Block device or special file, such as disks and partitions.
- c: Character device or special file, such as terminals and serial ports.
- p: Named pipe.
- s: Socket.
- D: Door.
The next nine characters represent owner, group, and others permissions in order (three characters each).

For example:

drwxrw-r-- example.com

d: Specifies that example.com is a directory.
rwx: The owner has read, write, and execute permissions.
rw-: The group has read and write permissions.
r--: Other users have read-only permissions.

How to Calculate Linux Permissions

Linux permissions use binary values to represent access levels:

Permission	Binary	Value
Read (r)	100	4
Write (w)	010	2
Execute (x)	001	1
None (-)	000	0

For example:

Read + Write: 4 + 2 = 6 (binary: 110, symbol: rw-)
Read + Execute: 4 + 1 = 5 (binary: 101, symbol: r-x)
Write + Execute: 2 + 1 = 3 (binary: 011, symbol: -wx)
Read + Write + Execute: 4 + 2 + 1 = 7 (binary: 111, symbol rwx)

Common Linux Permissions

644 — owner can read and write; group and others can only read:

Permission Type	Read (r)	Write (w)	Total Value
Owner (u)	4	2	6
Group (g)	4	0	4
Others (o)	4	0	4

755 — owner has full permissions; group and others have read and execute:

Permission Type	Read (r)	Write (w)	Execute (x)	Total Value
Owner (u)	4	2	1	7
Group (g)	4	0	1	5
Others (o)	4	0	1	5

777 — owner, group, and others all have full permissions:

Permission Type	Read (r)	Write (w)	Execute (x)	Total Value
Owner (u)	4	2	1	7
Group (g)	4	2	1	7
Others (o)	4	2	1	7

The root user can override all Linux user permissions on a file or directory. Using sudo while reading or executing a file enables root superuser privileges, overriding any existing permissions.

View File and Directory Permissions

1. Create a new sample file and directory:

$ touch file.txt && sudo mkdir /var/www/html

2. View the permissions on the file with ls -l:

$ ls -l file.txt

Your output should be similar to:

-rwxr-xr-x 1 user group 1024 Jan 1 12:00 file.txt

-: Shows that file.txt is a regular file.
rwx: The owner's permissions.
r-x: The group permissions.
r-x: Other users' permissions on the file.

3. View the directory permissions with ls -ld:

$ ls -ld /var/www/html

Your output should be similar to:

drwxr-xr-x 2 user group 4096 Jan 1 12:30 /var/www/html

d: Shows that /var/www/html is a directory.
rwx: The directory owner's permissions.
r-x: The group permissions.
r-x: Other users' permissions on the directory.

Change File and Directory Permissions

You can change permissions using the chmod command in two modes: numeric and symbolic.

$ chmod [permissions] [file or directory]

Change Permissions Using Numeric Mode

Assign permissions using three digits representing the owner, group, and others. Each digit is the sum of read (4), write (2), and execute (1):

Read (r): 4
Write (w): 2
Execute (x): 1
No permissions (-): 0

1. Change file.txt permissions to 755:

$ chmod 755 file.txt

This allows all users to read and execute the file, but only the owner to modify it.

2. Change the /var/www/html directory permissions to 755:

$ sudo chmod 755 /var/www/html

This allows all users to view the directory contents, but only the owner can add new files.

3. Recursively apply permissions to all files and subdirectories using the -R option:

$ sudo chmod -R 755 /var/www/html

This enables 755 permissions for all files and subdirectories in /var/www/html, allowing all users to navigate and read/execute files, but only the owner can modify contents.

Change Permissions Using Symbolic Mode

Symbolic mode uses letters for user classes:

u: File or directory owner.
g: The group associated with a file or directory.
o: Other system users who are neither the owner nor part of the group.
a: All users, including owner, group, and others.

And operators to assign, remove, or set permissions:

+: Adds a permission.
-: Removes a permission.
=: Explicitly sets the permission and overwrites the active permissions.

$ chmod [userclass][operator][permissions] [filename or directory]

1. Grant the owner read, write, and execute permissions on file.txt:

$ chmod u+rwx file.txt

2. Remove execute permission from the group on the /var/www/html directory:

$ sudo chmod g-x /var/www/html

3. Enable read permissions for all users on file.txt:

$ chmod a+r file.txt

4. Change the file owner's permission to read-only, removing write and execute:

$ chmod u=r file.txt

5. Apply permissions recursively on the /var/www/html directory with -R:

$ sudo chmod -R a+rx /var/www/html

This adds read and execute permissions for all users on /var/www/html and its contents.

6. Modify multiple user categories in a single command:

$ chmod ug+rw file.txt

This adds read and write permissions to both the owner and group on file.txt.

7. Modify different permissions for multiple user classes in a single command:

$ chmod u+rwx,g+w,o+x file.txt

This enables read, write, and execute for the owner, write for the group, and execute for other users on file.txt.

Set Special Permissions (Sticky Bit, SUID, SGID)

Sticky bit, Set User ID (SUID), and Set Group ID (SGID) are special permissions that let administrators control file and directory access in multi-user environments.

Sticky Bit

Sticky bit applies only to directories and ensures that only the owner can rename, delete, or move files in a directory.

1. Use the +t option to enable sticky bit on the /var/www/html directory:

$ sudo chmod +t /var/www/html

2. Or use the 1 numeric prefix to enable sticky bit while setting permissions, e.g. 755 with sticky bit:

$ sudo chmod 1755 /var/www/html

1: Enables the sticky bit.
7: Grants read, write, and execute permissions to the owner.
5: Grants read and execute permissions to the group.
5: Grants read and execute permissions to other users.

3. List the directory permissions to verify the sticky bit change:

$ ls -ld /var/www/html

Your output should be similar to:

drwxr-xr-t 2 user group 4096 Jan 1 12:00 /var/www/html

Sticky bit is active based on the t in the above output.

Set User ID (SUID)

SUID enables the execution of a file with the privileges of its owner instead of the active user running it — useful for executable files that may require elevated privileges.

1. Create a sample hello.sh script:

$ touch hello.sh

2. Use u+s with chmod to enable the SUID bit on hello.sh:

$ chmod u+s hello.sh

This allows all users to execute hello.sh with the permissions of the file owner.

3. Or use numeric mode by adding 4 as the leading digit:

$ chmod 4755 hello.sh

4: Enables the SUID bit.
7: Grants read, write, and execute permissions to the owner.
5: Grants read and execute permissions to the group.
5: Grants read and execute permissions to other users.

4. View the hello.sh permissions to verify the SUID bit is active:

$ ls -l hello.sh

Your output should be similar to:

-rwsr-xr-x 1 root root 1024 Jan  15 20:43 hello.sh

The SUID bit is active based on the s value in the owner permissions (rws).

Set Group ID (SGID)

SGID applies to executable files and directories and enables file execution using the group's permissions instead of the user's.

1. Enable SGID on the hello.sh script:

$ sudo chmod g+s hello.sh

2. Enable SGID on the /var/www/html directory:

$ sudo chmod g+s /var/www/html

3. Use the 2 numeric mode to enable SGID while setting 755 permissions on hello.sh:

$ sudo chmod 2755 hello.sh

2: Enables the SGID bit.
7: Grants read, write, and execute permissions to the owner.
5: Grants read and execute permissions to the group.
5: Grants read and execute permissions to other users.

4. View the hello.sh permissions and verify SGID is active:

$ ls -l hello.sh

Your output should be similar to:

-rwxr-sr-x 1 user group 123456 Jan  1 12:00 hello.sh

SGID is active based on the s in the group permissions (-rwxr-sr-x).

5. List the /var/www/html permissions and verify SGID is active:

$ ls -ld /var/www/html

Your output should be similar to:

drwxr-sr-x 2 user group 4096 Jan 1 12:00 /var/www/html

SGID is active based on the s in the group's permissions (drwxr-sr-x).

Change User and Group Ownership Permissions

You can change ownership using the chown command:

$ chown [options] user:group [filename/directory]

1. Grant linuxuser ownership of file.txt:

$ sudo chown linuxuser file.txt

This sets linuxuser as the owner while keeping group ownership unchanged.

2. Grant the www-data group ownership of the /var/www/html directory:

$ sudo chown :www-data /var/www/html

This sets www-data as the group owner while file ownership permissions remain unchanged.

3. Change both owner and group by separating them with ::

$ sudo chown www-data:www-data /var/www/html

4. Recursively change owner and group for all files and subdirectories with -R:

$ sudo chown -R www-data:www-data /var/www/html

5. Use --reference to copy the ownership of one file to another:

$ sudo chown --reference=/var/www/html file.txt

6. View the owner and group permissions to verify the changes:

$ ls -l file.txt && ls -ld /var/www/html

Your output should be similar to:

-rwxr-xr-- 1 www-data www-data 1024 Jan 1 12:00 file.txt
drwxr-xr-x 2 www-data www-data 4096 Jan 1 12:30 /var/www/html

Next Steps

Audit sensitive directories (like web roots and config folders) for overly permissive 777 settings
Combine SGID with shared group directories to keep team-owned files consistently group-writable
Script routine permission and ownership checks with find and chmod/chown for consistency across servers
Run man chmod and man chown for the full set of options on your system

For the full guide with additional tips, visit the original article on Vultr Docs.

Managing Node Applications with PM2

Sanskriti Harmukh — Wed, 14 Oct 2026 21:05:10 +0000

Node.js is an open-source JavaScript runtime built on Chrome's V8 engine, known for its speed and scalability in real-time and high-performance applications. Running Node applications in production comes with unique challenges — you need to handle unexpected crashes, manage processes, and ensure continuous uptime. PM2 is a production process manager for Node.js applications that simplifies this, letting you monitor, restart, and optimize your apps. This guide walks through installing PM2, running a sample Node.js app under it, and configuring PM2 for production use. By the end, you'll have a Node.js application managed by PM2 with automatic restarts, centralized logs, and startup persistence configured.

Prerequisites

Access to your workstation as a non-root sudo user, running macOS, Linux, or Windows, with Node and npm installed.

Install PM2 Using npm

1. Install PM2 via the npm package manager:

$ sudo npm install pm2 -g

Create an Example Node.js Application

1. Navigate to your home directory:

$ cd ~

2. Create a file named app.js using a text editor such as vim:

$ vim app.js

3. Add the following code to the file:

const http = require('http');

const hostname = '0.0.0.0';
const port = 3000;

const server = http.createServer((req, res) => {
    res.statusCode = 200;
    res.setHeader('Content-Type', 'text/plain');
    res.end('Hello World');
});

server.listen(port, hostname, () => {
    console.log(`Server running at http://${hostname}:${port}/`);
});

Save and exit the file. This example app listens on port 3000 and responds to HTTP requests with the message "Hello World".

Start the Application Using PM2

1. Start your Node.js application with PM2 (replace app.js with your entry file if needed):

$ pm2 start app.js

Your output should look similar to:

[PM2] Starting /home/linuxuser/app.js in fork_mode (1 instance)
[PM2] Done.
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name               │ mode     │ ↺    │ status    │ cpu      │ memory   │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 0  │ app                │ fork     │ 0    │ online    │ 0%       │ 4.4mb    │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘

Useful additional parameters when starting an app:

--name <app_name> - specify an application name
--watch - restart your application automatically when a file changes
--max-memory-restart <memory> - a memory limit that triggers an automatic restart when reached
--log <log_file> - specify a log file
--restart-delay <delay> - delay between automatic restarts in milliseconds
--time - prefix logs with time
--no-autorestart - turn off automatic application restarts
--cron - a cron job pattern for automatic application restarts
--no-daemon - do not run the application in a daemon process

Visit the official documentation to view all available parameters.

Manage the Application

1. Restart your PM2-managed application:

$ pm2 restart app

2. Reload your PM2-managed application:

$ pm2 reload app

3. Stop your PM2-managed application:

$ pm2 stop app

4. Delete your PM2-managed application:

$ pm2 delete app

List All Applications

1. List all the applications managed by PM2:

$ pm2 list

Display Logs

1. View the logs produced by your application in real time:

$ pm2 logs

2. View older logs by specifying the number of lines with the --lines flag:

$ pm2 logs --lines 200

Monitor Resource Usage

1. Monitor the resources consumed by your applications in near real time:

$ pm2 monit

Set Up a Web-Based Dashboard

1. Configure a web-based PM2 dashboard:

$ pm2 plus

Follow the prompts to create or log in to a PM2 account. If you don't have one, complete the required information and accept the license agreement. You'll see output during setup similar to:

[PM2 I/O] Do you have a pm2.io account? (y/n) n
[PM2 I/O] No problem! We just need a few details to create your account.
[PM2 I/O] Please choose a username: exampleuser
[PM2 I/O] Please choose an email: admin@example.com
[PM2 I/O] Please choose a password: ************
...
[PM2 I/O] Do you accept the terms and privacy policy (https://pm2.io/legals/terms_conditions.pdf)? (y/n) y
[PM2 I/O] Successfully authenticated
[PM2 I/O] Successfully validated
[+] PM2+ activated!
[PM2 I/O] Successfully connected to bucket PM2 Plus Monitoring
[PM2 I/O] You can use the web interface here: https://app.pm2.io/#/bucket/67cb27ccd623d57f6d4c63c0

Replace exampleuser with your preferred username, admin@example.com with your login email address, and provide a strong password to secure your PM2 account. Then open the provided web address, log in with your PM2 account credentials, and start monitoring your applications through the PM2 Plus dashboard.

PM2 Plus is a subscription-based offering that incurs a cost depending on the number of processes you want to monitor. Review the pricing details on the PM2 Plus page.

Configure PM2 to Start on Server Boot

1. Generate a startup script and configure PM2 to start on server boot:

$ pm2 startup

Your output should look similar to:

[PM2] To setup the Startup Script, copy/paste the following command:
sudo env PATH=$PATH:/home/linuxuser/.nvm/versions/node/v22.14.0/bin /home/linuxuser/.nvm/versions/node/v22.14.0/lib/node_modules/pm2/bin/pm2 startup systemd -u linuxuser --hp /home/linuxuser

2. Copy the command from the previous output and run it to make the PM2 process persistent:

$ sudo env PATH=$PATH:/home/linuxuser/.nvm/versions/node/v22.14.0/bin /home/linuxuser/.nvm/versions/node/v22.14.0/lib/node_modules/pm2/bin/pm2 startup systemd -u linuxuser --hp /home/linuxuser

Your output should look similar to:

...
[PM2] Writing init configuration in /etc/systemd/system/pm2-linuxuser.service
Created symlink /etc/systemd/system/multi-user.target.wants/pm2-linuxuser.service → /etc/systemd/system/pm2-linuxuser.service.
...

Configure Applications to Start on Server Boot

1. Save the running applications to the list:

$ pm2 save

Your output should look similar to:

[PM2] Saving current process list...
[PM2] Successfully saved in /home/linuxuser/.pm2/dump.pm2

2. Restore the saved applications manually from the list if needed:

$ pm2 resurrect

Your output should look similar to:

[PM2] Resurrecting
[PM2] Restoring processes located in /home/linuxuser/.pm2/dump.pm2
[PM2] Process /home/linuxuser/app.js restored
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name               │ mode     │ ↺    │ status    │ cpu      │ memory   │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 0  │ app                │ fork     │ 0    │ online    │ 0%       │ 21.3mb   │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘

Check for Updates

1. Print a list of outdated npm packages on your system:

$ npm outdated -g

This displays a list of globally installed npm packages that have newer versions available.

Update PM2

1. Update the PM2 package via npm (this stops all running PM2 processes until the upgrade completes, causing downtime):

$ npm install pm2@latest -g

2. Perform an in-memory update of PM2:

$ pm2 update

Best Practices and Security Considerations

Prevent Downtime With Cluster Mode

Your PM2 processes may restart multiple times during their lifecycle, causing temporary downtime. Cluster mode runs multiple instances of your application, and when a restart is required, only one instance stops and restarts at a time while the others keep serving traffic.

1. Delete the application before switching to cluster mode:

$ pm2 delete app

2. Start the application in cluster mode, specifying the number of instances with the -i parameter:

$ pm2 start app.js -i 3

This starts three instances of your application in cluster mode using PM2.

3. Scale the application by specifying the new number of instances:

$ pm2 scale app 4

This scales your app application up to 4 instances.

Eliminate Failed Requests With Graceful Shutdown

PM2 ensures graceful shutdowns by sending signals to your application so it can terminate cleanly, helping prevent data loss or corruption by allowing your app to finish active tasks before exiting:

Before shutdown, PM2 sends a SIGINT signal to your application.
Your application intercepts the signal.
It finishes active requests and closes open resources like database connections.
The application exits.

1. Add the following code to enable graceful shutdown in your application:

process.on('SIGINT', function() {
    // sigint signal received, log a message to the console
    console.log('Shutdown signal intercepted');
    // close the http server
    server.close();
    // exit the process
    process.exit();
});

This intercepts the SIGINT signal, logs a message to the console, closes the HTTP server, and exits the process cleanly.

Next Steps

Enable cluster mode in production to eliminate downtime during restarts
Set up PM2 Plus (or another monitoring tool) for centralized visibility across multiple servers
Add graceful shutdown handling to any application that holds open connections or resources
Explore PM2's ecosystem file (pm2.config.js) to manage multiple apps and environments from one file

For the full guide with additional tips, visit the original article on Vultr Docs.

Viewing and Managing System Logs Using rsyslog in Linux

Sanskriti Harmukh — Wed, 14 Oct 2026 21:04:30 +0000

Linux systems generate a continuous stream of logs from the kernel, services, and user applications, and managing them is essential for diagnosing issues, tracking activity, and maintaining operational visibility. rsyslog is a widely adopted logging daemon that collects, processes, stores, and optionally forwards these logs, supporting multiple input sources, advanced filtering, rule-based message routing, and remote logging capabilities beyond what basic syslog tools offer. This guide uses rsyslog to view and manage system logs, define custom rules, and forward logs to remote servers for centralized logging. By the end, you'll have custom rsyslog rules routing SSH and application logs to their own files, plus a client forwarding logs to a centralized remote server.

How `rsyslog` Works: The Input, Filter, and Output Model

rsyslog processes log messages through a structured pipeline composed of three core stages:

Input: rsyslog collects logs from various sources such as system sockets (imuxsock), kernel messages (imklog), plain-text files (imfile), or network protocols like TCP/UDP (imtcp, imudp). Each input module defines how and where logs enter the system.
Filter: After logs are received, rsyslog evaluates them using filters based on facility, severity, or custom expression rules, which determines which logs are processed further and how they're handled.
Output: Filtered logs are routed to output destinations such as local log files, SQL databases, remote syslog servers, or message queues, each managed by a specific module with custom formatting and delivery rules.

Common Log File Locations in Linux

Log files are typically stored under /var/log/ and record system events, application activity, kernel operations, and security information. File names and purposes can vary slightly across distributions, but the following are among the most common:

/var/log/syslog: Captures general system activity, including messages from system services and applications. Commonly used on Debian and Ubuntu-based systems for broad troubleshooting.
/var/log/messages: A general-purpose log file used on Red Hat-based systems (like CentOS and Fedora), logging system messages not related to authentication or the kernel.
/var/log/auth.log: Records authentication attempts, including user logins, sudo actions, and SSH access. Essential for auditing and security monitoring.
/var/log/kern.log: Contains kernel-generated messages not handled by other facilities. Useful for diagnosing hardware issues or kernel module problems.
/var/log/dmesg: Stores the kernel ring buffer from the most recent boot, showing hardware detection and initialization messages.
/var/log/cron.log: Logs the execution of scheduled cron jobs, useful when debugging missed or failed automated tasks.

View Logs with Basic Linux Commands

Before configuring rsyslog, it's useful to know how to inspect log files with common Linux utilities.

cat: Outputs the entire contents of a file to the terminal. Best used for small log files.
```
$ cat <LOGFILE_PATH>
```
more: Displays the contents of a file one screen at a time. Use Space to scroll forward, q to quit.
```
$ more <LOGFILE_PATH>
```
less: Opens the file in a scrollable viewer, letting you navigate forward and backward through large log files.
```
$ less <LOGFILE_PATH>
```
tail: Displays the last 10 lines of a file by default. Useful for checking recent log entries.
```
$ tail <LOGFILE_PATH>
```
tail -f: Continuously streams new log entries as they're written, ideal for real-time monitoring.
```
$ tail -f <LOGFILE_PATH>
```
zcat, zless, zgrep: Read or search compressed .gz log files created by log rotation without decompressing them first.
```
$ zless <GZ_LOGFILE_PATH>
```

Filter Logs Using `grep`, `awk`, and `sed`

grep: Searches for specific keywords or regular expressions within log files, useful for locating errors, warnings, or other patterns.
```
$ grep "error" /var/log/syslog
$ grep -i "failed" /var/log/auth.log
```
awk: Processes log entries by splitting them into fields, useful for extracting timestamps, usernames, or specific columns from structured logs.
```
$ awk '{print $1, $2, $5}' /var/log/syslog
```
sed: A stream editor that filters or transforms lines of text, useful for extracting a block of logs between two keywords or substituting text.
```
$ sed -n '/Start/,/End/p' /var/log/syslog
$ sed 's/error/ERROR/g' /var/log/syslog
```

Configure `rsyslog` Using `/etc/rsyslog.conf`

The behavior of rsyslog is primarily controlled through its main configuration file at /etc/rsyslog.conf, which defines how incoming log messages are processed, filtered, and directed based on their source (facility), importance (severity level), and destination (action).

The general format for a rule is:

FACILITY.LEVEL    ACTION

FACILITY: The source of the log message (for example, auth, cron, daemon, kern, mail, user).
LEVEL: The severity of the log message (for example, debug, info, notice, warning, err, crit, alert, emerg).
ACTION: The destination for the log (for example, a file, remote server, or program).

Common Configuration Examples

Send auth logs to a separate file:
```
auth.*    /var/log/auth.log
```
Logs from the auth facility (for example, SSH, sudo) go to /var/log/auth.log.
Store kernel logs in a separate file:
```
kern.*    /var/log/kern.log
```
All kernel messages are redirected to /var/log/kern.log.
Log only critical mail errors:
```
mail.crit    /var/log/mail_critical.log
```
Only critical-level messages from the mail facility are logged.
Log everything except mail and auth to a common file:
```
*.info;mail.none;authpriv.none    /var/log/general.log
```
Sends all info-level or higher messages to /var/log/general.log, except those from mail and authpriv.
Discard unwanted logs:
```
*.debug    ~
```
The tilde (~) discards all debug-level logs to reduce noise.
Log user messages to the console:
```
user.*    /dev/console
```

Add Custom Logging Rules

Log SSH Activity

1. Create a new configuration file in /etc/rsyslog.d/:

$ sudo nano /etc/rsyslog.d/10-ssh.conf

2. Add the following rule to capture all messages from the authpriv facility:

authpriv.*    /var/log/ssh.log

3. Restart the rsyslog service to apply the new configuration:

$ systemctl restart rsyslog.service

SSH-related log entries are now written to /var/log/ssh.log, making it easier to monitor authentication activity independently of other system logs.

Log Activity for a Custom Application

1. Create a new rsyslog configuration file in /etc/rsyslog.d/:

$ sudo nano /etc/rsyslog.d/20-myapp.conf

2. Add the following rule to match logs from your application:

if ($programname == 'myapp') then /var/log/myapp.log
& stop

3. Restart the rsyslog service to apply the changes:

$ systemctl restart rsyslog.service

All syslog messages from myapp are now written to /var/log/myapp.log, keeping application logs organized and separate from system logs.

Forward Logs to a Remote Server

Centralizing logs from multiple systems on a single server helps streamline monitoring and troubleshooting. rsyslog can forward logs from a client to a remote server over TCP or UDP.

Configure the Client and Server

1. Create a configuration file on the client to forward all logs to the remote server:

$ sudo nano /etc/rsyslog.d/60-remote.conf

2. Add the forwarding rule to the file:

*.* @@192.2.0.3:514

Replace 192.2.0.3 with your remote rsyslog server's IP address. Use a single @ for UDP, or double @@ for TCP.

3. On the remote rsyslog server, enable TCP log reception by editing the main config file:

$ sudo nano /etc/rsyslog.conf

4. Uncomment or add the following lines to enable TCP input:

module(load="imtcp")
input(type="imtcp" port="514")

5. (Optional) Organize incoming logs by hostname and program name on the server:

$template RemoteLogs,"/var/log/remote/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?RemoteLogs

This stores logs in structured folders like /var/log/remote/client1/myapp.log.

6. Restart rsyslog on both systems to apply the changes:

$ sudo systemctl restart rsyslog

After setup, the client system sends logs to the remote server, which organizes and stores them based on the defined templates.

Test the Configuration

1. Send a test log message:

$ logger -p local0.info "Test message from rsyslog"

Then check the appropriate log file (for example, /var/log/syslog or your custom-defined log path) to confirm that the message was recorded.

Best Practices for Using `rsyslog`

Use separate files: For better management, store logs from different services or applications in separate files.
Secure your logs: Protect log files with proper file permissions to prevent unauthorized access.
Use log rotation: Set up log rotation to avoid log files growing too large.
Centralized logging: Use remote logging to collect logs from multiple systems in one place.
Monitor your logs: Check log files often to identify and address issues promptly.

Next Steps

Set up logrotate alongside rsyslog to automatically archive and compress older log files.
Ship logs from the remote rsyslog server to a centralized SIEM or log aggregation platform.
Add facility-specific rules for other services you run, following the same pattern used for SSH and custom applications.
Automate alerting on critical log patterns using grep/awk pipelines or a log monitoring tool.

For the full guide with additional tips, visit the original article on Vultr Docs.

Managing Dependencies with Poetry for a Flask App on Ubuntu 24.04

Sanskriti Harmukh — Wed, 14 Oct 2026 21:03:57 +0000

Poetry is an open-source tool for managing and packaging dependencies for Python projects. It maintains a clean view of your dependency tree, streamlines the installation and management of dependencies, and simplifies packaging and distribution to PyPI. This guide installs Poetry with Pipx on Ubuntu 24.04, creates a virtual environment, and runs a Flask application using Poetry and Gunicorn. By the end, you'll have a Flask app managed entirely through Poetry, tested with Pytest, and served in production mode with Gunicorn.

Prerequisites

Before you begin, you need to:

Have access to an Ubuntu 24.04 server.

Install Poetry with Pipx

Installing Poetry in an isolated environment prevents conflicts between Poetry's internal dependencies and system-wide dependencies.

1. Update the APT package index:

$ sudo apt update

2. Install Pipx:

$ sudo apt install pipx -y

3. Add the Pipx libraries to your system's PATH:

$ pipx ensurepath

4. Enable tab completions for Pipx commands:

$ echo 'eval "$(register-python-argcomplete pipx)"' >> ~/.bashrc

5. Reload the .bashrc shell configuration to apply the changes in your active session:

$ source ~/.bashrc

6. Install Poetry using Pipx:

$ pipx install poetry

Your output should be similar to the one below:

installed package poetry 2.0.0, installed using Python 3.12.3
  These apps are now globally available
    - poetry
done! ✨ 🌟 ✨

7. View the installed Poetry version:

$ poetry --version

Your output should be similar to the one below:

Poetry (version 2.0.0)

Create a Poetry Project

1. Create a new flask-app Poetry project:

$ poetry new flask-app

This creates a flask-app directory with the following files:

flask_app: Contains an __init__.py file that specifies all Flask code, including modules, functions, and other related support components.
pyproject.toml: Includes information about the project, Poetry configuration, and installed dependencies.
README.md: Includes basic documentation about the project.
tests: Integrates and organizes the project's test cases.

Your output should be similar to the one below:

Created package flask_app in flask-app

2. Switch to the flask-app project directory:

$ cd flask-app

Create a Virtual Environment and Install Flask

1. Specify the Python interpreter for Poetry and create a new virtual environment:

$ poetry env use python3

This instructs Poetry to use the default Python3 interpreter on your system. You can also target a specific version, such as poetry env use python3.11, if that version is installed.

Your output should be similar to the one below:

Creating virtualenv flask-app-vnQ3DaWl-py3.12 in /home/linuxuser/.cache/pypoetry/virtualenvs
Using virtualenv: /home/linuxuser/.cache/pypoetry/virtualenvs/flask-app-vnQ3DaWl-py3.12

2. Activate the virtual environment:

$ poetry env activate

This outputs the command to run in order to activate the virtual environment manually. Your output should be similar to the one below:

source /home/linuxuser/.cache/pypoetry/virtualenvs/flask-app-vnQ3DaWl-py3.12/bin/activate

3. Activate the virtual environment using its path:

$ source /home/linuxuser/.cache/pypoetry/virtualenvs/flask-app-vnQ3DaWl-py3.12/bin/activate

Your output should be similar to the one below when the virtual environment is activated:

(flask-app-py3.12) linuxuser@:~/flask-app$

4. Install Flask using Poetry:

$ poetry add flask

This installs Flask with all dependencies in the active virtual environment, adds the installed Flask version to pyproject.toml, and creates a poetry.lock file for deterministic builds.

Your output should be similar to the one below:

Using version ^3.1.0 for flask

Updating dependencies
Resolving dependencies... (0.2s)

Package operations: 7 installs, 0 updates, 0 removals

- Installing markupsafe (3.0.2)
- Installing blinker (1.9.0)
- Installing click (8.1.8)
- Installing itsdangerous (2.2.0)
- Installing jinja2 (3.1.5)
- Installing werkzeug (3.1.3)
- Installing flask (3.1.0)

Writing lock file

5. Deactivate the virtual environment:

$ deactivate

Set Up Firewall Rules

Uncomplicated Firewall (UFW) is active on most Ubuntu 24.04 servers by default. Configure UFW to allow connections to the Flask application port 5000.

1. Allow connections to the Flask application port 5000:

$ sudo ufw allow 5000/tcp

2. Reload UFW to apply the firewall configuration changes:

$ sudo ufw reload

3. View the UFW status and verify that the rule is active:

$ sudo ufw status

Your output should be similar to the one below:

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere                  
5000/tcp                   ALLOW       Anywhere                  
22/tcp (v6)                ALLOW       Anywhere (v6)             
5000/tcp (v6)              ALLOW       Anywhere (v6)

Run the Flask App with Poetry

1. Retrieve the command to reactivate the virtual environment:

$ poetry env activate

Your output should be similar to the one below:

source /home/linuxuser/.cache/pypoetry/virtualenvs/flask-app-vnQ3DaWl-py3.12/bin/activate

2. Activate the virtual environment:

$ source /home/linuxuser/.cache/pypoetry/virtualenvs/flask-app-vnQ3DaWl-py3.12/bin/activate

3. Navigate to the flask_app directory:

$ cd flask_app

4. Create a file app.py using nano:

$ nano app.py

5. Add the following code:

from flask import Flask

app = Flask(__name__)

@app.route('/')
def home():
    return "<h1>Hello from Flask</h1>"

if __name__ == '__main__':
   app.run(host='0.0.0.0', port='5000')

Save the file with Ctrl+X, then Y, then Enter. This creates a webpage that returns Hello from Flask on the home route /.

6. Run the Flask application:

$ poetry run python app.py

Your output should be similar to the one below:

* Serving Flask app 'app'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on all addresses (0.0.0.0)
* Running on http://127.0.0.1:5000
* Running on http://<SERVER-IP>:5000
Press CTRL+C to quit

7. Access your Flask app using the following address in a web browser. Replace <SERVER-IP> with your server's IP address:

http://<SERVER-IP>:5000

8. To stop the Flask app, press Ctrl+C in your terminal session.

Create Group Dependencies

Poetry allows you to manage dependencies using groups. Follow these steps to create a group called dev and install Pytest for testing.

1. Install Pytest in the dev dependency group:

$ poetry add pytest --group dev

This creates a new section in pyproject.toml — [tool.poetry.group.dev.dependencies] — which includes dependencies such as pytest = "^8.3.4" under the dev group. Production dependencies should not be added to any group; they belong under the root dependencies ([tool.poetry.dependencies]) instead, so they aren't excluded during deployment or packaging.

2. Navigate to your project's tests directory:

$ cd ~/flask-app/tests

3. Create a new directory named unit:

$ mkdir unit

4. Navigate into the unit directory:

$ cd unit

5. Create an __init__.py file:

$ touch __init__.py

This marks the directory as a Python package, allowing you to import modules from it.

6. Create a test file test_app.py using nano:

$ nano test_app.py

7. Add the following code to the file:

import pytest
from flask_app.app import app  

@pytest.fixture
def client():
    with app.test_client() as client:
        yield client

def test_the_home_route(client):
    response = client.get('/')
    assert response.status_code == 200
    assert b"<h1>Hello from Flask</h1>" in response.data

Save the file with Ctrl+X, then Y, then Enter. This checks that the response body from app.py contains the markup <h1>Hello from Flask</h1>.

8. Run the test:

$ pytest

Your output should be similar to the one below:

platform linux -- Python 3.12.3, pytest-8.3.4, pluggy-1.5.0
rootdir: /home/linuxuser/flask-app
configfile: pyproject.toml
collected 1 item                                                             

test_app.py .                                                          [100%]

============================= 1 passed in 0.11s ==============================

Install Gunicorn, a WSGI HTTP Server, with Flask

A WSGI HTTP server such as Gunicorn handles client requests concurrently and efficiently, making it ideal for production-grade Flask applications.

1. Navigate to the flask_app directory:

$ cd ~/flask-app/flask_app

2. Add Gunicorn as a root dependency:

$ poetry add gunicorn

3. Create a wsgi.py file using nano:

$ nano wsgi.py

4. Add the following Python code:

from app import app

if __name__ == '__main__':
   app.run()

5. Run the Flask app with Gunicorn:

$ gunicorn --bind 0.0.0.0:5000 wsgi:app

6. Check that your Flask app is still accessible:

http://<SERVER-IP>:5000

7. Press Ctrl+C to stop the server.

Manage the Virtual Environment

1. Exit the virtual environment:

$ deactivate

2. List all virtual environments for your project:

$ poetry env list

The environment labeled (Activated) is the one in use. Your output should be similar to the one below:

flask-app-vnQ3DaWl-py3.12 (Activated)

3. Copy the environment name from the previous command's output and remove the virtual environment:

$ poetry env remove flask-app-vnQ3DaWl-py3.12

Your output should be similar to the one below:

Deleted virtualenv: /home/linuxuser/.cache/pypoetry/virtualenvs/flask-app-vnQ3DaWl-py3.12

4. Recreate the virtual environment:

$ poetry env use python3

5. Retrieve the command to reactivate the virtual environment:

$ poetry env activate

Your output should be similar to the one below:

source /home/linuxuser/.cache/pypoetry/virtualenvs/flask-app-vnQ3DaWl-py3.12/bin/activate

6. Activate the virtual environment by copying and running the activation command from the previous step:

$ source /home/linuxuser/.cache/pypoetry/virtualenvs/flask-app-vnQ3DaWl-py3.12/bin/activate

7. Install all the dependencies from the pyproject.toml file:

$ poetry install

With the pyproject.toml file present, the virtual environment and dependencies can always be recreated and reinstalled.

Modify the Virtual Environment Location

By default, Poetry creates a virtual environment in /.cache/pypoetry/virtualenvs. To keep the virtual environment organized alongside your project files, you can move its location to the root of your project.

1. Exit the virtual environment:

$ deactivate

2. List the virtual environments for your project:

$ poetry env list

Your output should be similar to the one below:

flask-app-vnQ3DaWl-py3.12 (Activated)

3. Remove the virtual environment:

$ poetry env remove flask-app-vnQ3DaWl-py3.12

4. Configure Poetry to allow the virtual environment to be created at the root of your project:

$ poetry config virtualenvs.in-project true

This overwrites the default configuration set by Poetry and allows the virtual environment to be created in the root of your project.

5. Verify that the configuration change is applied:

$ poetry config virtualenvs.in-project

Your output should be similar to the one below:

true

6. Create a virtual environment to verify:

$ poetry env use python3

Your output should be similar to the one below:

Creating virtualenv flask-app in /home/linuxuser/flask-app/.venv
Using virtualenv: /home/linuxuser/flask-app/.venv

7. Retrieve the command to activate the virtual environment:

$ poetry env activate

Your output should be similar to the one below:

source /home/linuxuser/flask-app/.venv/bin/activate

8. Activate the virtual environment:

$ source /home/linuxuser/flask-app/.venv/bin/activate

9. View your virtual environment information:

$ poetry env info

Your output should be similar to the one below:

Virtualenv
Python:         3.12.3
Implementation: CPython
Path:           /home/linuxuser/flask-app/.venv
Executable:     /home/linuxuser/flask-app/.venv/bin/python
Valid:          True

Base
Platform:   linux
OS:         posix
Python:     3.12.3
Path:       /usr
Executable: /usr/bin/python3.12

10. Install all the dependencies in this new environment:

$ poetry install

11. Run the Flask app using Gunicorn to verify that everything is running in the new environment:

$ gunicorn --bind 0.0.0.0:5000 wsgi:app

12. Check that your Flask app is accessible:

http://<SERVER-IP>:5000

Next Steps

Put Gunicorn behind a reverse proxy such as Nginx and run it as a systemd service for production deployments.
Add more Poetry dependency groups (for example, lint or docs) to keep tooling separate from runtime dependencies.
Wire the Pytest suite into a CI pipeline so tests run automatically on every push.
Publish the package to PyPI or a private repository once it's ready to share.

For the full guide with additional tips, visit the original article on Vultr Docs.

Installing and Using nftables in Linux

Sanskriti Harmukh — Wed, 14 Oct 2026 21:03:13 +0000

nftables is a modern packet filtering framework for Linux that replaces iptables. It offers a unified and efficient way to manage firewall rules across multiple protocols like IPv4, IPv6, and ARP, with better performance and a simpler configuration through a single interface instead of multiple tables. This guide installs and configures nftables on Linux, sets up basic firewall rules, manages network traffic, and implements common security policies. By the end, you'll have a persistent nftables firewall with SSH, HTTP, and HTTPS access rules, NAT/port forwarding, and logging in place.

Prerequisites

Before you begin, you need to:

Have access to a Linux-based instance as a non-root sudo user.
Have a basic understanding of network protocols and firewall rule management.

`nftables` versus `iptables`

iptables is the traditional firewall management tool on Linux, but its limitations include a complex rule structure, performance inefficiencies, and the need for separate tools to handle different protocols (IPv4, IPv6, ARP, and so on). nftables was introduced as a replacement, offering improvements in functionality, performance, and usability:

Feature	`iptables`	`nftables`
Packet Filtering	Requires multiple tools to manage different protocols (iptables, ip6tables, arptables)	Uses a unified approach with named tables and chains
Rule Syntax	Forces users to write complex chain-based rules with multiple entries	Offers a modern, table-based syntax that simplifies rule writing
Performance	Processes rules sequentially, which can slow down large rule sets	Uses optimized rule evaluation, reducing overhead
Atomic Rule Changes	Requires flushing and reloading entire rule sets for modifications	Supports atomic rule updates, preventing disruption
Logging	Uses legacy logging targets (LOG and ULOG frameworks)	Provides built-in logging with advanced tracing options (`meta nftrace`, `log`)
Connection Tracking	Requires external `conntrack` modules	Integrates connection tracking directly into the system
Multi-protocol Support	Requires different commands for each protocol	Manages all protocols with a single command set
Resource Usage	Consumes more memory by duplicating rules	Manages memory efficiently with a unified ruleset
Scripting & Automation	Relies on shell scripting and manual rule management	Supports JSON-based rule definition and direct scripting integration
Compatibility	Runs on all distributions but follows a legacy architecture	Works natively on modern systems while supporting `iptables` conversion

Install `nftables` on Linux

Install nftables using the package manager for your Linux distribution.

On Ubuntu and Debian

1. Update the server's package index:

$ sudo apt update

2. Install nftables using the APT package manager:

$ sudo apt install nftables -y

On Rocky Linux and AlmaLinux

1. Update the DNF package index:

$ sudo dnf update

2. Install nftables:

$ sudo dnf install -y nftables

On Arch Linux

1. Install nftables using Pacman:

$ sudo pacman -S nftables

`nft` Command Syntax

nft [ -nNscaeSupyjtT ] [ -I directory ] [ -f filename | -i | cmd ... ]

-n: Show rule numbers in output.
-N: Print more detailed output for rules.
-s: Show statistics for rules.
-c: Show chain name for each rule.
-a: Print the rule with its address family and protocol.
-e: Expand rule details.
-S: Print the ruleset (configuration).
-u: Enable update mode for dynamic changes.
-p: Print the current ruleset in a parsable format.
-y: Show detailed object types.
-j: Output in JSON format.
-t: Show output in table format.
-T: Show tokenized output.
-I directory: Use the specified directory for configuration files.
-f filename: Load rules from a file.
-i: Interactive mode to modify rules directly.
cmd ...: Command arguments to manipulate the nftables rules.

Manage the `nftables` System Service

The nftables service manages your firewall rules and ensures persistence across system reboots.

1. Enable the nftables service to start at boot:

$ sudo systemctl enable nftables

2. Start the nftables service:

$ sudo systemctl start nftables

3. Verify the service status:

$ sudo systemctl status nftables

Your output should be similar to the one below:

● nftables.service - Netfilter Tables
     Loaded: loaded (/usr/lib/systemd/system/nftables.service; enabled; preset: disabled)
     Active: active (exited) since Mon 2025-03-31 17:52:35 UTC; 5s ago
       Docs: man:nft(8)
    Process: 1838 ExecStart=/sbin/nft -f /etc/sysconfig/nftables.conf (code=exited, status=0/SUCCESS)
   Main PID: 1838 (code=exited, status=0/SUCCESS)
        CPU: 11ms

Mar 31 17:52:35 nftables-03 systemd[1]: Starting Netfilter Tables...
Mar 31 17:52:35 nftables-03 systemd[1]: Finished Netfilter Tables.

Tables, Chains, and Policies in `nftables`

In nftables, rules are organized into tables, which contain chains that define how traffic is processed. Each chain is associated with a hook (input, output, forward) that determines when it is evaluated, and chains have policies that define the default action when no explicit rule matches a packet.

nftables supports multiple address families for handling different network protocols:

inet: Unified handling for both IPv4 and IPv6.
ip: IPv4-only filtering.
ip6: IPv6-only filtering.
arp: ARP packet filtering.
bridge: Ethernet bridge packet filtering.
netdev: Packet filtering at the network device level.

Each chain is linked to a specific traffic type: input controls packets destined for the local machine, output manages packets leaving the machine, and forward handles packets routed through the machine.

The default policy determines whether unmatched traffic is accepted, dropped, or rejected:

policy accept - Allows all traffic by default (useful for testing or initial configurations).
policy drop - Discards unmatched packets (most secure, commonly used in production environments).
policy reject - Sends a rejection notice to the source (informative but reveals the presence of the server, which might not be ideal for security).

Warning: Using accept as the default policy is not recommended for production environments, as it allows all traffic unless explicitly blocked.

Manage the Default `nftables` Configuration

1. View the list of existing nftables rules:

$ sudo nft list tables

Your output should be similar to the one below:

table ip filter
table ip6 filter

2. View the default nftables configuration file:

$ cat /etc/nftables.conf

Your output should be similar to the one below:

#!/usr/sbin/nft -f

flush ruleset

table inet filter {
    chain input {
        type filter hook input priority 0; policy accept;
    }
    chain forward {
        type filter hook forward priority 0; policy accept;
    }
    chain output {
        type filter hook output priority 0; policy accept;
    }
}

Configure Basic `nftables` Firewall Rules

Create Initial Table and Chains

1. Create a new firewall table for IPv4 and IPv6 traffic:

$ sudo nft add table inet my_table

This creates a new table named my_table that can process both IPv4 and IPv6 packets using the inet family. Replace my_table with your preferred table name; it's used throughout the rest of this guide.

2. Add an input chain with a default accept policy:

$ sudo nft add chain inet my_table input { type filter hook input priority 0 \; }

This creates a filtering chain that applies to incoming traffic. type filter designates it as a filtering chain, hook input ensures it processes incoming packets, and priority 0 determines the execution order (lower values run first).

3. Create a forward chain with a default accept policy:

$ sudo nft add chain inet my_table forward { type filter hook forward priority 0\; }

This creates a filtering chain that applies to forwarded traffic, using hook forward to process forwarded packets.

4. Allow established connections to keep existing communications:

$ sudo nft add rule inet my_table input ct state established,related accept

This rule uses connection tracking: ct state enables connection tracking, and established allows packets from existing connections.

Add, List, and Delete Rules

1. Add a rule to block incoming traffic from a specific IP address:

$ sudo nft add rule inet my_table input ip saddr 192.0.2.10 drop

This adds a rule to the my_table table's input chain, blocking incoming traffic from the IP address 192.0.2.10.

2. View all active nftables rules to verify configurations:

$ sudo nft list ruleset

3. List the rules with handle numbers:

$ sudo nft list chain inet my_table input

Your output should be similar to the one below:

table inet my_table {
        chain input {
                type filter hook input priority filter; policy accept;
                ct state established,related accept
                ip saddr 192.0.2.10 drop
        }
}

4. Identify the handle number from the output and delete the corresponding rule:

$ sudo nft delete rule inet my_table input handle 0

This deletes the rule with handle number 0 from the input chain of the my_table table.

Implement Essential Security Rules

1. Allow SSH access to your server:

$ sudo nft add rule inet my_table input tcp dport 22 accept

This matches TCP packets targeting destination port 22 (the default SSH port) and accepts them.

2. Block a specific IP address that shows suspicious activity:

$ sudo nft add rule inet my_table input ip saddr 192.0.2.10 drop

This matches packets from the specified source IP address and drops them without notifying the sender.

3. Allow HTTP traffic on port 80:

$ sudo nft add rule inet my_table input tcp dport 80 accept

4. Allow HTTPS traffic on port 443:

$ sudo nft add rule inet my_table input tcp dport 443 accept

5. Drop all other incoming traffic:

$ sudo nft add rule inet my_table input ip daddr 0.0.0.0/0 counter drop

This drops all incoming traffic by default while allowing only the preconfigured rules (SSH, HTTP, HTTPS), effectively blocking unwanted access.

Enable NAT and Port Forwarding

1. Enable NAT functionality in the Linux kernel by editing /etc/sysctl.conf:

$ sudo vim /etc/sysctl.conf

2. Uncomment the following line:

...
# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1
....

Save and close the file.

3. Apply the changes:

$ sudo sysctl -p

4. Create a new NAT table to handle network address translation for your server:

$ sudo nft add table nat

5. Create a postrouting chain in the NAT table to process packets after routing:

$ sudo nft add chain nat postrouting { type nat hook postrouting priority 100 \; }

6. Add a masquerading rule for your internal network to enable internet connection sharing:

$ sudo nft add rule nat postrouting ip saddr 192.0.2.0/24 oif enp1s0 masquerade

This tells nftables to perform NAT (masquerading) for outgoing traffic from the 192.0.2.0/24 network. The oif enp1s0 part specifies that the rule applies to traffic leaving the enp1s0 interface, typically the interface connected to the internet.

Save Rules Persistently Across Reboots

1. Save your current firewall configuration to the /etc/nftables.conf file:

$ sudo nft list ruleset | sudo tee  /etc/nftables.conf

2. Apply saved rules from the configuration file:

$ sudo nft -f /etc/nftables.conf

3. Restart the nftables service:

$ sudo systemctl restart nftables

This reloads the firewall rules and ensures they persist after reboots.

Advanced Rule Configuration

1. Implement rate limiting for SSH connections:

$ sudo nft add rule inet my_table input tcp dport 22 limit rate 3/minute accept

This targets SSH traffic (port 22), limits connections to 3 per minute, and accepts connections within the limit to help prevent brute force attacks.

2. Create the prerouting chain in the nat table (if it doesn't exist):

$ sudo nft add chain ip nat prerouting { type nat hook prerouting priority 0 \; }

3. Add the port forwarding rule:

$ sudo nft add rule nat prerouting tcp dport 80 dnat to 192.0.2.10

This captures incoming HTTP traffic (port 80) and forwards it to an internal server (192.0.2.10), enabling you to host internal web services.

4. Enable logging for dropped packets:

$ sudo nft 'add rule inet my_table input log prefix "nftables-drop: " drop'

This logs packets before dropping them, with a custom prefix for easy filtering, and stores the logs in the system journal.

5. Flush all the nftables rules:

$ sudo nft flush ruleset

Warning: Be cautious when flushing the ruleset, as this removes all existing nftables rules, effectively resetting your firewall configuration.

Next Steps

Combine nftables sets and maps to manage large groups of IP addresses or ports more efficiently.
Script your nftables ruleset into a version-controlled configuration file for repeatable deployments.
Pair nftables logging with a centralized log aggregator to monitor dropped and rejected traffic.
Explore IPv6-specific rules using the ip6 address family for dual-stack environments.

For the full guide with additional tips, visit the original article on Vultr Docs.

Installing Docker with ROCm GPU Support on Ubuntu 24.04

Sanskriti Harmukh — Wed, 14 Oct 2026 21:02:19 +0000

Docker is a platform that packages applications and their dependencies into isolated containers so they run consistently on any system. GPU-enabled containers extend this by giving containerized applications access to a machine's Graphics Processing Unit (GPU) for compute-heavy workloads such as machine learning or video processing. This guide installs Docker from Docker's official repository on a ROCm-supported Ubuntu 24.04 system and accesses ROCm devices from inside a Docker container, assuming a GPU-enabled server with an AMD ROCm-capable GPU already set up. By the end, you'll have Docker installed and a temporary container running with verified access to the host's ROCm GPU devices.

Install Docker

1. Add Docker's official GPG key to the system:

$ sudo apt-get update
$ sudo apt-get install ca-certificates curl
$ sudo install -m 0755 -d /etc/apt/keyrings
$ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
$ sudo chmod a+r /etc/apt/keyrings/docker.asc

These commands ensure that packages downloaded from Docker's repository are authentic and secure, which is needed before adding Docker's repository and installing the Docker engine.

2. Add Docker's official repository to the system's APT sources list:

$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
    $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
    sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$ sudo apt-get update

3. Install Docker and its components:

$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

This installs Docker Engine, the Docker CLI, Containerd, the Buildx plugin, and the Docker Compose plugin.

Understand the ROCm Devices Exposed to Containers

Two host devices need to be exposed to Docker containers for GPU-accelerated workloads:

/dev/kfd (Kernel Fusion Driver): Manages communication between user applications and AMD GPUs for compute tasks. It also handles GPU memory, scheduling, and execution of computational tasks like deep learning or scientific computing.
/dev/dri (Direct Rendering Infrastructure): Provides access to the GPU for graphics rendering, and is also used for 2D/3D rendering and video processing tasks.

Both devices are required for exposing GPU capabilities to Docker containers for GPU-accelerated tasks such as machine learning and rendering. Learn more.

Access ROCm Devices from a Container

1. Pull a lightweight ROCm terminal container image:

$ docker pull rocm/rocm-terminal:latest

2. Run a temporary container with access to the GPU devices:

$ docker run --rm -it --device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined rocm/rocm-terminal

This command runs a temporary container with access to the necessary GPU devices (/dev/kfd and /dev/dri) on the host system.

3. Verify GPU availability from inside the container:

$ rocm-smi

The output should display all the detected GPU devices along with their specifications.

4. Exit and destroy the container:

$ exit

Next Steps

Install a machine learning framework such as PyTorch or TensorFlow with ROCm acceleration inside a container.
Build a custom Dockerfile that layers your application on top of a ROCm base image.
Orchestrate multiple ROCm-enabled containers with a container orchestrator such as Kubernetes.
Monitor GPU utilization with rocm-smi while running production workloads.

For the full guide with additional tips, visit the original article on Vultr Docs.

Installing K3s with ROCm GPU Operator on Ubuntu 24.04

Sanskriti Harmukh — Wed, 14 Oct 2026 21:01:22 +0000

K3s is a lightweight, certified Kubernetes distribution designed for resource-constrained environments such as edge computing and IoT devices. It simplifies deploying and managing Kubernetes clusters while staying compatible with standard Kubernetes tools and APIs. Paired with the ROCm GPU Operator, K3s can efficiently manage AMD GPUs within the cluster, automating GPU resource management and monitoring so you can deploy and scale GPU-accelerated workloads without manual configuration. This guide assumes a GPU-enabled Ubuntu 24.04 server with an AMD ROCm-compatible GPU and drivers already set up, and walks through installing K3s and Helm, then deploying cert-manager (for automated TLS certificate management) and the ROCm GPU Operator to enable AMD GPU management across the cluster. By the end, you'll have a K3s cluster with GPU resource management running.

1. Install and Configure K3s

1. Install K3s:

$ curl -sfL https://get.k3s.io | sh -

2. Create a .kube directory in your user home directory:

$ mkdir -p $HOME/.kube

3. Symlink the K3s config file as the default kubeconfig:

$ ln -s /etc/rancher/k3s/k3s.yaml $HOME/.kube/config

4. Change the config file permissions so Helm can load it:

$ sudo chmod 755 $HOME/.kube/config

5. Install Helm:

$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

6. Check the K3s service status:

$ sudo systemctl status k3s

7. Enable the K3s service to start on boot:

$ sudo systemctl enable k3s

2. Install the ROCm GPU Operator

This section installs cert-manager, a dependency for managing TLS certificates in Kubernetes, and deploys the ROCm GPU Operator, which handles GPU management in the cluster.

1. Add the Helm repository for cert-manager:

$ helm repo add jetstack https://charts.jetstack.io --force-update

2. Install cert-manager:

$ helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.15.1 --set crds.enabled=true

This installs cert-manager v1.15.1 — check the official documentation for the latest version before installing.

3. Install the ROCm GPU Operator:

$ helm install amd-gpu-operator --namespace kube-amd-gpu --create-namespace https://github.com/ROCm/gpu-operator/releases/download/v1.0.0/gpu-operator-charts-v1.0.0.tgz

This deploys the AMD GPU Operator into the kube-amd-gpu namespace, enabling management and utilization of AMD GPUs in the cluster.

4. List the deployed Kubernetes resources:

$ kubectl get all --namespace kube-amd-gpu

Output:

NAME                                                                  READY   STATUS    RESTARTS   AGE
pod/amd-gpu-operator-gpu-operator-charts-controller-manager-6bmkfwr   1/1     Running   0          6m
pod/amd-gpu-operator-kmm-controller-5f8d79b46-j4sbw                   1/1     Running   0          6m
pod/amd-gpu-operator-kmm-webhook-server-5d5fc8bdd6-p7qfs              1/1     Running   0          6m
pod/amd-gpu-operator-node-feature-discovery-gc-78989c896-lffs2        1/1     Running   0          6m
pod/amd-gpu-operator-node-feature-discovery-master-b8bffc48b-dtw7x    1/1     Running   0          6m
pod/amd-gpu-operator-node-feature-discovery-worker-m6jfz              1/1     Running   0          6m
.........

Pods can take 5–10 minutes to transition to a Running status.

Next Steps

Deploy a ROCm-accelerated workload (PyTorch, TensorFlow, or a custom container) onto the cluster to confirm GPU scheduling works end to end.
Look into running Docker with ROCm GPU support if you also need GPU access outside of Kubernetes.
Monitor GPU utilization and health through the metrics the GPU Operator exposes.
Review the ROCm GPU Operator usage docs for advanced scheduling and node-labeling options.

For the full guide with additional tips, visit the original article on Vultr Docs.

Installing phpMyAdmin on Ubuntu 24.04

Sanskriti Harmukh — Wed, 14 Oct 2026 21:00:41 +0000

phpMyAdmin is an open-source, web-based tool for managing MySQL databases. Installing it gives you a browser interface for creating, modifying, and deleting databases, tables, and fields, managing users, importing and exporting data, and running SQL statements without touching the command line. This guide installs MySQL and phpMyAdmin on Ubuntu 24.04, creates a sample database and user, and secures the phpMyAdmin endpoint with an extra layer of basic authentication. By the end, you'll have a working phpMyAdmin installation running over Apache, protected against unauthenticated access.

1. Set Up the MySQL Database Server

1. Update the APT package index:

$ sudo apt update

2. Install MySQL server:

$ sudo apt install mysql-server -y

3. Log in to the MySQL server:

$ sudo mysql

4. Create a sample database:

mysql> CREATE DATABASE my_company;

5. Create a new MySQL user with a strong password, replacing your_password:

mysql> CREATE USER 'company_admin'@'localhost' IDENTIFIED BY 'your_password';

6. Grant the user full privileges on the database:

mysql> GRANT ALL PRIVILEGES ON my_company.* TO 'company_admin'@'localhost';

7. Flush privileges to apply the changes:

mysql> FLUSH PRIVILEGES;

8. Exit the MySQL console:

mysql> EXIT;

2. Install phpMyAdmin

phpMyAdmin is available in the default package repositories on Ubuntu 24.04.

1. Update the APT package index:

$ sudo apt update

2. Install PHP and the required extensions:

$ sudo apt install php php-mysql php-mbstring php-json php-xml php-curl php-zip php-common -y

3. Install phpMyAdmin:

$ sudo apt install phpmyadmin -y

During setup, select apache2 as the web server to reconfigure automatically, keep the database configuration for phpMyAdmin set to Yes, and set a strong phpMyAdmin database password when prompted (enter it twice to confirm).

4. Verify the installation succeeded:

$ dpkg -l | awk '/phpmyadmin / {print}'

Output:

ii  phpmyadmin                           4:5.2.1+dfsg-3                          all          MySQL web administration tool

3. Configure phpMyAdmin

phpMyAdmin ships with a default Apache virtual host configuration that serves the /phpmyadmin endpoint using your server's IP or domain.

1. Enable the phpMyAdmin virtual host configuration:

$ sudo a2enconf phpmyadmin.conf

2. Restart Apache to apply the configuration changes:

$ sudo systemctl restart apache2

3. Confirm Apache is running:

$ sudo systemctl status apache2

4. Check the UFW status:

$ sudo ufw status

If UFW isn't installed, install it and allow SSH:

$ sudo apt install ufw -y && sudo ufw allow ssh

5. Allow the Apache Full profile through the firewall to enable HTTP and HTTPS:

$ sudo ufw allow 'Apache Full'

6. Reload UFW to apply the changes:

$ sudo ufw reload

4. Access phpMyAdmin

1. Open the phpMyAdmin endpoint in a browser using your server's IP address or domain name:

http://your_server_ip/phpmyadmin

2. Log in using the company_admin credentials you created earlier, then select the my_company database from the left navigation menu to manage it.

Avoid logging in to phpMyAdmin with the MySQL root account — accidental deletions, modifications, and security vulnerabilities are far more consequential from a superuser account. Always create a dedicated user scoped to the database(s) it actually needs.

5. Secure the phpMyAdmin Endpoint

Basic authentication adds a second password prompt in front of the phpMyAdmin interface.

1. Create an .htpasswd file with a new username, replacing your_username:

$ sudo htpasswd -c /etc/phpmyadmin/.htpasswd your_username

2. Open the phpMyAdmin Apache configuration:

$ sudo nano /etc/apache2/conf-available/phpmyadmin.conf

3. Add the AllowOverride All directive inside the /usr/share/phpmyadmin directory block, so it looks like this:

<Directory /usr/share/phpmyadmin>
    AllowOverride All
    ...
</Directory>

4. Create a new .htaccess file:

$ sudo nano /usr/share/phpmyadmin/.htaccess

5. Add the following authentication configuration to the file:

AuthType Basic
AuthName "Restricted Access to PhpMyAdmin"
AuthUserFile /etc/phpmyadmin/.htpasswd
Require valid-user

AuthType Basic sets the authentication type, AuthName sets the message shown in the sign-in prompt, AuthUserFile points to the file with allowed users and encrypted passwords, and Require valid-user enforces that a valid user must authenticate.

6. Restart Apache to apply the configuration changes:

$ sudo systemctl restart apache2

7. Reload the phpMyAdmin URL and confirm you're prompted for the new username and password, then log in with your MySQL credentials to manage databases:

http://your_server_ip/phpmyadmin

6. Troubleshoot phpMyAdmin

phpMyAdmin login fails:

$ sudo systemctl status mysql

Double-check your MySQL username and password, and review the config file for errors:

$ sudo nano /etc/phpmyadmin/config.inc.php

Internal Server Error:

$ sudo tail -f /var/log/apache2/error.log

Reinstall the required PHP extensions if any are missing:

$ sudo apt install php-mysql php-mbstring php-json php-xml php-curl php-zip php-common -y

Access Denied:

$ sudo mysql

mysql> CREATE USER 'your_username'@'localhost' IDENTIFIED BY 'your_password';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'your_username'@'localhost' WITH GRANT OPTION;
mysql> FLUSH PRIVILEGES;
mysql> EXIT;

Next Steps

Create additional MySQL users scoped to individual databases instead of reusing one admin account.
Put phpMyAdmin behind HTTPS with a reverse proxy or Apache TLS configuration.
Automate MySQL backups before making structural changes through phpMyAdmin.
Review Apache and MySQL logs periodically for unusual login attempts.

For the full guide with additional tips, visit the original article on Vultr Docs.

Deploying Fluxer - An Open Source Instant Messaging and VoIP Platform

Sanskriti Harmukh — Wed, 30 Sep 2026 19:54:43 +0000

Fluxer is a free and open-source instant messaging and VoIP platform for friends, groups, and communities, providing messaging, communities, text and voice channels, media uploads, search, presence, and voice and video communication from a web-based interface. Fluxer ships an official Docker Compose stack that runs the web application, API, gateway, administration dashboard, messaging services, media services, PostgreSQL, Valkey, NATS, Meilisearch, SeaweedFS, LiveKit, and Caddy. This guide deploys Fluxer on a Linux server using that Docker Compose stack, exposes it over HTTPS with Caddy as a reverse proxy, and creates the owner account. By the end, you'll have a self-hosted Fluxer instance running securely at your domain with messaging and voice/video channels ready to use.

Set Up the Directory Structure and Environment Variables

This section covers creating a working directory for Fluxer, downloading the official Docker Compose files, and configuring the environment variables required by the application.

1. Create a new directory for Fluxer and move into it:

$ mkdir ~/fluxer
$ cd ~/fluxer

2. Download the official Docker Compose manifest:

$ curl -fsSLO https://raw.githubusercontent.com/fluxerapp/fluxer/3b1eb56713d59181823df284b9646c890d2a3cba/deploy/self-hosting/docker-compose.yml

3. Download the default Caddy configuration:

$ curl -fsSLO https://raw.githubusercontent.com/fluxerapp/fluxer/3b1eb56713d59181823df284b9646c890d2a3cba/deploy/self-hosting/Caddyfile

4. Download the LiveKit configuration:

$ curl -fsSLO https://raw.githubusercontent.com/fluxerapp/fluxer/3b1eb56713d59181823df284b9646c890d2a3cba/deploy/self-hosting/livekit.yaml

5. Download the example environment file and save it as .env:

$ curl -fsSL https://raw.githubusercontent.com/fluxerapp/fluxer/3b1eb56713d59181823df284b9646c890d2a3cba/deploy/self-hosting/.env.example -o .env

The Fluxer directory now contains the following files:

Caddyfile
docker-compose.yml
livekit.yaml
.env

These files define the Fluxer containers, public routing, voice services, and application configuration.

6. Open the .env file:

$ nano .env

Set the following variables at the beginning of the file. Replace fluxer.example.com with your Fluxer domain and admin@example.com with your email address.

FLUXER_DOMAIN=fluxer.example.com
FLUXER_CADDY_SITE_ADDRESS=fluxer.example.com
FLUXER_VAPID_EMAIL=admin@example.com

Save and close the file. FLUXER_DOMAIN defines the public Fluxer address. FLUXER_CADDY_SITE_ADDRESS specifies the hostname Caddy serves.

Generate Fluxer Secrets

Fluxer requires separate secrets for PostgreSQL, Meilisearch, object storage, internal RPC communication, the administration dashboard, media uploads, and LiveKit.

1. Run the following loop to generate random hexadecimal values and insert them into .env:

$ for key in POSTGRES_PASSWORD MEILI_MASTER_KEY FLUXER_S3_SECRET_KEY FLUXER_SUDO_MODE_SECRET FLUXER_CONNECTION_INITIATION_SECRET FLUXER_GATEWAY_RPC_AUTH_TOKEN FLUXER_MEDIA_PROXY_SECRET_KEY FLUXER_ADMIN_SECRET_KEY_BASE FLUXER_ADMIN_OAUTH_CLIENT_SECRET LIVEKIT_API_SECRET; do sed -i "s|^$key=.*|$key=$(openssl rand -hex 32)|" .env; done

2. Generate the media upload relay secret in Base64 format and insert it into the .env file:

$ sed -i "s|^FLUXER_MEDIA_PROXY_UPLOAD_RELAY_SECRET_BASE64=.*|FLUXER_MEDIA_PROXY_UPLOAD_RELAY_SECRET_BASE64=$(openssl rand -base64 32)|" .env

Generate VAPID Keys

Fluxer uses VAPID keys for web push notifications.

1. Add your user to the docker group to run Docker commands without sudo:

$ sudo usermod -aG docker $USER

2. Apply the group change to the current session:

$ newgrp docker

3. Generate a VAPID key pair using a temporary Node.js container:

$ docker run --rm node:24-alpine npx --yes web-push generate-vapid-keys --json

Your output contains two values: publicKey (VAPID public key) and privateKey (VAPID private key). Copy the value displayed after publicKey and privateKey.

4. Add the public key to .env:

$ sed -i 's|^FLUXER_VAPID_PUBLIC_KEY=.*|FLUXER_VAPID_PUBLIC_KEY=YOUR_VAPID_PUBLIC_KEY|' .env

Replace YOUR_VAPID_PUBLIC_KEY with the publicKey value from your output.

5. Add the private key to .env:

$ sed -i 's|^FLUXER_VAPID_PRIVATE_KEY=.*|FLUXER_VAPID_PRIVATE_KEY=YOUR_VAPID_PRIVATE_KEY|' .env

Replace YOUR_VAPID_PRIVATE_KEY with the privateKey value from your output.

6. Restrict access to the .env file so that only your account can read or modify it:

$ chmod 600 .env

The .env file contains the credentials and secrets for your Fluxer deployment. Do not upload it to a public repository or include its contents in screenshots.

7. Verify the public Fluxer settings:

$ grep -E '^(FLUXER_DOMAIN|FLUXER_CADDY_SITE_ADDRESS|FLUXER_VAPID_EMAIL)=' .env

Output:

FLUXER_DOMAIN=fluxer.example.com
FLUXER_CADDY_SITE_ADDRESS=fluxer.example.com
FLUXER_VAPID_EMAIL=admin@example.com

Deploy Fluxer Containers

This section covers validating the Docker Compose configuration, deploying the Fluxer containers, and verifying that all services initialize correctly.

1. Navigate to the Fluxer directory:

$ cd ~/fluxer

2. Validate the Docker Compose configuration:

$ docker compose config --quiet

The command returns no output when the Compose configuration is valid.

3. Download the Fluxer container images:

$ docker compose pull

Docker downloads the Fluxer application images and supporting services required by the stack.

4. Create and start the containers:

$ docker compose up -d

The first deployment can take several minutes while Docker downloads images and Fluxer initializes its application, database, search, object storage, and messaging services.

5. Verify the containers:

$ docker compose ps -a

Every service should show a status of Up, and services with a health check also show (healthy). The seaweedfs-init container exits after creating the required object-storage buckets — an exit code of 0 for this container is expected.

6. View the Fluxer API logs:

$ docker compose logs -f api

Press Ctrl+C after the API completes its startup process.

7. View recent logs from all Fluxer services:

$ docker compose logs --tail=100

8. If a container fails to start, view its logs using the service name:

$ docker compose logs SERVICE_NAME

Replace SERVICE_NAME with the failed service, such as api, gateway, or livekit.

Configure Caddy as a Reverse Proxy to Expose Fluxer

Fluxer's official Docker Compose stack includes Caddy as the reverse proxy. Caddy listens on ports 80 and 443, automatically provisions HTTPS certificates for the hostname defined by FLUXER_CADDY_SITE_ADDRESS, and forwards requests to the appropriate Fluxer containers. The API, gateway, media service, administration dashboard, LiveKit signaling, and web application are all available through the same public hostname.

1. Validate the Caddy configuration:

$ docker compose exec caddy caddy validate --config /etc/caddy/Caddyfile

A successful validation indicates that Caddy can load the configuration.

2. View the Caddy logs:

$ docker compose logs --tail=100 caddy

Caddy automatically requests and renews TLS certificates when fluxer.example.com resolves to your server and ports 80 and 443 are publicly reachable.

Access and Configure Fluxer

This section covers accessing the Fluxer web application, creating the initial owner account, and reviewing the main instance settings.

Open https://fluxer.example.com in your web browser.
The Fluxer Set Up Welcome page should appear. Select your preferred language from the Language menu and click Next to continue.
Choose a theme for the Fluxer Admin interface and click Next to continue.
The Create the administrator account page appears. Fluxer displays a message that the instance does not have any accounts yet and that the first account you create automatically becomes the administrator. Click Next to continue.
Configure the administrator account. Enter all required information and click Next to create an account.
Open a new tab in your web browser and access the Fluxer administration dashboard at https://fluxer.example.com/admin.
Log in using the Fluxer account you created.
Review the requested permissions required by the administration application. Keep the required scopes enabled and click Authorize.
Fluxer redirects you to the Admin dashboard after authorization. The dashboard displays administrative tools in the left navigation menu.

Your Fluxer administrator account is now configured and ready to manage users, guilds, applications, moderation settings, and other instance-level features.

Next Steps

Fluxer is running with messaging, voice and video channels, and media sharing available to your users behind HTTPS. From here you can:

Create additional guilds and channels, and invite members to your community
Review moderation and permission settings from the admin dashboard
Configure object storage retention and monitor SeaweedFS, Meilisearch, and LiveKit resource usage as usage grows

For the full guide with additional tips, visit the original article on Vultr Docs.

Deploying Umami - Open-Source Analytics Platform

Sanskriti Harmukh — Wed, 30 Sep 2026 19:54:18 +0000

Umami is an open-source, privacy-focused web analytics platform designed as a self-hosted alternative to Google Analytics. It avoids cookies and personally identifiable information, removes the need for cookie banners, and captures real-time metrics including visitor devices, geographic data, referrers, and custom events through a lightweight tracking script. This guide deploys Umami on a Linux server using Docker Compose with PostgreSQL as the database, Redis for session caching, and Traefik for automatic HTTPS through Let's Encrypt, then walks through directory setup, environment configuration, and integrating the tracking script into a sample website. By the end, you'll have a self-hosted Umami instance tracking real traffic on your own domain.

Set Up the Directory Structure, Configuration, and Environment Variables

Umami requires a project directory with persistent storage for the PostgreSQL database, Redis cache, and Let's Encrypt certificates. Environment variables control the domain, database credentials, application secret, and tracking customizations.

1. Create the project directory and move into it:

$ mkdir -p ~/umami/{pgdata,redis,letsencrypt}
$ cd ~/umami

pgdata/: Persists PostgreSQL database files.
redis/: Stores Redis data for caching and session management.
letsencrypt/: Stores Traefik ACME certificates for HTTPS renewal.

2. Generate a strong random secret for the application:

$ openssl rand -hex 32

Save the output for use in the environment file.

3. Create the environment file:

$ nano .env

Add the following configuration. Replace umami.example.com with your domain name, admin@example.com with your email address, GENERATED_SECRET with the generated random string, and STRONG_DATABASE_PASSWORD with a secure database password.

# Domain Configuration
DOMAIN=umami.example.com
LETSENCRYPT_EMAIL=admin@example.com

# Application Secret
APP_SECRET=GENERATED_SECRET

# PostgreSQL Configuration
POSTGRES_DB=umami
POSTGRES_USER=umami
POSTGRES_PASSWORD=STRONG_DATABASE_PASSWORD
DATABASE_URL=postgresql://umami:${POSTGRES_PASSWORD}@postgres:5432/umami

# Redis Configuration
REDIS_URL=redis://redis:6379

# Tracker Customization
TRACKER_SCRIPT_NAME=custom-stats
COLLECT_API_ENDPOINT=/custom-api/send
DISABLE_TELEMETRY=1

Save and close the file.

Note: The TRACKER_SCRIPT_NAME and COLLECT_API_ENDPOINT variables add an alternate path for the tracking script and collection endpoint, alongside the defaults, to help bypass ad blockers that specifically target /script.js. The default paths remain active. DISABLE_TELEMETRY=1 prevents Umami from sending anonymous usage statistics.

Deploy with Docker Compose

The deployment stack runs four services: Traefik as the reverse proxy with automatic TLS, PostgreSQL for analytics data persistence, Redis for session caching, and the Umami web application. This configuration is based on the official Umami Docker Compose example, adapted to use Traefik and persistent storage.

1. Create the Docker Compose manifest:

$ nano docker-compose.yml

services:
  traefik:
    image: traefik:v3.7.0
    container_name: umami-traefik
    restart: unless-stopped
    command:
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--certificatesresolvers.letsencrypt.acme.tlschallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL}"
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./letsencrypt:/letsencrypt

  postgres:
    image: postgres:17-alpine
    container_name: umami-postgres
    restart: unless-stopped
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - ./pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d ${POSTGRES_DB} -U ${POSTGRES_USER}"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:8-alpine
    container_name: umami-redis
    restart: unless-stopped
    command: ["redis-server", "--appendonly", "yes"]
    volumes:
      - ./redis:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5

  umami:
    image: ghcr.io/umami-software/umami:3.1.0
    container_name: umami
    restart: unless-stopped
    env_file: .env
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.umami.rule=Host(`${DOMAIN}`)"
      - "traefik.http.routers.umami.entrypoints=websecure"
      - "traefik.http.routers.umami.tls=true"
      - "traefik.http.routers.umami.tls.certresolver=letsencrypt"
      - "traefik.http.services.umami.loadbalancer.server.port=3000"

Save and close the file. traefik handles reverse proxy and TLS termination, redirecting HTTP to HTTPS and provisioning Let's Encrypt certificates using the TLS-ALPN-01 challenge. postgres runs PostgreSQL 17 as the primary database for analytics data, website configurations, and user accounts. redis runs Redis 8 as the caching layer for server-side authentication sessions, reducing database load. umami runs the official Umami application pinned to version 3.1.0 and registers with Traefik for HTTPS routing on the configured domain.

2. Start the services:

$ docker compose up -d

3. Verify that all containers are running:

$ docker compose ps -a

The output displays four containers. postgres and redis show a healthy status; traefik and umami show as running, since those two have no configured health check.

4. View the service logs to confirm Umami connected to the database:

$ docker compose logs umami

The output displays Umami completing database migrations and reporting the application as ready.

Access and Configure Umami

After deployment, access the Umami dashboard using your configured domain, log in with the default administrator credentials, and rotate the password to secure the instance.

1. Open the login page:

Replace umami.example.com with your configured domain and open https://umami.example.com/login in a web browser.

2. Log in using the default administrator credentials. The username is admin and the password is umami.

Note: Change the default credentials immediately after the first login to secure your analytics instance.

3. Change the password:

At the bottom of the left sidebar, click admin and select Settings. Navigate to the Profile tab, click Change password, enter the current password (umami), enter your new secure password, and click Save.

Track a Sample Website

Umami tracks visitors through a lightweight JavaScript snippet embedded in your target website. The following steps add a website, retrieve the tracking code, embed it in a sample page, and verify that metrics appear in the dashboard.

Add a Website

Umami tracks each site separately, so a website entry must exist before its tracking code can be generated.

In the Umami dashboard, click Websites in the left navigation pane.
Click Add website.
Enter a name such as My Test Site and a domain such as test.example.com.
Click Save.

Retrieve the Tracking Code

Each website gets a unique tracking snippet tied to its website ID.

Find the new website in the list and click Edit.
Navigate to the Tracking code tab.
Copy the generated <script> tag.

Note: Because TRACKER_SCRIPT_NAME=custom-stats was set in the environment file, the generated tag uses /custom-stats as an additional path alongside the default /script.js. Using the custom path helps bypass ad blockers that specifically target the default filename.

Embed the Tracking Code

The script must load on every page you want Umami to track.

Open the main HTML document or the global layout component of your sample web application.
Paste the tracking code into the <head> section. Replace YOUR_WEBSITE_ID with the ID from your Umami dashboard and umami.example.com with your domain:

   <head>
       <meta charset="UTF-8">
       <title>Sample Web Application</title>
       <script defer src="https://umami.example.com/custom-stats" data-website-id="YOUR_WEBSITE_ID"></script>
   </head>

Save the file and open the sample web application in a browser.
Refresh the page a few times to simulate visitor traffic.

Verify Analytics Collection

Confirming that traffic appears in the dashboard validates that the tracking script is reaching the server.

Return to the Umami dashboard and select your website.
Review the main dashboard to verify captured page views, referrers, device types, and visitor data.
Click the Realtime tab to monitor active visitors navigating the sample site.

Next Steps

Umami is running with a privacy-first, self-hosted analytics dashboard tracking live traffic. From here you can:

Add more websites and compare traffic across properties from a single dashboard
Configure custom events to track conversions, signups, or other in-app actions
Enable email reports or connect the Umami API to pull metrics into external tooling

For the full guide with additional tips, visit the original article on Vultr Docs.

DEV Community: Vultr

Configuring Smart Routing in NVIDIA Dynamo

Prerequisites

Understanding KV-Aware Routing

How KV Cache Routing Works

Worker Selection Process

Performance Benefits

Clone the Dynamo Repository

Verify CUDA Version and Pull the Container Image

Configure Hugging Face Cache Permissions

Start Infrastructure Services

KV Router Configuration Parameters

Core Router Parameters

Worker Capacity Thresholds

Infrastructure Configuration

Parameter Tuning Guidelines

Set Up the Deployment Environment

Configure KV Routing for Aggregated Serving

Deploy Aggregated Serving with KV Routing

Verify KV Routing

Configure KV Routing for Disaggregated Serving

Deploy Disaggregated Serving with KV Routing

Verify Disaggregated KV Routing

Configure KV Routing with SGLang and TensorRT-LLM

Universal Environment Variables

Backend-Specific Configuration

Tuning KV Router Parameters

Overlap Score Weight

Router Temperature

KV Events Configuration

Active Block Busy Detection

Router Replica Synchronization

Monitor KV Router Performance

Next Steps

Automating Bash Scripts on Linux Using Cron and Systemd

Prerequisites

Create Bash Scripts

Automate the Bash Script Using Cron

Automate the Bash Script as a System Service Using Systemd

Next Steps

Managing File and Directory Permissions in Linux

Linux Permissions

String Representation of Linux Permissions

How to Calculate Linux Permissions

Common Linux Permissions

View File and Directory Permissions

Change File and Directory Permissions

Change Permissions Using Numeric Mode

Change Permissions Using Symbolic Mode

Set Special Permissions (Sticky Bit, SUID, SGID)

Sticky Bit

Set User ID (SUID)

Set Group ID (SGID)

Change User and Group Ownership Permissions

Next Steps

Managing Node Applications with PM2

Prerequisites

Install PM2 Using npm

Create an Example Node.js Application

Start the Application Using PM2

Manage the Application

List All Applications

Display Logs

Monitor Resource Usage

Set Up a Web-Based Dashboard

Configure PM2 to Start on Server Boot

Configure Applications to Start on Server Boot

Check for Updates

Update PM2

Best Practices and Security Considerations

Prevent Downtime With Cluster Mode

Eliminate Failed Requests With Graceful Shutdown

Next Steps

Viewing and Managing System Logs Using rsyslog in Linux

How rsyslog Works: The Input, Filter, and Output Model

Common Log File Locations in Linux

View Logs with Basic Linux Commands

Filter Logs Using grep, awk, and sed

Configure rsyslog Using /etc/rsyslog.conf

Common Configuration Examples

How `rsyslog` Works: The Input, Filter, and Output Model

Filter Logs Using `grep`, `awk`, and `sed`

Configure `rsyslog` Using `/etc/rsyslog.conf`

Best Practices for Using `rsyslog`

`nftables` versus `iptables`

Install `nftables` on Linux

`nft` Command Syntax

Manage the `nftables` System Service

Tables, Chains, and Policies in `nftables`

Manage the Default `nftables` Configuration

Configure Basic `nftables` Firewall Rules