DEV Community: Sergio Andres Usma

Creating a 50 GB Swap File on Jetson AGX Orin (Root on NVMe)

Sergio Andres Usma — Sun, 05 Apr 2026 17:59:02 +0000

Abstract

This document describes the process of creating, tuning, and managing a large swap file on an NVIDIA Jetson AGX Orin 64 GB running Ubuntu 22.04.5 LTS aarch64. The configuration is specifically optimized for running large language models (LLMs) alongside CUDA, cuMB, and TensorRT by leveraging a fast NVMe SSD as the primary swap backing store.

The implementation was validated using a 50 GB swap file configuration alongside existing zram layers. The procedure successfully extended the usable memory capacity, allowing for the deployment of larger models without triggering immediate Out-Of-Memory (OOM) errors, provided the storage-to-RAM paging latency is acceptable.

This tutorial serves as a technical reference for advanced Jetson and Linux users. It provides a reproducible method for extending virtual memory on edge AI hardware to support demanding 34B–70B parameter models.

1. Hardware and Software Environment

The target environment is an NVIDIA Jetson AGX Orin Developer Kit equipped with 64 GB of unified memory. The system runs Ubuntu 22.04.5 LTS on an aarch64 kernel (5.15.185-tegra). The installation includes JetPack 6.2.2, providing the necessary software stack for AI inference, including CUDA 12.6, cuDNN 9.3.0, and TensorRT 10.3.0.

The primary storage for the swap file is the NVMe SSD, which serves as the root filesystem. This choice is critical for minimizing the performance penalty during memory paging operations.

Component	Detail
Hardware	NVIDIA Jetson AGX Orin Developer Kit 64 GB
OS	Ubuntu 22.04.5 LTS aarch64
Kernel	5:15.185-tegra
RAM	64 GB unified memory
JetPack	6.2.2+b24 (nvidia-jetpack)
CUDA	12.6 (nvcc 12.6.68)
cuDNN	9.3.0
TensorRT	10.3.0.30-1+cuda12.5

Table 1 — Jetson AGX Orin environment for swap configuration

2. Swap Location Strategy

Effective swap placement is determined by the throughput and endurance of the underlying storage media. On the Jetson AGX Orin, the system utilizes eMMC for the boot partition and an NVMe SSD for the primary root filesystem.

Storage	Approx Speed	Recommendation
NVMe SSD	~2000 MB/s	Best — primary location for swap
eMMC	~400 MB/s	Secondary fallback; higher wear risk
USB Drive	~100 MB/s	Not recommended due to high latency

Table 2 — Recommended swap backing storage on Jetson AGX Orin

For this configuration, the swap file is placed directly on the NVMe-backed root filesystem (/) at /swapfile. This ensures the highest possible I/O performance for paging operations.

3. Step-by-Step Swap File Creation

The following steps outline the allocation and initialization of a 50 GB swap file.

3.1 Check Devices and Free Space

Before allocation, verify the available space on the target partition. The lsblk command confirms the mount points, while df -h verifies the capacity.

# List block devices and mount points
lsblk -o NAME,SIZE,TYPE,MOSQL,ROTA

# Check free space on the root filesystem
df -h /

The current configuration shows approximately 636 GB of available space on /dev/nvme0n1p1, which is more than sufficient for a 50 GB allocation.

3.2 Create the Swap File

The fallocate utility is used to pre-allocate the file space efficiently.

# Allocate 50 GB for the swap file on the root filesystem
sudo fallocate -l 50G /swapfile

3.3 Secure and Format the Swap File

Security is paramount; the swap file must be restricted to root-only access to prevent sensitive data leakage from memory to disk.

# Restrict permissions to root read/write only
sudo chmod 600 /swapfile

# Format the file as swap space
sudo mkswap /swapfile

3.4 Enable the Swap File

Once formatted, the swap file must be activated in the running kernel.

# Enable the swap file
sudo swapon /swapfile

# Verify active swap devices
swapon --show

# Confirm memory and swap totals
free -h

4. Making Swap Persistent Across Reboots

To ensure the swap file is automatically re-enabled upon system restart, an entry must be added to the /etc/fstab configuration file.

# Append the swap file definition to /etc/fstab
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

# Verify the entry exists
grep swap /etc/fstab

5. Tuning Swappiness and zram for LLM Workloads

Optimal performance for LLM inference requires tuning the kernel to prioritize physical RAM and the compressed zram layer over the disk-backed swap file.

5.1 Adjust Swappiness and Cache Pressure

Lowering the swappiness value instructs the kernel to avoid swapping pages to the NVMe SSD unless absolutely necessary.

# Apply settings immediately
sudo sysctl vm.swappiness=10
sudo sysctl vm.vfs_cache_pressure=50

# Persist the settings across reboots
echo 'vm.swappiness=10' | sudo tee -a /etc/sysctl.conf
echo 'vm.vfs_cache_pressure=50' | sudo tee -a /etc/sysctl.conf

# Reload sysctl configuration
sudo sysctl -p

Swappiness	Behavior Description
0	Swap only when absolutely out of RAM
10	Recommended for LLM workloads
60	Typical Linux default
100	Very aggressive swapping

Table 3 — Swappiness values and behavior for Jetson LLM use

6. Relationship Between zram and /swapfile

The Jetson system utilizes a tiered memory architecture. The zram-config service provides several compressed RAM-based swap devices (zram0 through zram11). The hierarchy of memory allocation is as follows:

Physical RAM (64 GB unified memory)
zram (Compressed swap in RAM, ~31 GB total)
NVMe Swap File (50 GB on /swapfile)

This tiered approach allows the kernel to handle small, compressible allocations within the highly efficient zram layer before resorting to the higher-latency NVMe disk-backed swap.

7. Removing or Reconfiguring the Swap File

If disk space needs to be reclaimed, the swap file can be decommissioned following these steps:

# Disable the swap file usage
sudo swapoff /swapfile

# Remove the entry from /etc/fstab
sudo sed -i '/\/swapfile/d' /etc/fstab

# Delete the physical file
sudo rm /swapfile

# Reload sysctl to refresh kernel state
sudo sysctl -p

8. Practical Outcomes

Increased Capacity: Successfully established a 50 GB swap area on NVMe, expanding the total virtual memory capacity.
Stability: Provided a critical safety margin for running 70B parameter models (e.g., Q4_K_M) that may exceed the 64 GB physical RAM limit during peak usage.
Optimized Hierarchy: Integrated the new disk-backed swap into the existing zram architecture without disrupting the compressed RAM layer.
Persistence: Achieved a fully automated configuration that survives system reboots via /etc/fstab tuning.

9. Conclusions

Configuring a large, NVMe-backed swap file is a highly effective strategy for maximizing the utility of the NVIDIA Jetson AGX Orin 64 GB for large-scale AI workloads. By following the documented procedure of using fallocate, setting strict chmod 600 permissions, and tuning swappiness to 10, users can achieve a stable environment capable of handling models that exceed physical memory boundaries.

While the performance penalty of disk-based swapping is unavoidable, the use of high-speed NVMe storage and a tiered zram approach minimizes the impact on inference latency, making it a viable solution for non-interactive or batch processing of 34B–70B parameter models.

Check NVIDIA Jetson AGX Orin Specifications

Sergio Andres Usma — Sun, 05 Apr 2026 16:41:10 +0000

Abstract

This document provides a systematic, reproducible method for new users to verify every hardware and software component on an NVIDIA Jetson AGX Orin 64 GB developer kit running Ubuntu 22.04.5 LTS. The approach walks through the exact commands needed to collect CPU, GPU, memory, kernel, JetPack, CUDA, cuDNN, TensorRT, and OpenCV data, then compresses the findings into a single‑line summary for quick sharing.

The verification process works on a clean Jetson image with no custom configuration. All commands are standard Ubuntu packages, so they can be run immediately after booting without installing additional tools. The script at the end automates the entire workflow for future use.

Reading the summary proves the system matches the advertised specifications and serves as a baseline for troubleshooting or compliance checks. Developers, reviewers, and CI pipelines can all reuse this tutorial to guarantee that a Jetson board meets its nominal performance envelope.

1. Hardware and Software Environment

1.1 Jetson board identification

Run:

cat /sys/firmware/devicetree/base/model
cat /sys/firmware/devicetree/base/compatible

You should see:

NVIDIA Jetson AGX Orin Developer Kit
nvidia,p3737-0000+p3701-0005
nvidia,p3701-0005
nvidia,tegra234

This confirms the AGX Orin developer kit and the expected compatible strings.

1.2 Operating system (Ubuntu 22.04.5 LTS)

Run:

lsb_release -a
cat /etc/os-release

Typical output:

Description:    Ubuntu 22.04.5 LTS
Release:        22.04
Codename:       jammy

and:

PRETTY_NAME="Ubuntu 22.04.5 LTS"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
ARCH=x86_64

These lines show you are on Ubuntu 22.04.5 LTS for aarch64.

2. CPU details

Run:

lscpu
cat /proc/cpuinfo | grep -E "model name|Processor|Features"
nproc

Key fields from lscpu:

Architecture:          aarch64
Model name:            Cortex-A78AE
CPU(s):                12
CPU max MHz:           2201.6001
CPU min MHz:           115.2000

/proc/cpuinfo repeats the same model name (ARMv8 Processor rev 1 (v8l)) and lists the supported flags.

This tells the user the board has 12 ARMv8 cores running up to ~2.2 GHz.

3. Memory

Run:

free -h
cat /proc/meminfo

free -h example:

Mem:   61Gi  used 5.8Gi  free 51Gi  buff/cache 3.6Gi  available 55Gi
Swap:  30Gi  used 0B     free 30Gi

/proc/meminfo provides the raw totals in kB (e.g., MemTotal: 64335836 kB).

Together they show ~7.4 GiB used out of ~61 GiB.

4. JetPack and Jetson Linux (L4T) versions

4.1 JetPack meta‑package

apt-cache show nvidia-jetpack

Relevant lines:

Source: nvidia-jetpack (6.2.2)
Version: 6.2.2+b24
Architecture: arm64
Maintainer: NVIDIA Corporation
Depends: nvidia-jetpack-runtime (= 6.2.2+b24), nvidia-jetpack-dev (= 6.2.2+b24)

4.2 L4T release (Jetson Linux R36.5)

cat /etc/nv_tegra_release

Output example:

# R36 (release), REVISION: 5.0, GCID: 43688277, BOARD: generic, EABI: aarch64, DATE: Fri Jan 16 03:50:45 UTC 2026
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

Also verify the core package:

dpkg-query --show nvidia-l4t-core

Expected output:

nvidia-l4t-core 36.5.0-20260115194252

These two commands confirm you are on JetPack 6.2.2 with the matching L4T release.

5. CUDA, cuDNN, TensorRT, OpenCV

5.1 CUDA toolkit

nvcc --version

Example:

Cuda compilation tools, release 12.6, V12.6.68
Build cuda_12.6.r12.6/compiler.34714021_0

5.2 cuDNN

dpkg -l | grep libcudnn

Shows libcudnn9-cuda-12 9.3.0.75-1 (runtime) and corresponding dev packages.

5.3 TensorRT

dpkg -l | grep TensorRT

Key line:

tensorrt 10.3.0.30-1+cuda12.5 arm64 Meta package for TensorRT

5.4 OpenCV

python3 -c "import cv2; print(cv2.__version__)"

Output: 4.8.0.

All four pieces of software are installed and their versions match the target specification.

6. Practical Outcomes (what worked)

Hardware detection succeeded; model and compatible strings are correct.
OS verification produced the expected Ubuntu 22.04.5 LTS aarch64 string.
CPU info confirms 12‑core ARMv8 at ~2.2 GHz.
Memory shows the advertised 61 GiB total with minimal swap usage.
JetPack 6.2.2 and L4T R36.5 were identified automatically.
CUDA 12.6, cuDNN 9.3, TensorRT 10.3, and OpenCV 4.8 are present in the exact versions.

7. Conclusion (recommendations)

The Jetson AGX Orin 64 GB developer kit is fully configured as advertised. The verification steps can be automated via the script provided in Section 8, making it suitable for CI pipelines, regression testing, or compliance reporting.

Note: The host name (ubuntu) and host display (NVIDIA Jetson AGX Orin Develop) are placeholders; you can replace them with the actual values shown by hostname and lscpu.

8. Automated script – `jetson_sysinfo.sh`

#!/usr/bin/env bash

# Simple system summary for NVIDIA Jetson AGX Orin

# Hardware
HW_MODEL=$(tr -d '\0' </sys/firmware/devicetree/base/model 2>/dev/null)
HW_MODEL=${HW_MODEL:-"Unknown"}

# OS
OS_DESC=$(lsb_release -d 2>/dev/null | cut -f2-)
OS_DESC=${OS_DESC:-$(grep -E '^PRETTY_NAME=' /etc/os-release 2>/dev/null | cut -d= -f2 | tr -d '"')}
ARCH=$(uname -m)
HOST=$(hostname)
KERNEL=$(uname -r)

# CPU
CPU_MODEL=$(grep -m1 "model name" /proc/cpuinfo 2>/dev/null | cut -d: -f2- | xargs)
[ -z "$CPU_MODEL" ] && CPU_MODEL=$(lscpu | awk -F: '/Model name/ {print $2}' | xargs)
CPU_CORES=$(nproc)
CPU_MAX=$(lscpu | awk -F: '/CPU max MHz/ {gsub(/ /,"",$2); print $2}')
CPU_MIN=$(lscpu | awk -F: '/CPU min MHz/ {gsub(/ /,"",$2); print $2}')

# Jetson / JetPack / L4T
L4T_CORE=$(dpkg-query --show nvidia-l4t-core 2>/dev/null | awk '{print $2}')
JP_SRC=$(apt-cache show nvidia-jetpack 2>/dev/null | awk -F': ' '/^Source:/ {print $2; exit}')
JP_VER=$(apt-cache show nvidia-jetpack 2>/dev/null | awk -F': ' '/^Version:/ {print $2; exit}')
JP_ARCH=$(apt-cache show nvidia-jetpack 2>/dev/null | awk -F': ' '/^Architecture:/ {print $2; exit}')
JP_MAINT=$(apt-cache show nvidia-jetpack 2>/dev/null | awk -F': ' '/^Maintainer:/ {print $2; exit}')
JP_DEPS=$(apt-cache show nvidia-jetpack 2>/dev/null | awk -F': ' '/^Depends:/ {print $2; exit}')
NVREL=$(grep -m1 '^# R' /etc/nv_tegra_release 2>/dev/null | sed 's/^# //')

# CUDA
NVCC_VER=$(nvcc --version 2>/dev/null | tail -n1)

# cuDNN
CUDNN_LINE=$(dpkg -l | awk '/libcudnn[0-9]-cuda-12/ {print $2" " $3; exit}')
[ -z "$CUDNN_LINE" ] && CUDNN_LINE="not found"

# TensorRT
TRT_LINE=$(dpkg -l | awk '/^ii  tensorrt / {print $2" " $3; exit}')
[ -z "$TRT_LINE" ] && TRT_LINE="not found"

# OpenCV
OPENCV_VER=$(python3 -c "import cv2; print(cv2.__version__)" 2>/dev/null || echo "not found")

# Print summary
echo "Hardware: ${HW_MODEL} 64GB"
echo "OS: ${OS_DESC} ${ARCH}"
echo "Host: ${HOST}"
echo "Kernel: ${KERNEL}"
echo "CPU: ${CPU_MODEL} (${CPU_CORES}) @ ${CPU_MAX%.*}MHz"
echo "CPU max MHz: ${CPU_MAX}"
echo "CPU min MHz: ${CPU_MIN}"
echo "Memory: ${MEM_LINE}"
echo "nvidia-l4t-core: ${L4T_CORE}"
[ -n "$NVREL" ] && echo "L4T release: ${NVREL}"
echo "Package: nvidia-jetpack"
echo "Source: ${JP_SRC}"
echo "Version: ${JP_VER}"
echo "Architecture: ${JP_ARCH}"
echo "Maintainer: ${JP_MAINT}"
echo "Depends: ${JP_DEPS}"
echo "nvcc: NVIDIA (R) Cuda compiler driver"
echo "${NVCC_VER}"
echo "cuDNN: ${CUDNN_LINE}"
echo "OpenCV Version: ${OPENCV_VER}"
echo "TensorRT: ${TRT_LINE}"

How to use

nano jetson_sysinfo.sh          # paste the script
chmod +x jetson_sysinfo.sh      # make it executable
./jetson_sysinfo.sh             # run – prints a compact summary

The script prints exactly the same one‑line summary shown in Section 6, making sharing as proof of configuration trivial.

Enabling Maximum Performance Mode on NVIDIA Jetson AGX Orin 64 GB

Sergio Andres Usma — Sun, 05 Apr 2026 15:58:34 +0000

Abstract

This document explains how to configure an NVIDIA Jetson AGX Orin 64 GB Developer Kit running Ubuntu 22.04.5 LTS and JetPack 6.2.2 to operate in maximum performance mode for AI workloads, especially LLM inference. It describes how to select the MAXN power mode, lock system clocks at their highest frequencies, and verify that the configuration is correctly applied with built-in NVIDIA tools and simple benchmarks. The tutorial targets users who want reproducible, high-throughput inference on a Jetson AGX Orin while retaining awareness of thermal and power constraints.

It documents the practical impact of enabling MAXN and jetson_clocks, showing how GPU frequency and token generation throughput can increase roughly threefold compared to default settings. The guide also covers how to persist these settings using a systemd service so that the device consistently boots into a high-performance state suitable for heavy AI workloads. Where relevant, it notes expected frequency values and normal operating temperatures for the Jetson AGX Orin platform.

The purpose of this tutorial is to serve as a reusable reference for configuring maximum performance on Jetson-based AI systems, integrated into a larger workflow that includes swap configuration and tool installation for LLM workloads. Readers with basic Linux and Jetson familiarity can follow step-by-step commands to prepare the device, validate the configuration, and understand when to switch between performance and power-saving modes.

1. Hardware and Software Environment

Your system is an NVIDIA Jetson AGX Orin Developer Kit 64 GB running Ubuntu 22.04.5 LTS (aarch64) with JetPack 6.2.2, CUDA 12.6, cuDNN 9.3.0, OpenCV 4.8.0, and TensorRT 10.3.0.30 installed. The CPU is an ARMv8 12-core processor with a maximum clock around 2.2 GHz, and the board exposes NVIDIA’s nvpmodel and jetson_clocks tools for power and clock management.

According to NVIDIA’s specifications, the Jetson AGX Orin 64 GB configuration can achieve up to 275 TOPS when configured in MAXN mode with clocks locked to their maximum frequencies. This tutorial assumes shell access with sudo privileges and that NVIDIA JetPack components are correctly installed from the nvidia-jetpack meta-package.

2. Why Maximum Performance Matters for AI

By default, without MAXN mode and jetson_clocks, the Jetson AGX Orin keeps GPU frequencies around 600 MHz to maintain thermal and power safety margins. Under these conservative defaults, a 7B LLM typically reaches only about 8 tokens per second during inference, which limits interactivity and throughput.

When MAXN and jetson_clocks are enabled, the GPU can run at approximately 1300 MHz, and end-to-end LLM inference throughput can increase to roughly 18–25 tokens per second on the same 7B model. This represents about a 3x performance improvement and makes interactive LLM usage and larger batch workloads more practical on the device.

3. Inspecting and Selecting Power Modes

Before changing anything, check the current power mode:

sudo nvpmodel -q

The command prints the active power mode, and the integer at the bottom of the output is the current mode ID (for example, 0 for MAXN). This lets you confirm whether the system already runs in MAXN or a more restrictive power profile.

To see all available power modes for the Jetson AGX Orin 64 GB under JetPack 6.2, run:

sudo nvpmodel -q --verbose | grep -A1 "MODE_NAME"

On this platform, the mode table typically looks like:

Mode ID	Name	TDP	CPU cores active	GPU max freq
0	MAXN	No limit (~60 W)	12	1300 MHz
1	MODE_50W	50 W	12	1100 MHz
2	MODE_30W	30 W	8	854 MHz
3	MODE_15W	15 W	4	612 MHz

Use mode ID 0 (MAXN) for the high-performance configuration described here.

4. Enabling MAXN Mode and Locking Clocks

To switch the Jetson into MAXN mode, run:

sudo nvpmodel -m 0

This update is written into /etc/nvpmodel.conf and therefore persists across reboots until you select a different mode. After this step, the Jetson operates under the highest power budget supported by its cooling solution, which is ideal for compute-heavy AI tasks.

Next, lock all clocks (CPU, GPU, and memory bus) to their maximum frequencies:

sudo jetson_clocks

This command is temporary and resets after each reboot, so it must be re-applied or automated to persist. Once applied, the system stops using dynamic frequency scaling and instead pins frequencies to their highest supported values for maximum compute performance.

5. Verifying Power Mode and Clock Frequencies

To confirm that MAXN is active and clocks are locked, run:

# Confirm power mode
sudo nvpmodel -q

# Check GPU and CPU frequencies
sudo jetson_clocks --show

The jetson_clocks --show output should include lines similar to:

CPU Cluster Switching: Disabled
cpu0: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 ...
GPU MinFreq=306000000 MaxFreq=1300500000 CurrentFreq=1300500000
EMC MinFreq=204000000 MaxFreq=3199000000 CurrentFreq=3199000000

For a correct configuration, CurrentFreq should match MaxFreq for CPU, GPU, and EMC entries, indicating that frequencies are pinned at their maximums. If you see lower current frequencies, reapply jetson_clocks or investigate thermal throttling conditions.

6. Making jetson_clocks Persistent with systemd

To ensure jetson_clocks runs automatically at boot, first try enabling the built-in service:

sudo systemctl enable nvargus-daemon 2>/dev/null || true
sudo systemctl enable jetson_clocks 2>/dev/null || echo "Service not found, creating..."

On some JetPack versions the jetson_clocks service may not exist, in which case you can create a custom systemd unit file. The following commands define such a service, reload systemd, and enable it:

sudo tee /etc/systemd/system/jetson_clocks.service > /dev/null << 'EOF'
[Unit]
Description=Lock Jetson clocks at maximum frequency
After=multi-user.target

[Service]
Type=oneshot
ExecStart=/usr/bin/jetson_clocks
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable jetson_clocks
sudo systemctl start jetson_clocks
sudo systemctl status jetson_clocks

Afterward, every boot should automatically apply jetson_clocks, and systemctl status jetson_clocks should report the service as active. This eliminates the need to manually run the command after each restart while keeping the configuration transparent and reversible via systemd.

7. Quick Performance Benchmark with LLM Inference

Once MAXN and jetson_clocks are active, you can validate real-world AI performance using an LLM benchmark. If you have an Ollama container running (as configured in a later phase of your workflow), execute:

# Benchmark: time to generate 100 tokens with a 3B model
docker exec ollama ollama run llama3.2 \
  --verbose \
  "Write a 100-word story about a robot" 2>&1 | grep -E "eval rate|tokens/s"

In MAXN mode on the Jetson AGX Orin, a llama3.2 3B Q4_K_M model is expected to reach around 25–40 tokens per second, significantly higher than default power modes. If observed throughput is substantially lower, recheck power mode, clock locking, and ensure the system is not thermally throttling or swapping heavily.

8. Monitoring Power, Temperature, and Thermal Safety

While models are running, monitor system health from a second terminal:

# Option A: tegrastats (every second)
tegrastats --interval 1000

# Option B: jtop (interactive dashboard)
jtop

In tegrastats, key fields include GPU@XXX°C for GPU temperature (targeting below about 85°C under sustained load), POM_5V_GPU Xm/Ym for GPU power draw in milliwatts, and Tboard@XXX for board temperature. MAXN mode is designed to work within the active cooling capabilities of the AGX Orin module, but blocked vents or poor airflow can still cause throttling.

Typical thermal ranges under continuous AI workloads are:

Component	Normal	Throttle starts	Emergency
GPU	50–75 °C	~85 °C	~95 °C
CPU	45–70 °C	~85 °C	~95 °C
Board	40–60 °C	—	—

If tegrastats shows throttle=1, improve ventilation or reduce workload intensity until the system stabilizes.

9. Choosing Performance vs Power-Saving Modes

Depending on your workload, you may want to switch between MAXN and more efficient modes. Common scenarios include:

Situation	Recommended mode	Command
LLM inference (7B–70B)	MAXN (0)	`sudo nvpmodel -m 0 && sudo jetson_clocks`
Vision / video processing	MAXN (0)	same as above
Compiling code (e.g., LLM)	MAXN (0)	same as above
Idle / light development	MODE_30W (2)	`sudo nvpmodel -m 2`
Background low-power tasks	MODE_15W (3)	`sudo nvpmodel -m 3`

Switching modes only changes the power envelope, while jetson_clocks controls frequency locking; together they give fine-grained control over performance versus efficiency. You can integrate these commands into your own scripts to toggle modes depending on job type or time of day.

10. Practical Outcomes

MAXN mode and jetson_clocks are enabled on the Jetson AGX Orin 64 GB, with GPU, CPU, and EMC frequencies pinned at their maximums for AI workloads.
A systemd service (built-in or custom) ensures jetson_clocks runs at boot so performance is consistent across reboots.
Simple LLM benchmarks confirm real-world throughput improvements (on the order of 3x token/sec) compared to default power modes.
Continuous monitoring with tegrastats or jtop provides visibility into temperature, power draw, and potential thermal throttling.
Clear commands exist to switch between high-performance and power-saving modes depending on workload requirements.

11. Conclusion

Configuring the Jetson AGX Orin 64 GB into MAXN mode with locked clocks is a necessary step to realize the board’s full 275 TOPS potential for LLM inference and other GPU-intensive workloads. The combination of nvpmodel for power profiles and jetson_clocks for frequency locking provides deterministic performance while staying within the cooling design limits of the developer kit.

With the steps in this tutorial, you can reproducibly enable, verify, and persist maximum performance settings, then validate them using practical AI benchmarks and runtime telemetry tools. In a larger workflow, this configuration forms the foundation for subsequent tasks such as creating swap space for very large models and installing build tools for optimized inference frameworks.

Exploratory Installation of Unsloth on NVIDIA Jetson AGX Orin 64 GB

Sergio Andres Usma — Sun, 05 Apr 2026 14:52:21 +0000

Abstract

This report documents an exploratory attempt to install and run Unsloth (including Unsloth Studio) on an NVIDIA Jetson AGX Orin 64 GB using a Docker-based workflow with dustynv/l4t-ml:r36.4.0 as the base image.

The process successfully validated GPU-accelerated PyTorch and Unsloth’s core Python package on Jetson, but exposed substantial friction and incompatibilities in getting Unsloth Studio’s full stack (Studio backend, frontend, Triton/TorchInductor/TorchAo dependencies, and custom virtual environment) to run reliably on this ARM-based edge platform.

The goal of this write-up is to provide a precise technical account so that other practitioners (and the Unsloth team) can (a) reproduce or avoid the same pitfalls, and (b) better assess the current suitability of Unsloth Studio for Jetson-class devices.

1. Hardware and Software Environment

The experiments were conducted on the following platform:

Device: NVIDIA Jetson AGX Orin Developer Kit (64 GB)
OS: Ubuntu 22.04.5 LTS, aarch64
JetPack / L4T: JetPack 6.2.2, L4T 36.5.0
CUDA: 12.6 (nvcc 12.6.68)
cuDNN: 9.3.0
TensorRT: 10.3.0
Docker: Engine with NVIDIA Container Runtime enabled (--runtime=nvidia)
Base ML image: dustynv/l4t-ml:r36.4.0 (from Jetson Containers), which provides:
- PyTorch compiled for Jetson (aarch64) with CUDA and TensorRT integration
- JupyterLab and common ML tooling

Host-side persistent storage for this project was centralized under:

~/unsloth/
  build/      # Dockerfile and build context
  work/       # notebooks, datasets, outputs
  cache/      # general cache inside the container
  hf/         # Hugging Face cache
  jupyter/    # Jupyter config
  ssh/        # SSH keys/config (optional)

This layout was bind-mounted into the container to ensure persistence across container rebuilds.

2. Docker Image Construction

2.1 Base Dockerfile

The starting point was a custom image layered on top of dustynv/l4t-ml:r36.4.0:

FROM dustynv/l4t-ml:r36.4.0

ENV DEBIAN_FRONTEND=noninteractive \
    PIP_NO_CACHE_DIR=1 \
    PYTHONUNBUFFERED=1 \
    SHELL=/bin/bash \
    JUPYTER_PORT=8888 \
    STUDIO_PORT=8000 \
    WORKSPACE=/workspace \
    HF_HOME=/workspace/.cache/huggingface \
    TRANSFORMERS_CACHE=/workspace/.cache/huggingface \
    HUGGINGFACE_HUB_CACHE=/workspace/.cache/huggingface

USER root

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl git wget ca-certificates build-essential pkg-config \
    python3-pip python3-dev python3-venv \
    openssh-server sudo nano htop tmux \
    libopenblas-dev libssl-dev libffi-dev \
    && rm -rf /var/lib/apt/lists/*

RUN mkdir -p /var/run/sshd /workspace/work /workspace/.cache/huggingface /root/.jupyter

RUN python3 -m pip install --upgrade pip setuptools wheel

# Remove Jetson-specific custom pip indexes to avoid transient outages
RUN python3 -m pip config unset global.index-url || true && \
    python3 -m pip config unset global.extra-index-url || true

# Generic Python dependencies via PyPI
RUN PIP_INDEX_URL=https://pypi.org/simple python3 -m pip install \
    fastapi "uvicorn[standard]" gradio \
    accelerate transformers peft trl datasets sentencepiece protobuf safetensors \
    huggingface_hub

# Install Unsloth (core + zoo) from GitHub/PyPI
RUN PIP_INDEX_URL=https://pypi.org/simple python3 -m pip install \
    "unsloth @ git+https://github.com/unslothai/unsloth.git" \
    "unsloth-zoo @ git+https://github.com/unslothai/unsloth.git" || true

# Optionally attempt bitsandbytes (may be fragile on Jetson)
RUN PIP_INDEX_URL=https://pypi.org/simple python3 -m pip install bitsandbytes || true

WORKDIR /workspace
EXPOSE 8000 8888 22

CMD ["/bin/bash"]

Key design choices:

Reuse NVIDIA’s l4t-ml stack instead of installing PyTorch/TensorRT manually, since it is tuned for Jetson.
Explicitly unset custom Jetson pip indexes before installing Unsloth, to avoid failures due to unavailable Jetson-specific mirrors while installing generic packages (e.g. fastapi).
Install Unsloth via GitHub (or PyPI) rather than using the x86-oriented Docker image unsloth/unsloth.

The image was built with:

cd ~/unsloth/build
sudo docker build --no-cache -t local/unsloth-studio:jetson-l4tml-r36.4.0 .

3. Container Runtime and GPU Validation

A persistent container was created with host networking and bind mounts:

sudo docker run -d \
  --name unsloth-studio \
  --restart unless-stopped \
  --runtime nvidia \
  --network host \
  --shm-size=16g \
  -e HF_HOME=/workspace/.cache/huggingface \
  -e TRANSFORMERS_CACHE=/workspace/.cache/huggingface \
  -e HUGGINGFACE_HUB_CACHE=/workspace/.cache/huggingface \
  -v ~/unsloth/work:/workspace/work \
  -v ~/unsloth/cache:/workspace/.cache \
  -v ~/unsloth/hf:/root/.cache/huggingface \
  -v ~/unsloth/jupyter:/root/.jupyter \
  -v ~/unsloth/ssh:/root/.ssh \
  local/unsloth-studio:jetson-l4tml-r36.4.0 \
  tail -f /dev/null

Inside the container, GPU support was verified with:

python3 -c "import torch;
print(torch.__version__);
print(torch.cuda.is_available());
print(torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'no cuda')"

This confirmed:

torch version 2.6.0 (from the l4t-ml stack),
CUDA available,
device name reported as “Orin”.

Thus, the base ML environment inside the container was correctly accelerated on Jetson.

4. Installing Unsloth Core

Within the container:

python3 -c "import unsloth; print('unsloth ok')"
unsloth --help

The CLI output showed the main Unsloth commands:

train
inference
export
list-checkpoints
studio (subcommand group)

However, importing Unsloth triggered a warning stacktrace related to Triton, TorchInductor, and TorchAo:

ImportError: cannot import name 'AttrsDescriptor' from triton.compiler.compiler
Errors inside torch._inductor.runtime.hints and torchao.quantization

This indicates that parts of the current Unsloth stack assume a Triton/TorchInductor/TorchAo configuration aligned with x86_64 desktop/server builds of PyTorch, which is not trivially compatible with the Jetson-specific PyTorch build shipping in l4t-ml.

Despite these warnings, the CLI remained usable for basic commands, and GPU acceleration for standard PyTorch operations was intact.

5. Attempting to Enable Unsloth Studio

5.1 CLI-Level Status

The unsloth studio subcommand was present:

unsloth studio --help

showed options such as --host, --port, --frontend, and subcommands:

stop
update
reset-password

Attempting to start Studio directly:

unsloth studio --host 0.0.0.0 --port 8000

returned:

Studio not set up. Run install.sh first.

This implies that Studio expects an auxiliary installation step that sets up its environment (frontend, backend, and venv).

5.2 Running `unsloth studio setup`

Unsloth documentation describes a developer mode where Studio is installed via uv and a dedicated virtual environment.

Following this pattern, the command:

unsloth studio setup

produced:

Successful installation of nvm, Node LTS, and bun
Successful build of the frontend (“frontend built”)
But then:

python venv not found at /root/.unsloth/studio/unsloth_studio
Run install.sh first to create the environment:
  curl -fsSL https://unsloth.ai/install.sh | sh

Thus, the CLI expects a virtual environment under /root/.unsloth/studio/unsloth_studio that appears to be normally created by the official install.sh script.

5.3 Manual Creation of the Studio Virtual Environment

Rather than relying on install.sh (which is tuned for other platforms and may interfere with the Jetson-specific PyTorch/Triton stack), a manual venv was created:

cd /root/.unsloth/studio
uv venv unsloth_studio --python 3.10
source /root/.unsloth/studio/unsloth_studio/bin/activate

uv pip install --index-url https://pypi.org/simple unsloth

This installed Unsloth (and a complete stack of dependencies) into the venv, including:

torch, torchao, triton
transformers, accelerate, peft, trl
bitsandbytes
unsloth, unsloth-zoo

Within the venv, unsloth studio -H 0.0.0.0 -p 8000 still failed due to missing backend dependencies (structlog), which were then installed.

However, repeated attempts to start Studio continued to reveal issues:

ModuleNotFoundError: No module named 'structlog' (due to pip confusion between global and venv environments)
Friction in adding pip to the venv (pip not present or not found via python -m pip)
A recurring tension between the uv-managed environment and the classical pip expectations coming from Studio’s backend modules.

Ultimately, even after installing the necessary Python packages, the CLI still treated Studio as “not set up” and insisted on running the global install.sh script.

6. Failure Modes and Root Causes

The main failure modes observed were:

Triton / TorchInductor / TorchAo incompatibilities
- Errors when importing Unsloth related to AttrsDescriptor in Triton and TorchInductor.
- These components are not officially supported or tuned for the Jetson-specific PyTorch build, causing runtime import and registration issues.
Studio’s tight coupling to its own venv and installer
- Studio expects a very particular environment layout under ~/.unsloth/studio/unsloth_studio created by install.sh.
- Deviating from the installer (e.g., manual or uv-only installation) leads to missing venv markers, which the CLI interprets as “Studio not set up.”
Tooling friction on Jetson (uv + venv + pip)
- The combination of uv-managed environments with a system Python and Docker base image that already has a global pip led to situations where:
  - The venv had no pip initially.
  - python -m ensurepip installed pip globally rather than into the venv.
  - The actual pip used to install backend dependencies was the global one, leaving the venv incomplete.
Mismatch with Jetson Containers philosophy
- Jetson Containers and l4t-ml are built around Nvidia’s optimized PyTorch/TensorRT stacks, while Unsloth Studio’s modern pipeline assumes desktop/server-class Triton and TorchInductor configurations.
- This leads to a mismatch that is non-trivial to reconcile in a maintainable way.

7. Practical Outcomes

Despite the failure to get Unsloth Studio fully operational, the following outcomes were achieved:

A validated GPU-accelerated Unsloth core environment on Jetson:
- unsloth CLI installed and usable.
- PyTorch 2.6.0 with CUDA on Orin working correctly.
A reusable Docker-based ML devbox (local/unsloth-studio:jetson-l4tml-r36.4.0) with:
- A clear persistent directory layout (~/unsloth).
- Host networking and shared volumes suitable for integration with other Jetson Containers (e.g., llama.cpp, vLLM, NanoLLM, llama-factory).
Empirical evidence that, as of this experiment, Unsloth Studio is not yet a drop-in web UI solution for Jetson AGX Orin, due to:
- Triton/TorchInductor/TorchAo assumptions, and
- Strong coupling to the install.sh-managed environment.

8. Recommendations for Jetson Practitioners

For current Jetson AGX Orin users:

Use Unsloth core selectively
- Unsloth’s Python API and CLI can still be valuable for fine-tuning/export workflows that do not rely heavily on Triton/TorchInductor-specific optimizations.
- Prefer using the Jetson-optimized PyTorch from l4t-ml and be cautious with features that depend on TorchInductor/Triton.
Rely on Jetson Containers for serving and fine-tuning
- For serving and fine-tuning large models on Jetson, the containers in the Jetson Containers ecosystem (llama.cpp, vLLM, MLC, TensorRT-LLM, NanoLLM, llama-factory) are significantly more mature and better integrated with JetPack and L4T.
Treat Unsloth Studio on Jetson as experimental
- Until there is first-class ARM/Jetson support (or a documented variant of install.sh and Studio’s backend explicitly targeting Jetson), Studio should be considered an experimental integration on this hardware.

9. Suggestions for the Unsloth Team

Based on this experience, the following changes would materially improve the viability of Unsloth Studio on Jetson and similar edge platforms:

Documented “headless / no-Triton” mode
- A configuration profile that can disable or bypass TorchInductor/Triton/TorchAo, relying purely on standard PyTorch kernels when running on unsupported architectures such as Jetson.
Explicit ARM/Jetson support statement and checks
- Clear statements in the documentation regarding ARM/aarch64 support status, with runtime checks that either:
  - Enable a safe, reduced feature set, or
  - Fail fast with a clear, actionable message.
Studio installation mode for preexisting Python stacks
- A variant of install.sh or studio setup that:
  - Can attach to an existing PyTorch environment (e.g., Jetson’s l4t-ml), and
  - Creates only the additional Studio-specific venv/backend/frontend without attempting to reconfigure PyTorch or Triton.
Minimal dependency profile for Studio backend
- A smaller “core backend” dependency set for Studio that avoids complex quantization stacks and heavy compiler integrations when running in constrained or embedded environments.

10. Conclusion

The experiment demonstrates that:

Installing Unsloth core on Jetson AGX Orin via a Dockerized l4t-ml base image is feasible, and the resulting environment is usable for GPU-accelerated LLM workflows.
However, enabling Unsloth Studio—the full web UI for training and serving—on Jetson currently encounters significant hurdles due to the interaction between Triton/TorchInductor, TorchAo, uv-managed venvs, and the assumptions baked into install.sh.

From a practical standpoint, Jetson users are better served today by combining Unsloth core (where useful) with the existing Jetson Containers ecosystem, while treating Unsloth Studio as an experimental component on this hardware.

From a community and engineering perspective, this experiment highlights concrete areas where incremental changes and documentation from the Unsloth team could unlock a powerful edge deployment story on Jetson-class devices.

DEV Community: Sergio Andres Usma

Creating a 50 GB Swap File on Jetson AGX Orin (Root on NVMe)

Abstract

1. Hardware and Software Environment

2. Swap Location Strategy

3. Step-by-Step Swap File Creation

3.1 Check Devices and Free Space

3.2 Create the Swap File

3.3 Secure and Format the Swap File

3.4 Enable the Swap File

4. Making Swap Persistent Across Reboots

5. Tuning Swappiness and zram for LLM Workloads

5.1 Adjust Swappiness and Cache Pressure

6. Relationship Between zram and /swapfile

7. Removing or Reconfiguring the Swap File

8. Practical Outcomes

9. Conclusions

Check NVIDIA Jetson AGX Orin Specifications

Abstract

1. Hardware and Software Environment

1.1 Jetson board identification

1.2 Operating system (Ubuntu 22.04.5 LTS)

2. CPU details

3. Memory

4. JetPack and Jetson Linux (L4T) versions

4.1 JetPack meta‑package

4.2 L4T release (Jetson Linux R36.5)

5. CUDA, cuDNN, TensorRT, OpenCV

5.1 CUDA toolkit

5.2 cuDNN

5.3 TensorRT

5.4 OpenCV

6. Practical Outcomes (what worked)

7. Conclusion (recommendations)

8. Automated script – jetson_sysinfo.sh

Enabling Maximum Performance Mode on NVIDIA Jetson AGX Orin 64 GB

Abstract

1. Hardware and Software Environment

2. Why Maximum Performance Matters for AI

3. Inspecting and Selecting Power Modes

4. Enabling MAXN Mode and Locking Clocks

5. Verifying Power Mode and Clock Frequencies

6. Making jetson_clocks Persistent with systemd

7. Quick Performance Benchmark with LLM Inference

8. Monitoring Power, Temperature, and Thermal Safety

9. Choosing Performance vs Power-Saving Modes

10. Practical Outcomes

11. Conclusion

Exploratory Installation of Unsloth on NVIDIA Jetson AGX Orin 64 GB

Abstract

1. Hardware and Software Environment

2. Docker Image Construction

2.1 Base Dockerfile

3. Container Runtime and GPU Validation

4. Installing Unsloth Core

5. Attempting to Enable Unsloth Studio

5.1 CLI-Level Status

5.2 Running unsloth studio setup

5.3 Manual Creation of the Studio Virtual Environment

6. Failure Modes and Root Causes

7. Practical Outcomes

8. Recommendations for Jetson Practitioners

9. Suggestions for the Unsloth Team

10. Conclusion

8. Automated script – `jetson_sysinfo.sh`

5.2 Running `unsloth studio setup`