DEV Community

Cover image for Panduan Teknikal: Compile llama.cpp di Debian 12/13 dan Cross Compile ARM64
hardyweb
hardyweb

Posted on

Panduan Teknikal: Compile llama.cpp di Debian 12/13 dan Cross Compile ARM64

#ai #linux #llm #tutorial

1. Pengenalan

llama.cpp ialah runtime inference LLM berasaskan C/C++ yang popular kerana ringan, pantas, dan sesuai untuk menjalankan model GGUF secara local. Ia boleh digunakan pada:

Server x86_64
Workstation Linux
Mini PC
Raspberry Pi
Orange Pi
SBC ARM64
Container Linux

Dalam deployment sebenar, terdapat dua pendekatan utama:

Native build
Compile terus pada mesin yang akan menjalankan llama.cpp.
Cross compile
Compile pada mesin lebih laju (contohnya PC x86_64), tetapi menghasilkan binary untuk platform lain (contohnya ARM64 Orange Pi).

Bahagian 1 — Persediaan Debian 12/13

1.1 Install dependency asas

sudo apt update

sudo apt install -y \
    git \
    build-essential \
    cmake \
    ninja-build \
    pkg-config
Enter fullscreen mode Exit fullscreen mode

Komponen utama:

Package Fungsi
git Ambil source code
build-essential GCC, G++, make
cmake Build configuration
ninja-build Build engine lebih pantas
pkg-config Cari library dependency

Bahagian 2 — Clone llama.cpp

git clone https://github.com/ggml-org/llama.cpp.git

cd llama.cpp
Enter fullscreen mode Exit fullscreen mode

Semak versi:

git log -1 --oneline
Enter fullscreen mode Exit fullscreen mode

Bahagian 3 — Compile Native (Mesin Sama)

Contoh:

Debian 12/13 x86_64
Debian ARM64
Orange Pi
Raspberry Pi

3.1 Configure CMake

Build menggunakan Ninja:

cmake -B build \
    -G Ninja \
    -DCMAKE_BUILD_TYPE=Release
3.2 Compile
ninja -C build -j$(nproc)
Enter fullscreen mode Exit fullscreen mode

atau:

cmake --build build
Enter fullscreen mode Exit fullscreen mode


plaintext
3.3 Hasil build

Semak:

ls build/bin
Enter fullscreen mode Exit fullscreen mode


plaintext
Contoh:

llama-cli
llama-server
llama-bench
llama-perplexity
Enter fullscreen mode Exit fullscreen mode


shell

Bahagian 4 — Enable OpenBLAS (Pilihan)

OpenBLAS boleh membantu operasi matrix CPU.

Install:

sudo apt install libopenblas-dev
Enter fullscreen mode Exit fullscreen mode


cmake
Build:

cmake -B build \
    -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DGGML_BLAS=ON \
    -DGGML_BLAS_VENDOR=OpenBLAS
Enter fullscreen mode Exit fullscreen mode


shell
Kemudian:

ninja -C build
Enter fullscreen mode Exit fullscreen mode

Nota Penting: CMake Cache

Jika pernah configure dengan:

-DGGML_BLAS=ON
Enter fullscreen mode Exit fullscreen mode

kemudian buang option tersebut, CMake masih menyimpan konfigurasi lama.

Contoh masalah:

BLAS not found
missing: BLAS_LIBRARIES

Penyelesaian:

rm -rf build
Enter fullscreen mode Exit fullscreen mode

Kemudian configure semula.

Sentiasa ingat:

CMakeCache.txt menyimpan konfigurasi lama.
Bahagian 5 — Cross Compile x86_64 → ARM64

Contoh:

PC Debian 12 x86_64
|
|
v
Orange Pi ARM64

Kelebihan:

Compile lebih cepat
Tidak membebankan SBC
Sesuai untuk production image

5.1 Install ARM64 cross compiler

sudo apt install -y \
    gcc-12-aarch64-linux--gnu\
    g++-12-aarch64-linux-gnu
Enter fullscreen mode Exit fullscreen mode 
sudo apt install -y \
    gcc-13-aarch64-linux--gnu\
    g++-13-aarch64-linux-gnu
Enter fullscreen mode Exit fullscreen mode

Semak:

aarch64-linux-gnu-gcc --version
Enter fullscreen mode Exit fullscreen mode

5.2 Configure cross build

Bersihkan dahulu:

rm -rf build-arm
Enter fullscreen mode Exit fullscreen mode

Kemudian:

cmake -B build-arm \
    -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_SYSTEM_NAME=Linux \
    -DCMAKE_SYSTEM_PROCESSOR=aarch64 \
    -DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \
    -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++
5.3 Compile
ninja -C build-arm -j$(nproc)
Enter fullscreen mode Exit fullscreen mode

Hasil:

ls build-arm/bin
Enter fullscreen mode Exit fullscreen mode

Bahagian 6 — Semak Architecture Binary

Gunakan:

file build-arm/bin/llama-server
Enter fullscreen mode Exit fullscreen mode

Contoh output berjaya:

ELF 64-bit LSB pie executable,
ARM aarch64,
dynamically linked
Enter fullscreen mode Exit fullscreen mode

Maksud:

Output Maksud
ELF 64-bit Binary 64-bit
ARM aarch64 Untuk ARM64
dynamically linked Perlukan shared library
PIE executable Linux security hardening
Bahagian 7 — Semak Dependency .so
Jangan guna ldd untuk cross binary

Jika compile ARM64 tetapi check pada PC x86:

ldd llama-server
Enter fullscreen mode Exit fullscreen mode

boleh gagal:

not a dynamic executable
Enter fullscreen mode Exit fullscreen mode

Sebab:

PC:

x86_64 loader

Binary:

ARM64 loader
Gunakan readelf
aarch64-linux-gnu-readelf \
-d build-arm/bin/llama-server | grep NEEDED

Contoh:

Shared library: [libllama.so]
Shared library: [libggml.so]
Shared library: [libstdc++.so.6]
Enter fullscreen mode Exit fullscreen mode

Cari semua .so

find build-arm -name "*.so"
Enter fullscreen mode Exit fullscreen mode

Contoh:

libllama.so
libggml.so
libggml-base.so
libggml-cpu.so
Enter fullscreen mode Exit fullscreen mode

Semak architecture:

file build-arm/bin/*.so
Enter fullscreen mode Exit fullscreen mode

Output:

ARM aarch64
Bahagian 8 — Dynamic vs Static Binary
Enter fullscreen mode Exit fullscreen mode

Semak:

file llama-server
Enter fullscreen mode Exit fullscreen mode

Contoh dynamic:

dynamically linked
Enter fullscreen mode Exit fullscreen mode

Perlu:

lib*.so

Contoh static:

statically linked

Tidak perlu .so.

Bahagian 9 — Installation ke Linux
Pilihan standard

Binary:

/usr/local/bin
Enter fullscreen mode Exit fullscreen mode

Library:

/usr/local/lib
Enter fullscreen mode Exit fullscreen mode

Contoh:

sudo cp llama-server /usr/local/bin/
sudo cp llama-cli /usr/local/bin/

sudo cp *.so /usr/local/lib/

sudo ldconfig
Enter fullscreen mode Exit fullscreen mode

Pilihan appliance / embedded

Untuk SBC:

/opt/llama.cpp/

    llama-server
    llama-cli
    libllama.so
    libggml.so
Enter fullscreen mode Exit fullscreen mode

Kemudian:

export LD_LIBRARY_PATH=/opt/llama.cpp
Enter fullscreen mode Exit fullscreen mode

Sesuai untuk:

Orange Pi
kiosk AI
edge inference node

Bahagian 10 — Deploy ke Orange Pi

Copy:

scp build-arm/bin/llama-server \
orangepi:/usr/local/bin/
Enter fullscreen mode Exit fullscreen mode 
scp build-arm/bin/llama-cli \
orangepi:/usr/local/bin/
Enter fullscreen mode Exit fullscreen mode

Jika perlu:

scp build-arm/bin/*.so \
orangepi:/usr/local/lib/
Enter fullscreen mode Exit fullscreen mode

Pada Orange Pi:

sudo ldconfig
Enter fullscreen mode Exit fullscreen mode

Semak:

uname -m
Enter fullscreen mode Exit fullscreen mode

Expected:

aarch64
Enter fullscreen mode Exit fullscreen mode

Bahagian 11 — Cadangan Production Architecture

Untuk sistem AI agent:

+----------------+
| Go Agent |
| Tool Router |
+-------+--------+
|
|
HTTP API
|
v
+----------------+
| llama-server |
| llama.cpp |
+----------------+
|
|
GGUF
Model

Kelebihan:

Go agent tidak perlu embed model
Model boleh tukar tanpa rebuild
llama.cpp boleh upgrade sendiri
Mudah scale ke banyak node
Kesimpulan

Workflow yang stabil:

Native
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release

ninja -C build
Cross Compile ARM64
sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu

rm -rf build-arm

cmake -B build-arm \
-G Ninja \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSTEM_PROCESSOR=aarch64 \
-DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc \
-DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++

ninja -C build-arm
Verification
file llama-server

aarch64-linux-gnu-readelf -d llama-server | grep NEEDED

find . -name "*.so"

Dengan proses ini, satu mesin Debian 12/13 boleh menjadi build server untuk menghasilkan node AI ARM64 seperti Orange Pi, Raspberry Pi, atau edge inference appliance.

Top comments (0)