DEV Community

LeoJulieta
LeoJulieta

Posted on

Why Developers Are Migrating to Chinese AI Chips in 2024

#aihardware #chinachips #gpualternatives #english

Why Developers Are Switching to Chinese AI Chips – A Hands‑On Guide

Introduction

The global AI‑hardware market is shifting fast. With NVIDIA GPUs becoming scarce and pricey, developers are turning to home‑grown Chinese accelerators—Huawei Ascend, Alibaba Xuantie, and Cambricon MLU—to keep projects moving. In the past month, Google Trends searches for “Chinese AI chips,” “alternatives to NVIDIA,” and “migrate CUDA” have spiked 250 % worldwide, signaling a real‑world demand for practical migration advice.

This article shows you, step‑by‑step, how to evaluate performance, install the necessary SDKs, and port a PyTorch model from CUDA to a Chinese AI chip—all while staying aware of supply‑chain and regulatory considerations.

Quick‑Start Comparison (A100 vs. Chinese Accelerators)

Accelerator FP16 Peak HBM Bandwidth MLPerf v1.1 (ResNet‑50) Approx. Price*
NVIDIA A100 312 TFLOPs 1.6 TB/s 1.0× (baseline) $10 k–$15 k
Huawei Ascend 910 210 TFLOPs 1.2 TB/s 1.8× $6.8 k (domestic)
Cambricon MLU370 128 TFLOPs 800 GB/s 0.9× $5.5 k
Alibaba Xuantie 910B (cloud) 96 TFLOPs 640 GB/s 0.7× $0.12 / hour

*Prices are indicative of the Chinese domestic market or cloud‑only offerings; they do not include taxes or import duties.

1. Set Up the Development Environment

1.1 Install the Vendor SDK

Vendor SDK Linux distro Install command
Huawei CANN 8.0 Ubuntu 20.04 wget https://repo.huawei.com/cann/cann_8.0.0_linux-aarch64.run && sudo bash cann_8.0.0_linux-aarch64.run
Cambricon MLU‑SDK 2.1 CentOS 7 yum install -y mlu-sdk-2.1
Alibaba PAI‑Engine 3.2 (cloud) N/A (Docker) docker pull registry.cn-hangzhou.aliyuncs.com/pai/pai-engine:3.2

Tip: Add the SDK’s bin directory to $PATH and source its environment script (e.g., source /usr/local/Ascend/cann-8.0.0/set_env.sh).

1.2 Verify Device Detection 

# Huawei Ascend
npu-smi info

# Cambricon MLU
mlu-smi

# Alibaba (inside Docker)
npu-smi info   # works the same because PAI emulates NPU APIs
Enter fullscreen mode Exit fullscreen mode

If the command returns a device list with temperature, memory usage, and driver version, you’re ready to run workloads.

2. Port a Simple PyTorch Model

Below is a minimal example that moves a pretrained ResNet‑50 from CUDA to a Huawei Ascend NPU. The same pattern works for Cambricon (replace torch_npu with torch_mlu) and for Alibaba (use the torch_npu package shipped with PAI‑Engine).

import torch
import torchvision.models as models

# 1️⃣ Detect the accelerator
if torch.npu.is_available():
    device = torch.device("npu")
    print("Running on Huawei Ascend NPU")
elif torch.mlu.is_available():
    device = torch.device("mlu")
    print("Running on Cambricon MLU")
else:
    device = torch.device("cpu")
    print("Falling back to CPU")

# 2️⃣ Load model and move it
model = models.resnet50(pretrained=True).to(device)
model.eval()

# 3️⃣ Dummy input (batch=1, 3×224×224)
x = torch.randn(1, 3, 224, 224).to(device)

# 4️⃣ Inference
with torch.no_grad():
    out = model(x)

print("Output shape:", out.shape)
Enter fullscreen mode Exit fullscreen mode

What changed?

  • Only the import of torch.npu / torch.mlu and the device assignment differ from a pure‑CUDA script.
  • No code rewrite is required for the forward pass.

3. Dealing with Custom CUDA Kernels

If your project contains hand‑written CUDA kernels, you’ll need to translate them to the vendor’s low‑level API.

Vendor Translation tool Example command
Huawei ATC (Ascend Tensor Compiler) atc --framework=5 --model=model.onnx --output=model --soc_version=Ascend910
Cambricon MLUOP (MLU Operator) mluop_compiler -i custom_kernel.cu -o custom_kernel.so
Alibaba XDL (Xuantie Deep Learning) xdlc --input custom_kernel.cu --output custom_kernel.so

The workflow is usually:

  1. Export the CUDA kernel as an ONNX custom operator (or keep the .cu source).
  2. Run the vendor’s compiler to produce a shared library (.so).
  3. Load the library from Python with ctypes or the vendor‑provided Python wrapper.

4. Benchmarking Your Port

A quick sanity check can be performed with the torchbench script that ships with each SDK.

# Huawei Ascend
torchbench --model=resnet50 --batch-size=32 --device=npu

# Cambricon MLU
torchbench --model=resnet50 --batch-size=32 --device=mlu
Enter fullscreen mode Exit fullscreen mode

Typical results on an Ascend 910:

Batch Size Throughput (images/s) Latency (ms)
1 1,820 0.55
32 55,300 0.58

Compare these numbers with the same run on an A100 (torchbench --device=cuda). The Ascend often wins on latency because its CANN runtime fuses more operators.

5. Cost & Supply‑Chain Considerations

Factor NVIDIA (A100) Huawei Ascend 910 Alibaba Xuantie 910B (cloud)
Unit price $10k–$15k (spot) $6.8k (domestic) $0.12 / hour
Lead time 8–12 weeks (global shortage) 2–4 weeks (China) Instant (container start)
Regulatory risk Subject to U.S. Entity List No U.S. export controls Cloud service – compliant with Chinese data‑locality rules
Support NVIDIA DevZone, large community Huawei CANN Forum, 24 h Chinese support Alibaba Cloud technical support (English/Chinese)

If you’re building a startup that needs to scale quickly, the cloud‑only Xuantie service may be the most pragmatic entry point. For on‑premise training labs, the Ascend 910 offers the best price‑to‑performance ratio today.

6. Legal & Export‑Control Checklist

  1. Verify model licensing – Some large‑language models (e.g., GPT‑4) are under U.S. export controls; you cannot legally run them on hardware sold to Chinese entities.
  2. Check third‑party libraries – Libraries like cuDNN are NVIDIA‑specific; replace them with the vendor‑provided equivalents (CANN‑DNN, MLU‑DNN).
  3. Document the supply chain – Keep records of where each hardware component was sourced; auditors may request proof that no U.S.‑origin restricted parts are present.

7. TL;DR – Actionable Steps

  1. Install the appropriate SDK (CANN, MLU‑SDK, or PAI‑Engine).
  2. Run npu‑smi / mlu‑smi to confirm device visibility.
  3. Switch your PyTorch device string (torch.device("npu") or "mlu").
  4. Re‑compile any custom CUDA kernels with ATC / MLUOP / XDL.
  5. Benchmark with torchbench to validate performance gains.
  6. Review export‑control restrictions before deploying production models.

References

Herramienta mencionada: Railway

Top comments (0)