In recent years, running Large Language Models (LLMs) like OpenAI's ChatGPT locally required either a CUDA-compatible GPU or an Apple Silicon Mac with plenty of RAM. However, we're now seeing the emergence of edge LLM modules designed for microcontrollers. In this tutorial, I'll show you how to combine M5Stack's credit card-sized computer (Cardputer) with the ModuleLLM to create a palm-sized device running a local LLM!
Check out the demo here: Twitter Video
What You'll Need
1. ModuleLLM
First, you'll need to get your hands on the ModuleLLM module. These have been selling out quickly at Switch Science (the Japanese distributor) and even on the official M5Stack Store. When I checked Aliexpress, I managed to snag the last one in stock! Production will likely resume after the Chinese New Year, with inventory possibly returning around March.
2. Cardputer
You'll also need the M5Stack Cardputer. The good news is that these seem to be back in stock recently!
Assembly
Disassembly
First, remove the ModuleLLM from its frame by unscrewing the four screws with a hex wrench.
Then disassemble the Cardputer. Be careful when removing the large battery, which is attached with double-sided tape (don't bend the battery forcefully as it could be a fire hazard!). Also remove the black component on top of the Cardputer.
Fitting the ModuleLLM
You'll need to carefully trim the case to fit the ModuleLLM inside. Cut out the back of the case with a utility knife to expose the pins (M.BUS). Make sure to insulate properly with Kapton tape or acetate tape to prevent short circuits.
Here's how it looks when assembled:
Connecting the Components
The ModuleLLM can communicate via either UART or TCP (port 10001). For this project, we'll use UART.
The wiring between the Cardputer and ModuleLLM is as follows:
| Cardputer (GROVE) | ModuleLLM (M.BUS) |
|---|---|
| G (Black) | GND |
| 5V (Red) | 5V |
| G1 (Yellow) | UART (Tx) |
| G2 (White) | UART (Rx) |
(Note: As of February 9, 2025, Tx and Rx connections have been swapped. The LLM630 Kit can now be connected directly and should work properly)
Caution: In this photo, G1 is connected to Tx and G2 to Rx.
Programming
Create the software by combining and modifying these official samples:
It's a good idea to add audio feedback for key presses.
Configure the UART TX and RX GPIO pins like this:
Serial2.begin(115200, SERIAL_8N1, 2, 1);
Complete!
Enjoy your palm-sized local LLM!
Updates
- February 3, 2025: Someone has already built their own version! Check it out here: https://posfie.com/@necobut/p/JKAorjR
- February 9, 2025: Source code is now available at https://github.com/GOROman/LLMCardputer
Top comments (0)