DEV Community

Cover image for The Philosopher Plush — a DIY AI toy that runs 100% local
Manolo Salsas
Manolo Salsas

Posted on

The Philosopher Plush — a DIY AI toy that runs 100% local

#python #raspberrypi #ai #opensource

Are you the kind of person who enjoys a good philosophical conversation but would rather not have to buy a round of beers to get one?

Now you can build a plush toy to have those chats with, no socializing required.

So what the heck is this?

It's a plush toy that sees you, recognizes you (even several different people) and remembers your past conversations, reads your emotions, listens to you, moves its head, and talks back as if it were a philosopher — though you can give it any personality you like.

It can run 100% on your local network, with no Internet. The code is on GitHub (link below).

Here you can see how it sounds with the different personalities.

What do I need?

  • Plush toy — €15
  • Raspberry Pi Zero WH (headers already soldered) + SD card — €40
  • USB hub — €5
  • USB microphone — €10
  • USB speaker — €15
  • CSI camera (OV5647) — €15
  • Servo to move the head — €5
  • Dupont cables — €5
  • Any computer. 8 GB of RAM is plenty.
  • An OpenAI-compatible AI inference server (Ollama, vLLM, LM Studio, …) with whatever model you want (I'd recommend at least 8B). I used an 8B Hermes model on a local server with 16 GB of VRAM.

Total: around €110, not counting the computer + inference server.

Assembly

All the ingredients before stuffing them into the toy. You'll spot an extra servo, it was meant to move the arms, but the board couldn't power everything, so only the head servo made the cut.

It's straightforward. First, flash the OS onto the SD card — Raspberry Pi OS Lite works fine, and you can install it with Raspberry Pi Imager.

Then just follow the detailed instructions in the repo: install the software on the Raspberry Pi and on the computer.

Finally, hollow out the plush toy and put the camera, the board, and the head servo inside. If they fit, you can also tuck in the USB hub, the mic, and the speaker. In my case those stayed outside.

All that's left is to fire it up and start philosophizing.

How does it work? 

External LLM  --HTTP-->  server (laptop, the brain)  <--WebSocket-->  toy (Pi Zero, I/O)
  OpenAI-compatible       STT · vision · TTS · memory                  mic · camera · speaker · servo
Enter fullscreen mode Exit fullscreen mode

The Raspberry Pi streams audio and video frames over a WebSocket server to the computer. The computer does almost everything: speech-to-text (STT), face and emotion recognition, adds the personality prompt, sends it to the inference server (which could be the same computer if it has enough VRAM/GPU), gets the response back, converts it to audio (TTS), and sends the audio back. It also sends signals to the board to move the servo (the head). Finally, the board plays the response.

I've defined a set of base personalities that you can switch via configuration. You can also switch languages (Spanish and English for now) and enable/disable it by voice with a keyword or phrase. There are other settings I won't cover here, but you can see them in the .env file.

Adding new personalities and languages is very easy.

Code

https://github.com/msalsas/philosopher

MIT License.

This is a non-profit project, I'm not after any kind of gain, and it doesn't collect any data. You can run the whole thing locally. If you'd like to contribute, please open an issue explaining it before sending a pull request.

Top comments (0)