Feed the Gargoyle: a stone imp that asks riddles, hears the answer and pays in candy, all offline on an Arduino UNO Q

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Nobody remembers who carved the imp.

The story on our street goes like this. Long ago, a small stone-born trickster spent one Halloween too many robbing sweets off other people's doorsteps. The carver who caught it did not smash it. He bound it into a block of stone with one curse: guard a single bowl of sweets for as long as the stone stands, and never take one. Then he walked away, and nobody has seen him since.

The imp was on our doorstep before the bowl was, crouched low with one eye half open. Every Halloween since, the sweets have been safe. Not one has gone missing. Nobody has ever seen it move. Some say that on the night itself, if you stand very still, you can hear stone shifting.

It has had a very long time to get bored, and a very long time to think. This year I woke it up.

Meet the gargoyle. A small stone imp, cheeky and a little greedy. The old bargain with a guardian spirit goes like this: you want what it guards, you pay the toll. The curse says it may never hand out sweets for free, but it never said anything about handing them out for a good answer. So the imp found its loophole. Its toll is a riddle, asked in a voice that sounds like it has been waiting inside a well. Answer right and it grudgingly drops one wrapped sweet out of its mouth. Answer wrong and it cackles and lets you try again (it enjoys this far too much). When it feels theatrical, it breathes out a puff of cold mist, and its eyes go from a sleepy glow to wide awake.

The whole creature runs on one Arduino UNO Q (4 GB). No cloud, no server, no subscription. Edge Impulse models see the visitor, hear the answer and read the costume. A Gemma 3 1B model on the same board wrote the first drafts of the imp's 164 lines, and those lines are voiced ahead of time so the imp never makes anyone wait.

The body in the photo above is a cardboard prototype, not the final stone look. The section on the body below says why, and shows the 3D-printed version I am planning.

What each half of the board does in the video:

• Linux side: the camera model sees the visitor and wakes the imp. The app picks a riddle line and plays it. The keyword model hears the answer. The app decides right or wrong, picks the reply line and sends the commands over the Bridge.
• Microcontroller side: it turns the servo paddle on dispense, checks the infrared break-beam to see the sweet fall, runs the eye LEDs and the mist switch, and reports drop_confirmed back to the app. It also keeps the watchdog running, so if the heartbeat stops, the servo and mist stop.

How a visit works

Here is the whole thing from the visitor's side. Each step names the part that does the work.

1. Someone walks up. A USB camera feeds a small Edge Impulse person detector. The imp's eyes glow and it wakes.
2. It greets them and asks a riddle. A rule picks the right category of line (greeting, riddle, costume remark) and a pre-voiced line plays through the speaker. If the camera is sure about a costume or prop, it says something about that, never about a face or a body.
3. The visitor answers out loud. A USB microphone feeds an Edge Impulse keyword model that listens for a short list of answers: candle, mirror, pumpkin, scarecrow, shadow, spider.
4. Right answer: the imp pays. The Linux side sends dispense over the Bridge. The microcontroller turns a servo paddle, a sweet slides out of the mouth, and an infrared break-beam confirms it fell. LEDs flash and a mist burst plays.
5. Wrong answer: the imp laughs. A pre-voiced line says try again. Nothing is punished.
6. If anything goes wrong, it fails safe. If the sweet does not drop twice in a row, the microcontroller reports a jam. If the Linux app crashes or freezes, a watchdog on the microcontroller stops the servo and the mist.

And the same thing as a system, with the Linux side, the Bridge and the microcontroller side, plus the offline line-writing strip along the top:

Demo

Everything in the build, laid out flat: the UNO Q in the middle, a USB hub, a USB camera with infrared LEDs, a USB audio adapter and two small speakers, a micro servo for the paddle, an infrared sensor module for the break-beam, two 7-pixel WS2812 rings for the eyes, and an ultrasonic mist maker on its small driver board.

What I tested on the bench with the real parts:

Test	Result	Board and setup
Paddle drop, break-beam on D2 confirms the fall	2 of 3 dispenses confirmed. The third was an empty paddle, not a stuck servo	UNO Q 4 GB, servo on D9, LOAD 10 and DUMP 80, 500 ms move
Two missed drops in a row	The sketch sends jam to the app and goes to a safe state	same board
Mist burst limits	A burst is cut at 3000 ms. A new request inside the next 6 s is refused	same board, mist switch on D4

Still to test: the watchdog with the real servo and mist maker drawing current, and a full run with a stranger at the door.

The body: cardboard today, stone later

The plan is a 3D-printed stone gargoyle, crouched on a cracked block with its tongue out over a small cauldron. This is my own model, the one I want to print:

The model is my own design.

I do not have access to a 3D printer, and printing a body that size means borrowing one for days. So the body you see in the photos and the video is cardboard: a box from a horn speaker, cut open where the parts need to look out.

Why cardboard works for this stage:

• It costs nothing and I could start at once. The box was already here.
• It is easy to change. The camera sits on top, the LED rings look out through two holes, and the sound comes through a cut-out. When a part was in the wrong place, I moved it with a knife and tape. A wrong print costs hours of printer time.
• It tests the layout before I print. The layout is what the printed body must get right: the camera high enough to see a visitor, the eyes where a child would look, the speaker and microphone with a clear path to the visitor, and the paddle and break-beam above the chute so the sweet slides out clear.
• It proves the electronics. The camera, models, voice, LEDs, servo and mist do not care what the body is made of, so the game runs the same.

What cardboard does not do: it is not the final look, it is not weatherproof, and it is not a good home for a real porch. The printed version keeps the same layout, with the eyes and spine lights in the head and back, the camera, microphone and speaker in the block, and the paddle and break-beam behind the tongue. If you have a printer, the print is the next step. If you do not, a box gets you most of the way, as it did for me.

Code and models

Feed-the-Gargoyle on GitHub: the MCU sketch, the Python app, the line-bank scripts, and the benchmark logs behind every number below.

Each deciding model is its own public Edge Impulse project, so you can open it, retrain it or reuse the data:

• Sound
• Visitor
• Keywords
• Costume
• Props

How I Built It

Two processors, two jobs

The UNO Q has a small microcontroller next to a Linux processor, and the build leans on that split.

The MCU sketch owns everything that moves or can fail dangerously: the paddle servo, the break-beam, the mist switch, the LEDs and the watchdog. If the app stops sending its heartbeat, the imp goes still and the mist goes off.

The Linux side runs the camera, microphone, models and speaker. The two halves talk over the Bridge. The app calls dispense, set_fog, set_mode and heartbeat. The sketch calls back with drop_confirmed or jam.

I wired the power the same way I split the work. The LEDs, servo and mist maker run from their own 5 V rail with a kill switch, fuse and 1000 uF capacitor, and the board has its own USB-C PD supply. The two share one ground. The schematic is in KiCad and the files are in the repo (hardware/kicad/).

The pins are marked as calibration values in the sketch until I finish bench-checking each one. The 5 V rail has a 2.5 A fast-blow fuse, sized from a worst case of about 1.8 A (LEDs at their firmware caps, servo stalled, mist on). I still need to confirm that with a meter on a full visit.

The cascade: small models, each with one job

Instead of one big model doing everything, I chained small ones. Each answers one narrow question, so each can be tested and swapped on its own.

Question	Model	Result	Role in the build
Is someone there?	FOMO detector, trained on open images of people	F1 0.69 on the Studio validation set. Finds 75.8% of visitors and stays quiet on 100% of empty scenes	Wakes the imp. Studio estimates 22 ms per frame, 305 KB peak RAM and 81 KB flash for the int8 EON build (a Studio estimate, not a board measurement)
What did they answer?	Keyword spotting, MFCC, 6 words plus noise and unknown	80.2% on 207 test windows (Studio float32). Strongest on mirror (92%), pumpkin (91%) and candle (88%)	Picks the answer from a short list
What are they wearing?	Image classifier for costumes: pirate, princess, superhero, undead	64.4% on 59 test images (Studio float32)	Behind a confidence gate: it speaks about a costume only when it is sure
What are they holding or wearing on their head?	Image classifier for props: hat, glasses, crown, scarf, none	69.0% on 287 test images (Studio float32), up from 55.9%	Same confidence gate. Crown, glasses and hat are at 81 to 83%
Was that a loud noise?	A plain loudness gate in the app	The Edge Impulse sound classifier reached 62.5% on 333 test windows, so the simple gate does the job	Reacts to a roar or a clap

Every model went through the same loop, from open data to a file the board runs:

The costume and props .eim files load side by side on the UNO Q and each answers in about 100 ms per frame (float32 build, board idle, camera frame already in memory).

The costume matrix is drawn from the per-class test results and adds up to 62.7% (37 of 59), one image under the 64.4% Studio headline. The matrices show where each model is solid and where the next data goes: scarf and "none" in props, shadow against scarecrow in the keywords. The visitor detector never fires on an empty scene in the validation set, so the imp does not wake for nothing. It misses about one visitor in four frames (24.2% in validation), and I have not yet seen how that plays out at a real door.

Where the data came from. Costume and props used open data only: Open Images V7 boxes for props, Wikimedia Commons categories such as zombie walks and cosplay for costumes. I removed near-duplicates, capped images per photographer, and held some photographers out for testing so one person's photos cannot sit in both splits. A CLIP check then screened every crop and threw out any that looked like a child, since no child's photo goes into these sets. That check was strict: only 3 ghost and 12 witch images survived, so those two classes are not in the model.

Why the sound model became a gate. I built the sound set from open FSD50K clips and tried two feature types. A scream and a hand clap look alike to a model this small, so the imp reacts to "loud for long enough" and does not pretend to tell them apart.

The voice

A gargoyle has to sound like one and still be understood by a six-year-old. Piper (en_GB-alan-medium) renders each line one sentence at a time on the board, and the ending sets the delivery. A "!" is quick. A "?" rises and waits. A "." falls. The last sentence is slow. A numpy and scipy chain then lowers the pitch, adds a sub layer with a slow growl wobble, adds presence for consonants and applies a cave reverb. The laughs, growls and stone door are CC0 recordings from OpenGameArt, with the licence read on each page.

To keep it clear, I scored every line with Whisper base.en on my PC and counted word errors against the script. The voice I kept scores a mean of 8% and a median of 0% across all 164 lines. My first, more ancient-sounding versions were unintelligible, so the imp got a clearer voice instead. My favourite moment is the rising "stone door?" at the end of a question. A speech recogniser is not a child at a door, so a listening test with real people is still to do.

Where Gemma fits

Gemma 3 1B (Q4_0) runs on the board under llama.cpp, in the container the Arduino LLM brick ships. I measured it before deciding its job.

Speed. I wanted the imp to answer the moment a visitor speaks, with no pause while it thinks. Gemma 3 1B on the 4 GB UNO Q generates about 4.4 tokens per second at 3 threads, so a 12 token reply takes 7 to 8 s from request to end (5 runs, 40 token prompt, board idle, Ethernet), and 13 s while the camera, sound and keyword models are also running. A child at a door should not wait that long, so the replies are written ahead of time and the board spends its time on listening and reacting. From the decision to sound at the speaker takes 116 to 142 ms (5 runs per method, board idle, speaker 20 cm from the microphone, measured to the first chunk the microphone hears).

Writing. Gemma is a good brainstorming partner and a strict editor makes it a useful one. I asked it for imp lines in three prompt designs, on the board, with a 14 word limit and a cue-word check.

Prompt	Lines accepted
Whole persona as the prompt	0 of 39
Short prompt, three examples	8 of 48 (17%)
Same, banning filler words, up to 8 tries per line	22 of 86 (26%)

So the work flows like this: Gemma proposes variants offline, a validator rejects any line that mentions a face, a body, an age or a size, and I approve what goes in. The result is 164 lines in 22 categories, rendered to audio ahead of time. At the door the imp only has to pick the right line and press play, and sound starts 116 to 142 ms later (the playback measurement above; the line lookup itself I have not timed). The LLM's work arrives already baked into audio.

Next: start writing when the visitor appears. The bank is fixed, so it cannot say anything specific to the person in front of it. The next step is to start Gemma the moment the camera sees a visitor and let it prepare a reply for their costume or prop while the imp plays its greeting from the bank. The greeting covers the 7 to 8 s Gemma needs, so the visitor never waits. This is not built yet and I have not measured it on the board.

Two more things I learned along the way:

• Prompt caching in llama-server cut a repeated 168 token prompt from 21.5 s to 0.8 s (best case, identical prompt). The persona rules now sit in a fixed prefix and only the tail changes.
• I had been told 4 threads resets this board. I ran 960 s at 4 threads (150 requests, board otherwise idle) and it did not reset. It gave 5.4 tokens per second against 4.4 at 3 threads. With the camera, sound and keyword models running too, 4 threads gave no speed gain, so I kept 3 and left the cores to the other models.

Safety rules in the build

• Wrapped sweets only.
• It never comments on a face or a body, and mentions a costume only when the classifier is confident.
• Frames stay in memory. Nothing is stored or sent.
• LEDs, servo and mist maker run from their own 5 V rail, not from the board.
• The microcontroller watchdog fails safe: no heartbeat, no motion, no mist.

Why Does Open Innovation Matter?

With the router unplugged the gargoyle still works. A trick-or-treater at my door depends on no server, no monthly bill and no company keeping a model alive.

Open weights and open data gave me three specific things.

I could time it. Gemma ran on my own board, so I measured it and gave it the job it can do: writing lines ahead of time. A hosted model would have given me a bill and "it works".

I could change course. I gave the LLM an offline writing job, gave sound a loudness gate, and rebuilt training sets from open datasets and Commons categories in an afternoon with no permission requests.

Anyone can check my numbers. Every figure here comes from a script and a log in the repo, and every model is a public Edge Impulse project. If a number does not hold on your board, tell me.

The Spirit of It

I wanted a creature that guards a bowl of sweets and sends a shiver down your back without ever really scaring you.

Most Halloween props shout at you. This one asks. A voice rises out of the stone, the eyes open, the mist curls out, and then it waits, a little too patiently, for your answer. That pause is the fun part. You are no longer walking past a decoration. You are in a conversation with something that has been shut in a block of stone for a very long time, and it has opinions about your answer.

It laughs at a wrong answer and tells you to try again. It never sulks, and a child who has never met it can still win on the second go. The visitor does the work, and the stone pays in candy.

The curse gave me the rules for the build. A guardian bound to one spot for life has to be dependable. So the microcontroller watchdog stops the servo and the mist if the app goes quiet, only wrapped sweets go in, and it never says a word about a face or a body. It may remark on a pirate hat or a pair of glasses, and only when it is sure. A spooky guardian is a joy. A surprise in the wrong place is not.

It also has to work on any doorstep, with no router, no account and no company keeping a model alive. The riddles, the voice and the reactions all sit on the board, because a guardian bound to the stone should not need a server somewhere else.

And the mystery stays. Nobody knows who the carver was. Nobody knows what the imp did with its centuries. The gargoyle will only tell you it is bored, it likes riddles, and it is a little too pleased when you get one wrong.

Prize Categories

Best Use of Arduino (UNO Q). The project uses both halves of the board on purpose. The MCU owns the servo, break-beam, mist switch and a fail-safe watchdog, and reports a jam back to the app when a sweet fails to fall. The Linux side runs the camera, microphone, five Edge Impulse models and Gemma-written voice lines on the 4 GB board with no network. I tested the Bridge in both directions, and tested the servo, break-beam and mist limits on the real parts. The cascade, heartbeat and latency logs are in the repo.

Best Use of Gemma. Gemma 3 1B runs on the board and writes the imp's voice. I measured its speed (4.4 tokens per second) and its writing (up to 26% of lines accepted with the best prompt), so I know where it earns its place: proposing line variants that a validator and I filter into a voiced bank, which then plays instantly at the door.