This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass.
What I Built
“Touch grass” usually means stepping away from the screen. My interpretation is slightly more hardware focused:
Go touch hardware.
I’m building a pixel-art animation experience for an MLH activity at the Grace Hopper Conference that encourages attendees to move beyond software-only experimentation and try something physical, visual, and creative.
Instead of simply generating an image for someone to consume, the experience helps them create a small pixel-art animation that can become part of a hands-on hardware interaction. It is meant to make hardware feel approachable for someone who might be new to hardware, or who have previously only worked entirely in code, cloud tools, or browser tabs.
If your first contribution is drawing a few blades of pixelated grass, then animating it to "blow in the wind" I'm counting that as touching grass. Maybe.
The larger goal is to use art as a low stakes entry point into hardware. People do not need an electronics background to begin. They can start with an idea, turn it into a tiny animation, and then see how creative software can connect to something tangible.
Who It’s For
I’m designing this for people attending the Grace Hopper Conference, particularly:
- Software developers who have never experimented with hardware
- Students and early-career technologists who may find electronics intimidating
- Creative coders who enjoy art, animation, or playful interfaces
- Anyone who wants a small, approachable reason to try something outside their usual technical lane
The project treats creativity as an invitation. You do not have to begin by understanding every component. You can begin by making something you want to see move.
Demo
The project is currently being developed in AI Studio. You can try it yourself even if you do not have hardware using the Python troubleshooting mode! If you have an Arduino Q, you can also copy the code it generates into Arduino App Lab yourself.
I am still refining the local AI-assisted animation workflow. It's giving me some issues with running locally, and with the potential volume I'm trying not to waste a bunch of tokens/resources on a lightweight intro to the Arduino Q. This site will likely change frequently as the conference gets closer and I talk with my other coworkers attending. But this first pass to get people introduced to the Arduino Q is hopefully fun and a step towards the goals of one of the activities we will be hosting.
Here is also a short video of the butterfly preset on an Arduino Q!
Code
Repository: https://github.com/mdunn09045/arduino-q-site
The completed repository will include the animation interface, deterministic frame-generation logic, local model integration, and instructions for running the project.
How I Built It
The core idea is to divide the work between AI and ordinary application code.
A small language model should not need to generate every pixel of every animation frame. That approach would be slow, difficult to validate, and unreliable on lightweight hardware. Plus, AI is kinda bad at design. That's where the human making the first frame comes in.
Instead, the local model creates a constrained animation plan: what extra pixels should appear, how the subject should move, and how the motion should loop. Deterministic code then converts that plan into valid pixel-art frames.
The planned stack includes:
- Transformers.js for running an open model directly in the browser
- SmolLM2-135M-Instruct in ONNX format as a lightweight local model
- WebGPU acceleration when supported
- A CPU/WASM fallback for broader device compatibility
- Deterministic rendering code that turns structured plans into consistent animation frames
This hybrid design lets the model contribute interpretation and creative direction while regular code handles the precision required by a small pixel grid.
It also creates a useful constraint: the AI is a collaborator, not the entire application.
How It Gets People Into the World
The screen is the beginning of this experience, not the destination.
A participant uses the interface to design a tiny animation, but the point is to carry that creation into a shared, physical conference experience. The project gives software-focused attendees a friendly reason to approach hardware, experiment, and talk with other people.
Pixel art works especially well for this because it is immediate and forgiving. A person can create something recognizable with only a few built in LEDs and frames. There is no requirement to be an experienced artist or hardware engineer.
The hoped-for journey is:
idea → pixel art → animation → hardware → conversation
That last step matters. Conferences can easily become rows of people looking at separate screens. A physical creative artifact gives people something to gather around, ask about, and build upon together. They can also share their creation for others to see! I also built in a moderator setting just in case.
Why Does Open Innovation Matter?
Open-source AI makes the project more portable, inspectable, and adaptable.
Running the model in the browser means the experience does not need to send every creative prompt to a remote service. It can avoid API keys, reduce ongoing costs, and keep more of the interaction on the participant’s device. For a conference with a lot of people, load could potentially be a concern if the activity gets a lot of attendees at once.
It also gives me control over the complete pipeline. I can change the model, inspect its output, constrain the format, and design graceful fallbacks. The project is not permanently tied to one hosted provider or one proprietary response format.
That freedom is especially valuable for conference demos, workshops, and educational settings. A project should not stop being useful because a credit balance runs out or a network connection becomes unreliable.
Most importantly, an open stack makes the experiment easier for someone else to study and remix. An attendee should be able to leave thinking, “I could build my own version of that” and have access to the pieces needed to try.
Challenges and Lessons
The largest technical challenge is balancing useful AI assistance with the limits of a lightweight browser model.
A small model cannot reliably emit large, perfect grids of pixels across many animation frames. The better architecture is to ask it for a compact, structured creative plan and let deterministic code perform the exact rendering.
That process has reinforced a useful lesson: adding AI does not mean asking a model to do everything. The best result often comes from giving it one focused role inside a well-designed system.
What’s Next
Before the conference, I plan to:
- Fine tune the presets
- Test the share function more thoroughly
- Add starter ideas for attendees who do not know what to draw(not just presets)
- Test performance on different browsers and devices
- Capture a complete demo
- Let coworkers try it and include their feedback
To be perfectly honest, we might pivot the activity entirely. I still think what I learned creating this is great, even if it ends up not being used in the end.
And yes, I will likely make one of the presets or suggestions be grass. I have a challenge theme to honor.
My Agent Session
I used an AI-assisted development session to explore the architecture for adding lightweight, open-source AI to the pixel-animation workflow.
Session: Codex helped me figure out what model to use on the actual site. The app itself was coded using AI Studio.
The key architectural decision was to let the model generate a constrained animation plan while deterministic code produces the actual frames.
Prize Categories
I'm doing this for fun, not for prizes :) Please do not consider me for any.
Top comments (0)