Where Does This Belong? An AI Workbench Reset Assistant for a Coworker

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

Figure: The demo interface before analysis. Left: the frame used as the reference layout. Right: the frame used as the current layout.

What I Built

I built Shared Workbench Reset Assistant for one coworker who has trouble remembering where objects originally belonged on our shared workbench.

The idea is simple:

1. Keep a reference photo of the intended layout,
2. Compare it with a photo taken after use, and
3. Prepare a short checklist for putting things back.

The reference photo provides a visual memory of the workspace.

The prototype uses an open-weight vision-language model to draft observations and a cleanup checklist. Both remain editable, and the user must review them against the photos before exporting a result.

Demo

The three demo screenshots show

1. the photo inputs,
2. reviewed observations, and
3. final reviewed checklist.

For a reproducible demonstration, I used public Tabletop Tidying Up (TTU) dataset images.

The sample shown below contains a red cup, a beverage can, and a remote control. I used the first frame as the reference layout and the last frame as the current layout. These roles were assigned for this prototype; they are not verified official tidy/untidy labels.

The workflow is:

1. Analyze the two photos independently. (Screenshot 1)
2. Correct and confirm the model's observations. (Screenshot 2)
3. Generate a checklist from those reviewed observations. (Screenshot 3)
4. Edit and confirm the final checklist, then download the result as JSON. (Screenshot 3)

Figure 1: Left: the frame used as the reference layout. Right: the frame used as the current layout.

Figure 2: Observations corrected with coding-assistant help and checked against the photos.

Figure 3: Final checklist corrected with coding-assistant help and confirmed against the reference photo.

Run the Local Demo

Follow the project README to prepare the Python environment, download the Q8_0 model and vision projector, and build llama-server.

Open two WSL terminals. Replace /path/to/hacktoberfest-2026 with the path to your local repository. The commands below use the virtual environment path specified in the README.

In terminal 1, start the model server:

cd /path/to/hacktoberfest-2026/dev-challenge/weekend-challenge
bash start_model_server.sh

After it reports that it is listening on port 8080, start the app in terminal 2:

source ~/.venvs/hacktoberfest-2026/bin/activate
cd /path/to/hacktoberfest-2026/dev-challenge/weekend-challenge
python -m streamlit run app.py \
  --server.address 127.0.0.1 \
  --server.port 8501 \
  --browser.gatherUsageStats false

Replace /path/to/hacktoberfest-2026 with the path to your local repository.

Open http://localhost:8501 in your Windows browser and select the sample. Review and correct both observations and the checklist against the photos before exporting. This address works on the machine running the app; it is not a public demo link. No hosted inference API key is needed.

How I Built It

Code

Project repository · Weekend project and setup

The main files are app.py for the Streamlit interface, workbench.py for local inference requests and image preparation, and start_model_server.sh for starting llama.cpp. Model weights are downloaded separately and excluded from Git.

Model and Runtime

• Model: SmolVLM2 2.2B Instruct
• Quantization: Q8_0 for both the model and vision projector
• Inference server: Local llama.cpp server
• Interface: Streamlit
• Environment: WSL Ubuntu 24.04
• Hardware: Intel Core Ultra X7 358H laptop with 32 GB of RAM
• Execution: CPU only; no GPU or model fine-tuning

Why Does Open Innovation Matter?

Open weights and open-source inference tools let me run the model on my own laptop without sending photos to a hosted model API. They also let me compare models, inspect failures, and adapt the workflow without relying on a hosted model provider.

What’s Next?

This project is the starting point for my month-long exploration of VLA and Physical AI. It uses a VLM to draft text observations and instructions; it does not execute a VLA policy or control a robot.

AI Assistance Disclosure

I used Codex to help implement the app, debug experiments, revise the demo observations and cleanup instructions, and draft this post. I checked the final observations and checklist against the photos. The original local SmolVLM2 outputs are preserved separately from the Codex-assisted revisions.

Experiment results and the reviewed JSON export are stored locally and excluded from Git. The screenshots show the workflow after human review and correction. The prototype has not yet been tested by my coworker and needs broader evaluation.

Data Attribution

The sample images are third-party demo data from TTU, created by Hogun Kee, Wooseok Oh, and Songhwai Oh (2024). See the project README for the source, preserved upstream license notice, citation, and licensing caveat.

References

• Hacktoberfest Weekend Challenge: Build for a Friend — challenge prompt and submission instructions.
• Tabletop Tidying Up (TTU) Dataset — source of the tabletop demo images; see the project README for attribution and licensing details.
• SmolVLM2 2.2B Instruct GGUF — model and vision projector used for local Q8 inference.
• llama.cpp — local multimodal inference runtime and server.
• Streamlit — framework used for the browser interface.
• DevRelay — tool used to update the DEV draft and publish the submission; no agent-session transcript was uploaded for this project.
• Intel Physical AI Studio — context for my separate open-source contributions and longer-term Physical AI goals; not integrated into this prototype.