CheckMate: A 3D spatial twin of my best friend's hostel room so he stops leaving his charger behind

shruti rai — Sun, 04 Oct 2026 17:26:34 +0000

This is my submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

What I Built

My best friend Kanwal has a habit of leaving things behind.

Not small, insignificant things. Important things. Things he will desperately need approximately 45 minutes after leaving the hostel.

Like his charger.

Picture this: you're preparing for a college presentation. Your laptop is charged, your slides are ready, your bag is packed, and you're already running late. You board the train, settle into your seat, and then it hits you.

"Bhai, charger toh switchboard mein hi reh gaya."

And just like that, your perfectly planned trip has a problem.

The annoying part is that Kanwal doesn't forget because he doesn't know what to pack. He forgets because packing is a physical task, while most reminders live on a screen.

A checklist can tell you to pack your charger. It cannot, by itself, connect that reminder to the charger sitting beside your desk.

I could have made him another checklist app. There are already plenty of those.

Instead, I wanted to try something different.

CheckMate is a trip-aware packing assistant built around a 3D representation of my best friend's actual hostel room. It combines an interactive spatial interface, tracked belongings, AI-assisted packing recommendations, memory of recurring mistakes, and a Hinglish voice coach that reminds him to check before leaving.

Basically, I built software to stop one very specific person from making the same very specific mistake.

And yes, the charger has a starring role.

Demo

🌐 Live app: https://checkmate-ebvi.onrender.com/

💻 GitHub repository: https://github.com/shrutirai29/hacktober

The deployed project brings together the room interface and packing-assistant experience. AI and voice capabilities depend on the relevant backend services and deployment configuration.

Why a 3D Room Instead of Another Checklist?

Here's the idea that got me interested in this project.

Kanwal's room already contains most of the information he needs to pack. His laptop is on the desk, his backpack is beside the chair, and his charger has a favourite spot near the wall socket.

The problem is that his brain doesn't always connect the packing decision to the physical object.

A conventional checklist represents belongings as text:

Laptop
Charger
Water bottle
ID card

CheckMate represents the room as a space that you can explore and interact with.

That changes the experience from simply asking "What do I need?" to asking "What do I still need to collect from my room?"

That's the distinction I wanted to explore.

The 3D Room: Getting the Details Right

The main interface is a custom 3D room built using Three.js.

I modelled the scene around the reference photograph of Kanwal's hostel room instead of starting with a generic bedroom template.

The scene includes a birchwood study desk, black desk legs, a purple swivel chair, a bed with a lavender duvet, a floating bookshelf, books, a panda figurine, trailing pothos vines, postcards, a backpack, and the wall switchboard.

The switchboard might be a strangely specific detail to care about in a 3D model.

But when your entire project exists because someone keeps forgetting a charger, it suddenly becomes important.

I worked on the furniture geometry, materials, lighting, object placement, and camera positioning to make the room feel familiar. I also had to pay attention to the little things that make 3D scenes look wrong: objects clipping into furniture, awkward proportions, and items appearing disconnected from their surroundings.

There's also a 📸 Photo Angle preset that moves the camera to a predefined view designed to match the reference photograph.

It makes the scene easier to recognise and gives the user a familiar perspective of the room.

The goal isn't to make a random 3D bedroom look pretty. It's to make a space feel recognisable enough that the user can relate the interface to their real surroundings.

Clicking on Objects Is Only the Beginning

CheckMate includes interactive belongings such as the MacBook, 65W charger, Hydro Flask, and backpack.

These objects can be selected in the scene, with visual indicators and status badges helping distinguish tracked items. Pulsing radar-style rings draw attention to the objects that matter.

But the more interesting problem is the state of each object.

There are three different questions an assistant needs to distinguish:

Does the item exist?
Where is the item?
Has the item actually been packed?

Knowing that Kanwal owns a charger doesn't mean it's in his bag. Knowing that it usually sits beside his desk doesn't prove that it's still there today.

CheckMate's interactive tracking interface is a starting point for connecting these states. Reliable automatic detection of real-world objects would require additional image-understanding capabilities, sensor input, or user confirmation.

I don't want a packing assistant that confidently tells someone their charger is on the desk when it has actually disappeared into a backpack.

A convincing 3D model is nice. Correct information about the physical world is what would make it genuinely useful.

Giving Packing Some Actual Context with Gemma

Not every trip needs the same things.

A two-day college presentation, a weekend hackathon, and a trip home all have different packing requirements.

That's why I wanted CheckMate's recommendations to depend on more than a fixed list.

The AI reasoning workflow uses Google Gemma to help generate packing suggestions based on the available trip context, tracked belongings, and relevant history.

The inputs can include:

Trip type and duration
Relevant weather information, when available
Items represented in the room
Current tracked packing states
Previous records of forgotten essentials

The idea is to generate a packing manifest that considers what to take, what to leave behind, and what deserves extra attention.

For example, a presentation trip might require the laptop, charger, ID card, and presentation materials, but not a pile of unrelated textbooks.

And if the charger has repeatedly been forgotten before presentations, it should receive more attention than an item Kanwal never forgets.

This is where contextual reasoning becomes useful. A model can help turn the available information into relevant recommendations rather than returning the same generic checklist for every trip.

The important limitation is that AI recommendations are only as reliable as their inputs. The model can reason about reported item states and past events; it cannot independently verify a physical object merely because the object appears in a 3D scene.

Memory: Because Apparently We Need to Learn This Lesson More Than Once

One of the ideas I wanted to incorporate was episodic memory.

People have patterns. Some of us repeatedly forget our ID cards. Some forget umbrellas whenever it rains. And Kanwal seems to have a particularly complicated relationship with his charger.

A useful packing assistant should be able to use relevant history instead of treating every trip as the first one.

For example, a memory record might indicate that the charger was forgotten on previous presentation trips. That information can then be used as a priority signal when generating future recommendations.

Instead of treating all items equally, CheckMate can give recurring problems more attention.

This is an important design goal: the assistant should become more useful by remembering relevant mistakes, not by making the user repeat the same instructions every time.

The memory feature is intended to support this workflow; its usefulness ultimately depends on the records available and how consistently item states and past events are captured.

The Voice Coach: Your Friend, But With Less Patience

I didn't want CheckMate to stop at a list of missing items.

When you're late for a train, you're probably not going to stand in your room and carefully read a dashboard. You need a quick reminder that gets straight to the point.

So I added a voice-coaching workflow using ElevenLabs.

The voice briefing turns the relevant packing information into a short, conversational message. The tone is deliberately Hinglish, because a reminder from a hostel friend shouldn't sound like a corporate notification.

Imagine this:

"Kanwal bhai, sun! Laptop toh bag mein daal liya, par charger check kiya? Nikalne se pehle switchboard dekh le. Delhi jaake phir battery ke bharose presentation mat chalana!"

A little rude? Maybe.

More memorable than "You have three tasks remaining"? I'd like to think so.

The application also includes a browser-based Web Speech API fallback for speech playback when the generated voice service isn't available.

The briefing is designed to use the trip details and critical items that need attention. A future version could make this even better by generating reminders directly from a verified room scan rather than relying on tracked states alone.

How the Pieces Fit Together

Here's the high-level workflow behind CheckMate:

       Room Reference / Spatial Scene
                    |
                    v
          Three.js Room Interface
                    |
                    v
          Interactive Item Tracking
                    |
                    v
       Trip Details + Item States
                    |
                    v
          Gemma Reasoning Workflow
                    |
                    v
       Contextual Packing Manifest
                    |
                    v
          ElevenLabs Voice Coach
                    |
                    v
          Final Exit Check

The 3D interface provides spatial context. The tracking layer represents the state of belongings. The reasoning workflow generates recommendations from the available context, and the voice layer turns important reminders into a spoken briefing.

The final exit check is the natural next step: before leaving, the user should be able to review the critical items and confirm that everything is actually packed.

The Tech Stack

Technology	What it does
React + Vite	Frontend application
Three.js	Interactive 3D room and spatial scene
Google Gemma	AI-assisted packing reasoning
Python + FastAPI	Backend for AI-related processing
ElevenLabs	Generated voice briefings
Web Speech API	Browser speech fallback
Render	Deployment and hosting

I wanted the experience to feel like one application rather than a collection of unrelated demos. The challenge was connecting a visual interface, an AI reasoning workflow, and voice interaction around one practical task.

The Privacy Question

A room photograph is personal.

It can reveal documents, photographs, medication, and details of someone's everyday life that they never intended to share.

That's why privacy matters for an application built around personal spaces.

Rendering a 3D scene in the browser doesn't automatically mean that the original photograph never leaves the device. Image uploads, backend processing, and external AI or voice services all need to be considered separately.

For a more mature version of CheckMate, I'd prioritise local image processing where practical, transparent disclosures about external service calls, and clear controls over what data is retained.

A personal assistant should be useful without making users uncomfortable about the information they're sharing.

What I'd Build Next

CheckMate is a starting point, and there are a few things I'd love to improve.

Automatic room understanding. I'd like users to upload a room photograph and have the application identify furniture and belongings, reducing the amount of manual scene setup.

More reliable spatial tracking. The system should distinguish an item's usual location from its current, verified location and its packing status.

Better long-term memory. Repeatedly forgotten essentials should influence future packing recommendations, while allowing users to correct or remove inaccurate records.

A smarter final scan. Before leaving, the user could review potential omissions identified by image recognition and confirm uncertain items.

More personalised briefings. The voice coach could adapt to trip urgency, preferred language, and the user's most important reminders.

These improvements would move CheckMate closer to the experience I originally imagined: an assistant that connects the room around you with the task you're trying to complete.

Why I Built It

I like the idea behind this challenge because it gives you a reason to build something for a specific person instead of inventing a problem for a portfolio.

CheckMate started with one annoying habit and a fairly simple question: why do our productivity tools keep asking us to remember everything when so much of what we need to do is connected to the physical world?

That question led me to experiment with 3D interaction, contextual AI reasoning, memory, and voice.

And honestly, I enjoyed building something where the technical decisions were connected to a problem I could actually observe.

Will a 3D room magically prevent someone from forgetting their belongings? Of course not. The tracking needs to be accurate, and the reminders need to be based on reliable information.

But I think there's something worth exploring in making digital assistants aware of more than just text on a screen.

For now, I'm hoping CheckMate makes Kanwal pause for five seconds, look at the switchboard, and save himself another charger-related crisis.

Because the real success metric isn't how many technologies I managed to put into one project.

It's whether my friend reaches his destination with his charger.

And whether the presentation survives past slide four.

Prize Categories

These are the categories relevant to the technologies used in CheckMate.

Featured categories — $200 per winner

Render — Best Use of Render: The application is deployed on Render.
Gemma — Best Use of Gemma: Google Gemma is used in the packing-reasoning workflow.

Partner categories — $100 per winner

ElevenLabs — Best Use of ElevenLabs: Voice generation powers the conversational packing-briefing experience.

These are the categories I'm targeting based on the project's integrations. Prize eligibility depends on the challenge's rules and the implementation demonstrated in the submission.

Try CheckMate