DEV Community: JohnX4321

MLH Hack At Home 2020: Hearth

JohnX4321 — Tue, 29 Sep 2026 00:09:41 +0000

This is a submission for the MLH x DEV Writing Challenge

What I Built

Hearth is a voice-controlled smart home agent built for the moment we were all in: stuck at home, working from the same room we slept in, and slowly losing track of what day it was.

The theme for Hack at Home was "help folks stay happy at home." In May 2020, the problem in my apartment wasn't a lack of smart devices. It was that home had turned into office, gym, classroom, and bedroom at once, with nothing separating them. So Hearth does three things:

Voice control that knows your devices. "Turn off the desk lamp" works because Hearth syncs the real device names from Home Assistant into its language model, instead of relying on generic "light" commands.
Modes for a life that happens in one room. "Work mode" brightens the desk light, sets the room to 71°F, and turns on an "on a call" LED outside the door. "Done for the day" shuts the work lights off, switches to warm lighting, and reminds you to step outside.
Pattern suggestions from sensor history. A small Python job looks at a week of Home Assistant state history and suggests automations you keep doing by hand, like "You turn on the kitchen light around 7:40 AM every weekday. Want me to do that for you?"

How it works

Wake word: Picovoice Porcupine runs on a Raspberry Pi 4 with a USB mic. It's fully offline, and nothing is sent to the cloud until the wake word fires.
Speech-to-text: Google Cloud Speech-to-Text (streaming) transcribes the command after the wake word.
Intent understanding: Dialogflow ES maps the transcript to intents like device.set, mode.activate, and history.ask. Device names come from a custom entity type that we sync from Home Assistant's entity registry through the Dialogflow API. When a light is renamed in HA, Hearth picks up the new name within a minute.
Fulfillment: A Python Flask webhook on Google Cloud Run turns intents into Home Assistant REST API calls.
Safety: Locks and the garage never actuate from voice alone. The webhook sends a Firebase Cloud Messaging push to the Android app, and nothing happens until someone taps Approve.
Pattern mining: A nightly job pulls history from Home Assistant's recorder, buckets it into 15-minute windows with pandas, and finds repeated manual actions. It also runs scikit-learn's Isolation Forest to flag oddities, like the garage being open at 2 AM.
Android app: Kotlin, XML layouts, ViewModel + LiveData, and coroutines for the network calls. It has three screens: devices, pending approvals, and suggested automations.
Domain: The dashboard lives at [hearth-home].tech, registered through Domain.com during the weekend.

What I learned

Generic intents break on real houses. Our first Dialogflow agent used the built-in "device" concepts and had no idea what "the Ikea lamp" was. Syncing Home Assistant's real friendly names into a custom entity type, with synonyms, was the single biggest accuracy improvement of the weekend.
Plan for your dependencies disappearing. We originally wanted Snips for fully on-device voice, until we found out Sonos had shut down the Snips console after acquiring it. Porcupine for the wake word plus cloud STT was the realistic fallback.
Safety has to live in the code. "Always confirm locks" as a design rule is only real if the webhook physically can't call the lock service without an approval token. That took 30 minutes to build and we never worried about it again.
Latency is the whole experience. Wake word → cloud STT → Dialogflow → webhook → Home Assistant came in around 2.5 seconds at first. Switching to streaming recognition and keeping the Cloud Run instance warm got it to about 1.8 seconds.

Demo

Links: I have lost access to that github account and repositories and don't know where the stuff exists. Oops. Sorry!!

This is a text walkthrough of what I demoed. The setup: a Raspberry Pi 4 running Porcupine and Home Assistant 0.109 (a mix of real smart plugs and bulbs plus HA's demo integrations), the webhook on Cloud Run, and the Android app on a Xiaomi Redmi Note 3.

Disclaimer: This part has been generated with AI assistance to make it look proper for Markdown

Architecture

 🎤 USB mic ──► Porcupine (wake word, on the Pi)
                        │
                        ▼
          Google Cloud Speech-to-Text (streaming)
                        │
                        ▼
                  Dialogflow ES ◄──── custom entity type synced
                        │             from Home Assistant registry
                        ▼
         Flask webhook on Cloud Run ──────► Firebase Cloud Messaging
                        │                          │
                        ▼                          ▼
           Home Assistant REST API        📱 Android app (approvals,
           (lights, plugs, thermostat,       suggestions, devices)
            lock, garage)
                        │
                        ▼
           Nightly pattern job (pandas + scikit-learn)

Step 1: Work mode

🗣️ "Hey Hearth, start work mode."

[dialogflow] intent=mode.activate  mode=work  confidence=0.94
[webhook]    → light.turn_on  light.desk_lamp  brightness_pct=100
             → climate.set_temperature  climate.bedroom  71
             → switch.turn_on  switch.door_on_call_led
[webhook]    3/3 succeeded
[response]   "Work mode is on. Good luck out there."
[latency]    1.8s wake-to-response

The LED outside the door turns red. That's for the roommate who kept walking in during Zoom lectures.

Step 2: Device names that actually work

🗣️ "Turn off the Ikea lamp."

[dialogflow] intent=device.set  device=@ha_device:"Ikea lamp" → light.living_lamp_2
[webhook]    → light.turn_off  light.living_lamp_2
[response]   "Done."

"Ikea lamp" is a synonym we added in Home Assistant. It showed up in Dialogflow on the next sync, and no retraining was needed.

Step 3: The safety gate

🗣️ "Unlock the front door."

[dialogflow] intent=device.set  device=lock.front_door  action=unlock
[webhook]    lock.* requires approval → FCM push sent, request held
[response]   "I've sent a confirmation to your phone."

The phone shows: "Unlock Front Door? Voice request, 8:42 PM" with Approve / Deny. The lock only opens after Approve.

Step 4: Asking what happened

🗣️ "When was the garage last opened?"

[dialogflow] intent=history.ask  device=cover.garage_door
[webhook]    → GET /api/history/period  cover.garage_door  (last 24h)
[response]   "The garage opened at 6:12 PM and closed at 6:15 PM."

Step 5: Suggested automations

The Suggestions screen in the Android app shows what the nightly job found in a week of history:

Suggestion	Pattern found	Confidence
Kitchen light on at 7:40 AM on weekdays	Manually turned on 7:30–7:50, 5 of 5 weekdays	0.88
Warm lighting at 6:00 PM ("done for the day")	Desk lamp off + living lamp on within 15 min, 4 of 5 days	0.74
Alert if the garage is open after 11 PM	Isolation Forest flagged 2 late-night opens	0.69

Tapping a suggestion shows the Home Assistant automation it would create:

alias: "Hearth: Weekday kitchen light"
trigger:
  - platform: time
    at: "07:40:00"
condition:
  - condition: time
    weekday: [mon, tue, wed, thu, fri]
action:
  - service: light.turn_on
    entity_id: light.kitchen

Accept sends it to Home Assistant, and Reject teaches the job to stop suggesting that pattern.

What didn't make it

Voice recognition struggled with the fan running in the room, so some of our testing used typed input in the Dialogflow console. The pattern miner also needs a real week of history. For the demo, we replayed a week of logged data from my apartment.

Partner Technologies

Google Cloud

Most of Hearth runs on Google Cloud:

Speech-to-Text: We used streaming recognition with phrase_hints populated from our device names, which noticeably helped with odd names like "the Ikea lamp." The biggest gotcha was a silent stream timing out and ending recognition. The fix was to only open a stream after Porcupine fires.
Dialogflow ES: We built intents for devices, modes, and history questions. Custom entity types synced from Home Assistant through the Dialogflow EntityTypes API were the key trick. The console's training-phrase interface made it easy to iterate on phrasing at 3 AM.
Cloud Run: Hosts the Flask fulfillment webhook. Deploying a container in a couple of minutes suited a hackathon well. Cold starts added about a second at first, so we set up a cheap scheduled ping to keep it warm during the demo.
Firebase Cloud Messaging: Carries the approval pushes to the Android app. We used data messages rather than notification messages, so the app controls exactly how the approval card appears.

Domain.com

We used default domains for the project dashboard provided by the Cloud Run service.

Hackathon Experience

I attended Hack at Home, the first MLH Summer League hackathon, held online the weekend of May 2, 2020. Like everyone else that spring, I hacked from home.

I was following up the events page and checking the updates constantly while making sure project meets the guidelines.

I teamed up with my College Friends to build this project, who had experience in API architecture and any Synthesis, Data tasks and Hardware, while I took care of System Design and Android side.

We enjoyed building the project and learnt a lot going through the documentations and performance while doing the same. This provided a way to experience the moment during the pandemic outside and work on upskilling our skills, during the time and be distracted for a while.

What I'll remember most: It was my first MLH hackathon experience and although virtual, I had a wonderful experience and although my memory may have faded on the specifics, I still remember it as a valuable experience.

Thanks to MLH and everyone was part of organizing it.

GameJam Cipher

JohnX4321 — Fri, 19 Jun 2026 00:30:05 +0000

This is a submission for the June Solstice Game Jam

What I Built

I built a simple cipher game with 6 levels to decrypt the cipher text based on frequency of words and hints to obtain the plaintext version.

Video Demo

Code

samarthsubramanya / JuneSolstice_GameJam_2026

☀ Solstice Cipher

A code-breaking puzzle game built for the June Solstice Game Jam — and a small ode to Alan Turing.

Six encrypted messages stand between you and the longest day of the year. Each one is a substitution cipher hiding a message about a June celebration — International Sushi Day, the World Cup, Pride, Juneteenth, Turing himself, and the solstice. Crack each cipher using frequency analysis (the same statistical reasoning Turing's team used at Bletchley Park) and the sun climbs a little higher with every solve. Break all six and you bring the light all the way back.

Theme

The June solstice is a global moment of transition — the longest day in the north, the shortest in the south.

The game makes that literal: decoding is the act of bringing back the light. A night sky brightens into dawn as your accuracy rises, a Pride rainbow…

View on GitHub

june-solstice-game-jam-2026.vercel.app

How I Built It

I built it using ReactJS and hosted on vercel. I implemented a simple frontend with no backend dependency session-specific experience for simplistic approach Using hardcoded values and configuration.

Prize Category

Generated with Assistance from Google AI through Antigravity.

Google I/O 2026 - Android XR and AI-Studio Android Development

JohnX4321 — Sat, 23 May 2026 20:17:54 +0000

This is a submission for the Google I/O Writing Challenge

There were two Google I/O 2026 announcements that excited me the most. The renewed emphasis on Android XR and the ability to create Android Applications using Google AI Studio. As a Mobile Applications Developer having worked on AR,VR,Wearables and even Electric Vehicles, this stood out to me the most as they impact the interaction of development with software platforms.

Android XR and Ambient Computing

Google showcased their smart glasses and integrated XR experiences, now powered by Gemini which provides contextual assistance, navigation, messaging and visual understanding. AI is becoming an always-available layer instead of standalone features.
The interaction model has moved away from traditional keyboards, touchscreen and apps to multimodal interaction like voice, vision and spatial awareness. It feels like a natural interaction with environment.
Multiple engineering areas have been integrated into a common ecosystem including on-device AI inference, low-latency systems, wearable hardware, CV and AI agents.. The smartglasses also look professional and consumer preferred than experimental designs.
This announcement pushes the tech environment to a new computing paradigm.

Building Android Apps with Google AI Studio

The other announcement was the ability to build android apps using Google AI studio with few prompts, sync to Github and Publish to App Store from a single screen. Here's a voice recorder I tried implementing with the studio

johnx4321 / NeoAudioRecorder-AiStudio

Neomorphic Audio Recorder for Android built using Google AI Studio entirely

Run and deploy your AI Studio app

This contains everything you need to run your app locally.

View your app in AI Studio: https://ai.studio/apps/6a9dda79-ec82-4106-b8f4-353c3d3df7a2

Run Locally

Prerequisites: Android Studio

Open Android Studio
Select Open and choose the directory containing this project
Allow Android Studio to fix any incompatibilities as it imports the project.
Create a file named .env in the project directory and set GEMINI_API_KEY in that file to your Gemini API key (see .env.example for an example)
Remove this line from the app's build.gradle.kts file: signingConfig = signingConfigs.getByName("debugConfig")
Run the app on an emulator or physical device

View on GitHub

You can also preview the app and view the studio prompts from the link in README of the github repo.

This is not a simple AI autocomplete and for students, indie developers and small teams who are working on initial POCs, this significantly accelerates development speed. There are few hiccups in initial design to final implementation where things may not turn out visually as expected, but that's where the experienced developers come into picture. Developers can also learn iteratively, without memorizing all the API's.

Overall Takeaways

AI is integrated more by Ecosystem developers into their tools. How users interact and how developers build is also changing. The barrier to entry for development is also being lowered with such integration of AI.

The future provides endless opportunities for people who use it appropriately.

FieldFix - AI Diagnosis

JohnX4321 — Sat, 23 May 2026 19:28:47 +0000

This is a submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

This system allows Field Technicians to enter relevant problem statements to the system which the AI System performs diagnosis on using Gemma4 to output resolution plan and likely cause.

Demo

Code

johnx4321 / gemma4-fieldfix

FieldFix Gemma

FieldFix Gemma is an offline-first repair copilot for field technicians. It combines local retrieval over equipment manuals with Gemma 4 reasoning to turn rough field notes, readings, and optional photos into a safe diagnostic checklist and dispatch-ready work order.

Why Gemma 4 4B

This project intentionally targets the small Gemma 4 model class, especially the 4B model.

Field technicians often work with weak connectivity.
Repair notes, measurements, and manuals are private operational data.
The task is narrow enough for a small local model when paired with retrieval.
Low latency matters more than open-ended general intelligence at the point of repair.

The app keeps retrieval cheap and local, then asks Gemma 4 to do the real judgment: rank causes, apply safety constraints, decide the first checks, and produce structured work-order JSON.

Architecture

server.js: static app server and /api/diagnose endpoint.
src/diagnosisEngine.js: local manual retrieval, prompt construction, validation, and…

View on GitHub

How I Used Gemma 4

I used Gemma4 4b latest model. The Frontend prompts the model with relevant data which ingests the data and the symptoms and performs relevant analysis and throws back to the UI to indicate the resolution and fix. This can speed up resolution of complex scenarios and assist technicians to cover more repairs per day or be determinate in their solution.

AI Blog Writer

JohnX4321 — Fri, 15 May 2026 01:03:59 +0000

This post is my submission for DEV Education Track: Build Multi-Agent Systems with ADK.

What I Built

I built an AI Blog Writer which accepts a title and style/prose of writing and generates textual content in that style for the blog.

Cloud Run Embed

Your Agents

There are 4 Agents + 1 Orchestrator to coordinate between the agents
i) Researcher
This receives the user input topic/title. It produces a structured brief covering key concepts, trends. It doesn't write in prose and just gathers foundation for other agents' working.

ii) Outliner
This agent receives the topic and research brief from previous agent. It is a content strategist which determines the structure : titles, sections, conclusion. It writes only the skeleton
iii) Writer
This agent receives the data from previous agent and produces long form prose sentences and writes the complete draft.

iv) Editor
This agent receives the draft and validates it. Hook Strength, Clarity, Flow, Grammar, Tonal Consistency. It outputs final polished blog post.

Key Learnings

The capability of the agents to stick to their specific role was insightful. The technical depth of the blog and transformation of the prose was also a reflection of the capability.
One of the challenging part was to get all of it together and deploy due to IAM issues on Cloud. But was able to resolve all, and successfully deploy.

Teapot - Brew Tea from AI

JohnX4321 — Sat, 11 Apr 2026 22:29:34 +0000

This is a submission for the DEV April Fools Challenge

What I Built

TeaPot.cloud - An enterprise-grade, AI-powered, cloud-native tea brewing infrastructure platform that is constitutionally, fundamentally incapable of brewing drink. You can go in and experience how making tea with AI looks like but you won't be able to drink one though :'). Equally Expensive as well.

Demo

devchallenge-aprilfools-2026-t9w8.vercel.app

Code

samarthsubramanya / devchallenge-aprilfools-2026

How I Built It

Built with NextJS and Lucide-React components majorly

Prize Category

Honestly, I would like to be considered for Google AI Usage and Community Favorite if possible.

Supporting Teammate: Claude

Logo Generation from prompts

JohnX4321 — Sun, 15 Mar 2026 02:42:51 +0000

This post is my submission for DEV Education Track: Build Apps with Google AI Studio.

What I Built

I wanted to build a Business Logo Generator based on prompt about the business and it's description.
The prompt I used was:

You are a professional software engineer. Can you create an application which takes word prompts about a business and generate an icon for the same ?

The rest of the prompts was to fix certain UI issues which I observed.

Demo

Applet Link: https://ai.studio/apps/70865850-95ce-44e3-ad5d-08955c29e137
Screenshots:

My Experience

AI studio allows creation of Technical ideas quickly and efficiently. it also allows immediate deployment through Google Cloud Run and anyone can get their ideas running quickly in a short while than developing from scratch. Human intervention or validation is also necessary because the products generated are not up to the mark always.

Notes App

JohnX4321 — Tue, 10 Feb 2026 02:01:19 +0000

This is a submission for the GitHub Copilot CLI Challenge

What I Built

I built a simple note taking application with a priority level set. The Copilot model struggled to achieve functionality with other ideas, I thought to implement. Hence, settled for this.

Demo

Github Link: https://github.com/JohnX4321/CopilotChallengeNotesApp?tab=readme-ov-file

My Experience with GitHub Copilot CLI

I used the github copilot to generate the template and test the functionality. It sped up the development time and also to debug issues.

CSS Art 2024 Submission, Earth Day Edition Frontend Challenge

JohnX4321 — Sat, 20 Apr 2024 11:52:13 +0000

This is a submission for Frontend Challenge v24.04.17, CSS Art: Earth Day.

Inspiration

The CSS art represents planet earth with its flora in the form of a tree. The Wind animation indicates the presence of Air required for both float and fauna to survive.

Demo

Journey

I have kept the content simple. Will work on enhancing my skills to create more complex representations for the next challenges.

#GithubHack23: Grocery List App using Expo

JohnX4321 — Wed, 10 May 2023 13:54:51 +0000

What I built

I have built a simple Grocery List Application which you can utilize while shopping. It has a simple and minimalistic design without extensive customization and does get the job done

Category Submission:

Phone Friendly

App Link

https://github.com/JohnX4321/GroceryList/releases/tag/1.0

Screenshots

Android & iOS

Description

A simple Application to list out items for shopping at the supermarket (not specific to groceries). You can check the item to move it to completed section and revisit at the end to cross check if it has been picked or not. In the end you can clear the items to begin the cycle freshly again. Built using Expo , React Native , Redux & React Native paper.

Link to Source Code

https://github.com/JohnX4321/GroceryList

Permissive License

MIT License

Background (What made you decide to build this particular app? What inspired you?)

I wanted to build a simplistic app which would come in handy in every day use without additional feature or complexity. There is no Online Sync , Login or integration with other services.
I find it handy to list out items needed to buy before heading out to the grocery mart. I have been inspired by simple Dark/Light theme and determination to use Redux functionality in the app.

How I built it (How did you utilize GitHub Actions or GitHub Codespaces? Did you learn something new along the way? Pick up a new skill?)

Utilized Github codespaces to develop the application along with expo tunnelling on the codespace server to debug and test the application. I learned automation and remote development skills along the way.