It is a question of seconds when it comes to kitchen safety. It can be a gas leak, working with a hot pan without gloves, or just the ability to keep a recipe healthy, having a second set of eyes to help out can be the difference. In the case of the Gemini Live Agent Challenge, I developed the Safe and Healthy Chef a raspberry pi 5-based system, with a multi-agent AI architecture, to conduct safety, ingredient, and environmental real-time monitoring.
[?] The Architecture: 7 Agents, 1 Mission.
The project is not merely a single script, it is a team of seven specialized agents going to work. This practice of divide and conquer makes sure that the mission-critical operations such as gas detection are given the priority to non-mission-critical activities such as routine environment logging.
The Real-Time Voice: Chef AI (agentlive.py).
Live Agent is the core of the system. It gives a voice interface with low-latency using gemini-2.5-flash-native-audio-latest. You will be able to say literally, Chef AI, is it safe to touch the pan? and get an instant response. It is also in charge of a running live OpenCV overlay which displays a heads-up display of all kitchen metrics.
The Vision Specialists: Vision Safety and Vision Ingredients.
Safety Agent: Checks whether hands are covered with gloves and searches cooking pans. Whenever it finds a bare hand in the vicinity of a pan, it will set off a voice alarm.
Ingredient Agent: For no oil recipes no boil recipes. It is equipped with Gemini Vision that identifies the excessive salt, chili powder or the outlawed existence of oil.
The Hardware Sentinels: Gas and Environment.
Gas Agent: A high frequency gadget of the MQ2 sensor. In the event of gas being detected, a voice alarm and a system wide warning of danger are triggered.
DHT11 Agent: Monitors the temperature and humidity in the kitchen after every 5 seconds and warns the cook when the place is too hot to be considered safe.
The Brain: The orchestrator (agentorchestrator.py)
There is too much data and a person should make a choice of what is important. The Orchestrator receives reports in all the agents, and he gathers alerts using Gemini and he prioritizes those alerts (e.g., Gas > Safety > Ingredients > Environment).
The Memory: Cloud storage (agentstorage.py).The system will automatically upload the photos of alerts and JSON logs to the Google Cloud Storage (GCS) to maintain accountability. In case of a safety violation, an image is backed up on the cloud immediately.
Deep Dive: It works, Make it Live.
The multimodal input/output on the edge was one of the greatest problems. My USB microphone and speaker streams went through PyAudio, and the high-resolution frame captures of the Vision agents were through Picamera2.
With the help of the lgpio library to achieve the bit-banging of the DHT11 protocol and observe the MQ2 digital output, it was possible to close the divide between the physical hardware and cloud-based AI.
Python
A peek into the logic of prioritization of the Orchestrator.
ORCHESTRATORPROMPT = “””
Determine the most urgent problem of all agents.
Rank: gas, safety, ingredient, environment.
Assign the chef one definite thing to do immediately.
“””
The important characteristics of the GeminiLiveAgentChallenge.
Multimodal Interaction: Makes a smooth transition between seeing (Vision), hearing (Audio) and acting (Hardware control).
Proactive Safety: Does not simply wait to be asked a question but intervenes when the chef thinks there is a danger such as a bare hand stretching out to touch a hot pan.
Cloud Integration: Integrated with the Google Cloud to the full extent of logging and analysis of long-term data.
Reflection
The construction of the Gemini Live Agent Challenge challenge got me to consider AI as not a chatbot only, but as a self-willing collaborator within a physical location. Raspberry Pi 5 offers the optimal edge-computing capabilities to bring this vision to a reality.
You can see the project on GitHub https://github.com/vijayece19/safe-healthy-chefand get in touch with me about it!
"Developed for the #GeminiLiveAgentChallenge 2026. 🚀 Check out how I turned a Raspberry Pi 5 into an intelligent kitchen sentinel using Gemini Live and a suite of hardware sensors."
GoogleGemini GoogleGemini AI RaspberryPi KitchenSafety MultiagentSystems GeminiLiveAgentChallenge.
Top comments (0)