Paws the Panic

This is a submission for the MLH x DEV Writing Challenge

What I Built

Paws the Panic is an AI powered emergency platform that connects pet owners with communities and rescue teams to help and aid lost and injured pets.

Our inspiration for Paws the Panic actually came from one of our teammates who has lost their cat multiple times. We wanted to help owners and shelters/emergency services come together to identify lost pets during natural disasters or even during normal times to be reunited with their families!
After some research, we found that around 20% of pets that get lost, stolen, or affected by disasters are unaccounted for by emergency services and other people. These animals usually never end up returning to their owner because of these fragmented communication channels. Existing platforms treat everyday pet loss and emergency disaster responses as separate problems, leaving a major gap for unifying owners and shelters systems to reach and help each other out. Our app serves as not just a pet locator but also as a community for people to share and search for pets!

What it does

For users:

1. During onboarding, users will be able to submit basic pet information with pictures, health conditions, age, and gender
2. Users can then select if they found a pet somewhere in the neighborhood or in the wild as well as report a lost pet.
3. Selecting the "found" pet will allow the user to submit photos of the pet and based on the camera feed, PawTricia, our helpful AI tool will evaluate the current environment as well as the condition the pet is in and if needed, give additional steps to the user to assist the pet while also sending an alert request to shelters/emergency services and categorizing the request based on how dire the situation is so providers can properly allocate resources for the pet.
4. Selecting the "lost" pet will allow PawTricia to analyze your current pet pictures against photos that shelters or other people have recently uploaded to find where your furry friend is quickly.
5. Paws the Panic also has an educational standpoint to help educate users on what to do during a natural disaster through 5 game simulations(Hurricane/flooding, Tornado, Wildfires, Winter storms , Earthquakes) if their pet is missing or if they find an injured animal, and how the live connection between shelters/emergency services and owners plays out to help users get a better picture of how the process works. The simulation also includes prevention measures to educate users on how to be better prepared to handle these situations with a pet safely by their side.
6. Users will also receive notifications on emergency alerts and what to do with their pet in that situation, alerts about possible pet match finds for missing pets, resolved cases for pets and owners reuniting, and lastly general protection and prevention measures users can learn on a daily basis.

For shelters/emergency services:

1. Shelters will go through an authenticated log in to be able to access shelter specific dashboards. These dashboards immediately show the first responders "last pet seen" locations and timestamps with photos and an AI evaluation response from when the user took the photo. Shelters will be able to help find and track these reported animals and update their registry of pets at the shelter.
2. Shelters can check the photos against registered users photos in that location to match the pet with AI vision and contact the owners accordingly.
3. Requests and alerts of pets will go to shelters with enough supplies and resources listed and reroute requests to other shelters with more capacity to take on animals, trying to minimize the risk of overcrowding and disease spread.

How we built it

We used React Native to build a mobile app with TypeScript, SpaceTime database.
Nvidia Nemotron was used for our PawTricia AI bot to help users learn about disasters, image recognition for pet matching, as well as AI camera feed to identify the environment and the pet's current state while providing users with immediate feedback and sending responses to shelters and emergency services.

My Role

I mostly worked on the frontend and coordinated the visual design direction for our project while also conducting user research and diving into a couple of existing case studies about lost and found pet apps to see where the flaws are and how to improve the situation. I wanted to try a more AI first approach to the design side, so I created very simple mock UI which I would then ask Figma Make and other agents to take into account the design system and build on top of our codebase. However, I built most of these screens on the wrong structure, so we needed to scrap my existing design and pivot fast (TT).

Demo

Try out our demo here~ repository

and check out our DevPost!

Challenges we ran into

It was difficult to access certain API keys as their requirements were really strict for usage, forcing us to bend our project in order to best implement these key features in the way we desired. As well as this, the journey from converting our Figma UI into actual frontend was a struggle as it was difficult to convert some features we originally desired into actual code. Hence, we had to adapt and make the best to create as innovative features that fit the mission we strove for.

Accomplishments that we're proud of

We're really proud of executing this idea that we are very passionate about and deeply care about into a working app that, hopefully, can be integrated into shelter/emergency services support systems as well as people looking to find their furry friend. We really wanted to think outside of the box when coming up with ideas. Paws in Panic really hones in on standing up for an underrepresented group, in this case, pet! Animals can't speak for themselves so we must advocate for them and help them out in many situations.

Our app logos and design systems were also fun to make as well as exploring and experimenting on various art styles to play in our simulation game. Also maxxed all our AI agent credits LOL.

What we learned

1. We Learned how to adapt open source computer vision models to our specific needs and then deploy it in our app.

2. We learned how to implement Spacetime DB to sync to clients automatically in order to update client side features in real time.

3. We learned how to use Nvidia Nemotron to interpret user requests and then use a set of tools to help guide the user to the next step.

What's next for Paws the Panic

1) We plan to refine the simulation features to include a greater diversity of situations/pets, cleaner transitions, and more UI that can help simplify users needs but still bring them a moment of peace during their time of distress.
2) We will continue to test and feed the different machine learning features in order to get the most accurate results for the image and result recommendation systems.
3) The next thing for the app is the usage of more databases/APIs!

Hackathon Experience

We attended HackUTD!! The atmosphere was really nice and all the organizers were willing to help and jump in at all times :)) There was so much cool merch (WE GOT CUTE LITTLE POYO NIGHT LAMPS OMGG) and fun events with great opportunities to network with company sponsors and recruiters! Something I will specifically remember is singing to Justin Beiber's Baby at 1:15 AM hehe.