DEV Community

Ryan Bergman
Ryan Bergman

Posted on Edited on

EVAde - Space Apps Health Monitoring Submission. Designed and Programmed by HS Juniors.

This is a submission for the MLH x DEV Writing Challenge

Our Project

Evade is a prognostic health assessment tool for Astronauts.

PSYCHOLOGICAL MONITORING - JOURNAL

At the beginning of the hackathon, we decided to plan a LOT, and something we noticed was mentioned was "behavioral health conditions." This steered our project in a different direction than being just a physical health assessment that tracks cardiovascular health issues. We ended up extending our project to include a journal section. This journal section would help us gather the psychological state of the astronaut. In space, communication with others is often limited, therefore, we implemented an LLM, to which the astronaut can communicate back and forth with, and a Twillio voice assistant that does the same. Something unique we did was setup a SPACEXSI voice agent that communicated directly with the user or through a twilio tunnel over their phone number. This single-handedly allowed for voice journaling capabilities. You may be thinking, how exactly did we extract data from the journal text/voice entries? To implement this, we stored the inputs from the astronaut in a Cloudflare database, to which we connected several AI models (with a focus on incorporating SpaceX SI models) through WorkersAI. These models work directly with the database, and they review the journal entries of the astronaut and predict the mood of the astronaut and details/what the astronaut may need to improve their psychological health (in example, a call with their family). This will come in handy when we discuss a different aspect of our app, the flight surgeon admin panel.

PHYSICAL/CARDIOVASCULAR CONDITION MONITORING - DASHBOARD

Aside from accounting for/tracking the psychological aspects, as well as sleep, of the astronaut, we wanted to discuss our expectations and process of development for the monitoring of the physical/cardiovascular condition of the astronaut. Something relevant to this is our team dynamics. Our team consists of 4 high school juniors and 1 sophomore, and we came together as one team because of our diverse experiences, leading to us each playing a unique role in the development of this project (Ryan B (me) - Frontend design, connecting backend to Frontend, and incorporating some LLMs, Ryan L - Data aggregation and inferencing, Daniel - voice assistant, domain registration, and deployment to the domain, Michael - development of non-generative, custom models and training them, and Decong, authentication, hosting of local models on his machine, debugging, and pull request reviewer). Michael has been researching ECG and EEG inferencing through machine learning. Our initial hopes were to have more hardware available and actually build a health monitoring system that captures data live, but this didn't end up happening. So, we developed a manual logging health feature in our app. Something we really wanted to do, but didn't get to, due to time constraints, was using a raspberry pi and connecting it through Bluetooth to a host that runs the web app and having that raspberry pi run various simulations. This would have really helped us take this project to another level with a future in sight, but this may be something we implement after global judging, to extend this project. That being said, on the physical side of the monitoring system, Michael's machine learning experience allowed him to train over 16 unique models (with some of them unfortunately being left out due to implementation issues, the fact that some of them relied on metrics that could not be simulated/relied on EEG data, and a broader issue of agentic engineering that we will address in the continued portion of this section), and clicking on some cardiovascular metrics on our website allows one to calculate and see relevant statistics using Michael's trained models.

PHYSICAL HEALTH (continued - implementation of local models)

We aimed to incorporate all of Michael's models, which included seizure detection. All of Michael's models were generally trained to an AUROC of greater than 0.8 generally, but the reason why we couldn't implement some of them was due to our lack of experience with handling his local AIs and his lack of experience with frontend. Through Cloudflare workers, we created a gateway for external python code and the local models to communicate with our project. Connecting this to our frontend resulted in us relying on AI and what it was describing was going on under the hood, but we had no way of proving that this was actually occurring. So, we had to research and implement the functionality above with the local models and cardiovascular metrics and ensure everything was up to par, which took a lot of time and led to us ultimately being unable to complete this.

We encourage everyone to check out the models section of our website. This is located in the demo section. Here we describe what the models are, their status (only 5 being deployed at the moment), and the relevant git commits that highlight their entry point/incorporation into our app.

ADDITIONAL DASHBOARD FEATURES

It didn't sit right with us to implement the bare minimum + a psychological section. Astronauts deserve much more than that (and justifiably so). This led to us implementing a return forecast and a weather forecast for spacewalking. The return forecast is meant to highlight when an astronaut must return home due to the most critical health circumstance. There are many factors that impact an astronaut's health, but the most critical one dictates when the astronaut must return. This was implemented through AIs baked into our database through Cloudflare and their synthesis of the stored inputs as well as partial input from Michael's models. Furthermore, the weather forecast is reliant on data we collected. Then this data is processed by Michael's EVA-12 model which then predicts when the next time the astronaut should space walk.

We encourage users to check out the Data section of our UI. This was Ryan L's work, and he did a great job incorporating several sources.

ACCESSIBILITY SECTIONS

Part of this challenge was implementing a simple UI with potential for multiple ways to analyze the data, gamifying the astronauts' experiences, and ensuring the astronauts were able to have directions in assessing their health. We implemented several things to cover all of these challenges. We added mission, calorie tracking (using the USDA api), and water tracking, and these contributed to XP/tracked levels to cover the gamifying aspect of this. We implemented multiple ways to view graphs, enhanced display of past/inputted data to cover the data portion of things. Lastly, we incorporated a tutorial section (without videos because we are all under 18), with a checklist of directions and a way to gain XP from reviewing this.

FLIGHT SURGEON UI
A big part of this project, which we planned for if we had extra time, was a panel for the flight surgeons. This panel also led to the messages section of our app. Flight surgeons have access to all astronauts (in both the overview and messages). They can click on an astronaut to see a highlight of their statistics, an AI summary, graphs, model overviews, and much more. On top of the Overview section we added a Astronauts section which facilitates examining specific astronauts and their data. As for the messaging part of things, astronauts have access to all flight surgeons (and not other astronauts), and vice versa for flight surgeons. This was by design but we will probably give everyone access to everyone in a future improvement. The messaging section works and we highly encourage users to go through the demo of our project and message/commit actions that will be reflected to the flight surgeons from the astronaut UI, and switch through the UIs (button at the top) to see how everything communicates. Something Flight surgeons exclusively see is the summary of the journal entries from the astronauts. This is, too, by design because if the astronauts know that their journals are explicitly being monitored they may be discouraged to submit entries. Something else that is unique is our implementation of emergencies. Astronauts can manually press and hold an emergency button on their side of things, BUT the most unique aspect of this is the AI emergency detection we have baked into the journal LLM and Voice AI. We recommend you test with the Voice AI and say indications of critical conditions to test this (our journal LLM broke 1 minute before the submission and we fixed it the minute after the deadline to global). Emergencies are highlighted and flight surgeons can resolve them.

SELFIES

Something we did not get to finish was the selfie section. Currently selfies will be stored, to the benefit of the astronauts, but there is not an AI that extrapolates mood and etc from them. They are stored in R2 objects on our database.

Demo

Link to website:

TRY SIGNING IN AND THE DEMO SECTION! You can sign in by clicking the sign in button at the top or the launch app button :)

Partner Technologies

We used many partner technologies. We recommend reading the above to gain a sense of everything (as we go very in depth on this). We are running out of time to submit this, therefore, I have to be brief. We hosted on godaddy, incorporated Twillio AI, Cloudflare database and authentication (which we wish we had mentioned in our presentation), and more.

Hackathon Experience

This was our first hackathon and it was really enjoyable

Aside from that, this is the section where we would like to discuss issues that we faced. We faced overambition, and our project constantly had improvisations last minute, just as this dev post is last minute lol. We probably did not cover everything here, but we encourage you to find out as much as you can from the UI! We had merge conflicts (a crazy source control graph due to multiple people working at the same time and switching from AUTH0 to Cloudflare authentication - this is what kept us up very late). Overall, we immersed ourselves, trying to highlight our skills, but something we could've improved on was the presentation. We were ambitious in trying new things, including but not limited to the Asteryx React-based framework. We learned that we should, in the future, really dial down our productions (because this is clearly too much to explain) and work more on the presentation.

Excuse the typos, the inconsistent bolding and capital letters, and poor grammar and spelling. Unfortunately, it is(was*) 12:03 am and this is being submitted late to my knowledge :(

IMAGES (thank you all for the opportunity!!):

Top comments (0)