Origin Story Part I
This is my first post for a project I first started working on in 2017; this first post will probably be long and more story than technical but I want to give the back story and history of this project as I commit to giving regular updates on its progress.
The Project
The project is a c++, computer vision based system that detects a users heartrate and other signals to infer and flag potential health issues. You see an old demo of it running on a raspberry pi here
Motivation
When I started the project the main driving force was curiosity and a frustration with a product I had developed in my first job after college, I also made the assumptions that if I mapped the path correctly I could use the project as a vehicle to really push my development skills to the next level. The final aspect of starting the project was I wanted something to use to get involved in the then new startup and development community in Dublin at the time.
As I sit here now typing I know that the startup lifestyle of extreme focus and half decades of iterations on a single product isn't for me; but at the time I was excited about the freedom of working of projects I was passionate about.
Project Backstory
I had found my time in college very frustrating, juggling three part time jobs with then undiagnosed ADHD and Dyslexia made for a miserable experience on my part. It might not come as a great surprise that my grades were equally as miserable, however glimmers of light did appear. I really excelled in the labs and in the few software modules we had. It was just a shame that the labs only provided a pass fail grade and the software modules were exterminated with classic pen and paper end of semester tests(I still get a cold sweat thinking about exam halls). When you are failing at most subjects I think its natural to fixate on the few that you excel at, for me this was electronics and coding, while ADHD is a barrier to classic academics its can be a superpower for Hardware and Software development in which getting lost for hours tinkering left me with a love of both and an intimate understanding for any technology I worked on.
Hitting the job market after college was a rough experience to say the least, while all my friends were starting exciting well paid graduate programs my inbox was worryingly empty. Luckily I was tossed a life line by a new Entrepreneurship program and scholarship at my university. I threw together an idea and quickly built a prototype using a raspberry pi and some python, the prototype itself has been lost to time.
This was a hardware based project that used a wearable monitor that were all the rage at the time. The idea was simple use heart rate as an early warning system. It wasn't the most original idea, I knew from tracking my own heart rate in the gym and on runs I could spot myself getting sick before the major symptoms showed up. It wasn't much of a logical jump to realise I could do some basic signal analysis to automate this and maybe even mine more info.
Having a demo really helped and I got a place on the program, it had a monthly stipend and more importantly came with a place to live in the middle of a rent crisis. I went to the daily mentoring sessions and the networking sessions, I drank the startup culture cool aid of fake it till you make it; for a few months at least.
After three months the program came to an end, at the time (and now) people generally didn't invest in 20 somethings with nothing but an idea. So I brought what I had built to a few local large companies and managed to sell the idea to a company that made horse blankets. The salary was terrible barely liveable and I was the only engineer in the whole company but the experience was incredible. This company had manufacturing on site and in Asia, with in six months we had developed a product and brought it to market, we won numerous trade show innovation awards and I got to travel a lot. In the end I moved on from the project to bigger and better opportunities, I was happy with my experience and felt lucky but I was still frustrated with not being able to develop my original vision.
I wanted to build a cheap passive system for users to get an early warning on health issues as prevention is the best treatment, in the end I had built a temperature sensor that guilt-ed users into buying heavier horse blankets from my employer.
One of the main reasons we couldn't add heart rate sensors was because of the requirement of skin contact, this led to chaffing and all kinds of infections on animals. It was fundamentally an issue of comfort and passive monitoring, similar to products designed for humans the value of the data only comes after long periods of time so any product really need to be a silent observer with minimal obstruction. Based on this frustration I came to the conclusion I would need to build a device that was non contact and remote.
Ideation and iteration
My first guess was audio, it was a simple assumption I can hear a heartbeat if i'm close and a doctor uses a stethoscope. I built some prototypes and it was a clear improvement but the hardware was big and bulky and there was serious issues with cost and battery life, the value of the data in a medically un-certified device isn't high enough to justify large investments in time or money for most people.
With the hardware there was also a large investment upfront that wasn't feasible for a few broke graduates barely able to pay rent.
As hardware wasn't an option the next logical choice was to use mobile phones. Building an app wouldn't require any capital and comes with a ready made distribution path. The audio was the first thing I tried but most people don't put t phones close to their hearts on a day to day basis. It was pretty clear to me there wasn't much going on around a persons pocket to pick up a heart rate with so the mobile device quickly became a dead end.
My next instinct was that a heart rate is signal that repeats regularly so If i can measure something its present in I should be able to filter it out with a Fast Fourier. This train of thought lead me to a number of papers published on using computer vision to detect heart rate.
Heart rate from computer vision
This part was really exciting I could see from the papers that by using computer vision it was possible not only to find the heart rate but that it was also really accurate if done right. The best part was it was all software based I didn't need to invest in an expensive hardware platform I could use a laptop or a smart phone.
I came a cross a method developed in MIT that used head movement and not the more popular skin pigmentation that you might be familiar with, this came with advantages in accuracy and universality. They built their system in matlab and it was a batch post process approach. I got really excited at this point and decided I wanted to build a real time version. This was exciting for a few reasons the first and obvious was it solved my problem but the other reason was I needed a software project to get lost in to push my software skills to switch from a hardware engineer who could code to a pure software engineer.
I stayed up all night and built a quick prototype in python using opencv. It was held together with wishful thinking and spaghetti code but it proved the concept to me. I was between jobs at the time and free lancing to try cover my rent so I developed the idea into a system for hospitals to automate a task for nurses in which they have to take the heart rate of 8 patients every hour and use it in a calculation (in pen and paper) to asses the risk to a patients life. My idea was to automate this task take the cognitive load off overworked nurses who often had all there attention on one or two patients who were really struggling. I knew the certification process would be really hard and need investment and that would probably require a patent and business plan that made opensource impossible but at the time I thought it was the best approach. I have since realised the error in my logic and the naivety in thinking only enough profit would be made to fund the system.
From prototype to proto business plan
I used the prototype and idea to win a place on a state run entrepreneur program that came with a monthly stipend and mentoring. The stipend was incredible for the first time in my adult life I could breath, I wasn't terrified of eviction or where my next meal would come from. I would work on the business side during the day and code late into the night. This was an incredibly productive part of my life however I will stop now and pick up the story in my next post next week.
Next up Part two
In the next part I will talk about why I switched to C++, how I learned this amazing language and my late discovery and total conversion to opensource development.
Top comments (0)