DEV Community: Ivelin Ivanov

Use JavaScript for frontend code and Python for backend

Ivelin Ivanov — Tue, 19 May 2020 21:01:45 +0000

Form or function? Which comes first?Most successful products strike a good balance.

The following is a biased opinion based on personal experience with the respective programming platforms, their developer communities and libraries.

It has been almost a year since I started ambianic.ai on github, which gave me the opportunity to work almost daily on JavaScript(often mixed with TypeScript) and Python code. Previously I've had years of real world experience with Java, SQL and to lesser extend C, C++, Haskell, Prolog, Lisp, XSLT and various other imperative, functional, logical and OO programming languages.

One of the early technology decisions I had to make for Ambianic.ai was choosing platforms for:

The frontend - in this case a Progressive Web App
The backend - a local edge device that runs video processing and AI inference without sharing any data with any cloud accounts. Here is a high level architecture diagram.

At the time the decision was based on days of reading and researching. Eventually I settled on:

Frontend: JavaScript/TypeScript with VueJS, Vuetify and many other npm libraries.
Backed: Python 3 with TensorflowLite, Gstreamer and many other pypi libraries.

Intuitively it felt right to place a bet on browser based advancements for the UI. It seemed like there is a much bigger critical mass of effort put into solving the user - computer interaction problem via browser based technologies.

Simultaneously I had a chance to contribute to Tensorflow IO and consecutively meet with core contributors at the 2019 TF Contributor Summit. It was quite clear that most of the ML APIs were Python first. That made it an intuitively appropriate choice for the backend platform.

With some months of hand on experience, I have a bit more solid argument base to stay the course.

Why JavaScript is the right choice for UI

The web browser ecosystem is dominated by people who are working non-stop to solve the problem of intuitive, frictionless user - computer interaction. End users come in all shapes and forms, from a large variety of cultures, languages and professional backgrounds.

Building apps with great UX is a very very hard problem. People are emotional, subjective, irrational and inconsistent. Making users happy requires implementing and reimplementing the same functionality endlessly with iterative tweaks, A/B tests and multiple alternative ways for users to access the same features. This is much more of an art form than science.

As the Michael Jordon of AI said in a recent interview, we are as close to figuring out the human brain as the Ancient Greeks were close to landing on the moon. We are still dreaming more than understanding.

On top of that, user devices and browser versions span thousands of different permutations. Frontend code has to accomodate all kinds of screen sizes, outdated browsers, internet bandwidth and input modes.

All these issues reflect how browser based JavaScript APIs and libraries are designed and implemented. They have become very effective in solving messy UI problems. And that also has impact on readability of JS code. It often seems repetitive, irrational and sub-optimal from a scientist' point of view. Even though the resulting app may be perceived as user friendly and intuitive by 98% of users.

But then again do you know many scientists who are great at building slick UI? So let's leave the UI to the frontend artists.

But you know what scientists are really really good at?

Why Python is great for backend code

Scientists are great at minimizing, optimizing, mathematically correct, consistent, clean and beautiful code. Scientists are not shy to throw away hypothesis that are proven wrong and quickly replace with better, tighter, improved ones.

Python evolved mainly through the exact sciences community - physicists, mathematicians and computer scientists. Its adoption and use in the industry has also evolved through numerical and data problem solving.

Python code is often clean, consistent and self-explanatory. To some extent because of its syntax style, but I think to a much bigger extend due to the kind of people who wrote it. The pythonic way is to do things consistently across the board. Do one thing and do it right. No much room for artistic variations.

One of the most prominent examples is the lack of a do while loop. The commonly accepted reason for the rejection of this construct is that it breaks the Python block indentation rule. Get with the program, like it or not, science moves on.

This tradition of exactness and minimalism has led to an ecosystem of libraries and best practices that make Python a much easier choice for backend code where hard business problems are solved without immediate concern for user interaction.

REST, GraphQL, gRPC and other remote interfaces are here to stay as the essential glue between frontend and backend code. I am convinced that it is better in the long run to add an extra layer of connectivity than forcing a UX developer to think like a data scientist or vice versa. I think even generalists appreciate and use the right tools for the job.

Video cameras tested with Ambianic.ai

Ivelin Ivanov — Mon, 11 May 2020 18:22:24 +0000

This post is periodically updated to list specific camera models tested with Ambianic.ai by people in the community. Obviously, there are no guarantees and warranties of any kind. This is just a voluntary effort to help you find a camera that works for your environment.

Outdoor:

Brand	Model	Price	Store URL
Anpviz	5MP PoE Dome	$53	Amazon

Indoor:

Brand	Model	Price	Store URL
Reolink	C1 Pro	$55	No longer sold
Reolink	E1 Pro	$50	Reolink

If you know of a camera that works with Ambianic.ai, please post a comment about it below. Thank you!

Ambianic.ai user personas and product direction

Ivelin Ivanov — Mon, 11 May 2020 17:41:01 +0000

Over the course of the COVID19 lockdown, we participated in 12 virtual hackathons and were fortunate to have over 230 volunteers supporting the project.

One of the discoveries in the course of this experience was that there are two major types of user personas we see:

DYI folks and developers who love the ultimate control and flexibility of an Open Source privacy preserving surveillance solution.
End users who love the concept of a no nonsense privacy preserving solution to help care remotely for family members.

User group 1 (or to be Bay Area buzzword compliant, we can call it a cohort) cares more about freedom to fork, hack and integrate the code any way they like with any other systems they want to. This group is about 2% of all people who have been involved in some capacity with Ambianic.ai.

User group 2, doesn't have the skills, time or plain interest to get messy with integrating an AI component with an existing camera system. They would rather buy a new turn-key intelligent camera up to $200. If they had an old camera installed (presumably a $50-100 investment), they would rather return it and replace with the a new one or simply donate it to a charity. If the old camera has been used more than 6 months, then its a depreciated asset that served its purpose and can be now written off. $100 is generally a tolerable pain threshold as compared to spending hours on integration or hiring an expert to do the work, which would cost much more than a new drop-in replacement.

Armed with this user data with quantitative and qualitative parameters, we had an open discussion on our slack workspace and decided that both user groups are important to the long term success of the project.

User Group 1 is vital, because these are the kinds of folks who can thoroughly dissect the software and offer valuable constructive criticism, contribute code, docs and in the future help grow the core team.

User Group 2, the remaining 98%, is obviously vital, because these are the majority of end users who we ultimately aim to provide value to.

Just to be clear, Ambianic.ai is primarily focused on adding tangible value to real users today. The fact that its open source does not contradict the primary goal in any way. Open Source is meant to enforce its validity of true value by removing proprietary layers of obscurity and superficial flashy claims.

As a high level roadmap decision, we are moving forward by acknowledging that both User Group 1 and 2 are important and the project should proceed in a way that doesn't lock out either one from participating.

To that extend, Luca Capra (one of our new team members) summarized the way forward better than I could:

I am convinced that every niche expect and deserve a specific UX. I think the core could be flexible enough to support generic functionalities while delegating domain specific challenges to a vertical application. This could happen through different layers of integrations, eventually reusable. An example could be an object detection pipeline for training and execution, adapted to home care but also to security. Same functional perimeter but with different specializations based on user requirements

Here is a high level architecture diagram that we will use as a North Star reference.

Community feedback is always welcome; please use the discussion box below.

Ambianic.ai UI mockup for edge device connectivity, May 2020

Ivelin Ivanov — Mon, 11 May 2020 15:56:21 +0000

One of the recent product discussions related to Ambianic.ai UX has been related to enabling remote users to setup initial connection to an edge device that is not on their local network.

The specific use case that came up was inspired by the COVID19 hackathons we participated in.

What if a family member can ship a pre-configured Ambianic Edge system to their isolated elderly or disabled dependents homes so they can better take care of them remotely. Currently the UI can automatically discover and pair up with an Edge device running on the same local network (e.g. home WiFi). But what if family members are not able to physically visit the home where Ambianic Edge will run.

A simple problem statement resulted in an extended UX and p2p connectivity (WebRTC) design discussion. We started with a high level state machine diagram, which helped but wasn't specific enough.

Then Bjorn added a sequence diagrams to illustrate his proposed implementation in PR 347.

As the PR discussion went on, new edge cases came up, which forced us to think hard about what's most intuitive to the end user.

This UI mockup diagram illustrates our latest agreement. Its open to community feedback; use the discussion box below.

Ambianic.ai p2p connectivity state machine, May 2020

Ivelin Ivanov — Mon, 11 May 2020 15:33:28 +0000

As the dev community around Ambianic.ai expands and new ideas are brought in, it feels like a good time to document the current p2p state machine. This should allow more productive discussions on our roadmap and design.

The state machine diagram below complements the recent deep dive WebRTCHacks article Private Home Surveillance with the WebRTC DataChannel.

Here is the source of the diagram to fork and edit. It was created with the excellent diagrams.net (aka draw.io) tool.

Feel free to ask questions or comment in the discussion below.

Ambianic.ai Continuous Integration and Deployment

Ivelin Ivanov — Thu, 23 Apr 2020 00:33:27 +0000

Ambianic.ai is now completely CI and CD enabled across all of its major components running respectively on mobile, cloud and Edge/IoT devices:

Progressive Web App (ui.ambianic.ai):
- Built with VueJS, Vuetify, and PeerJS
- Hosted on Netlify
Plug-And-Play Signaling Server:
- Built with Node.js and PeerJS Server
- Hosted on Heroku
IoT/Edge image:
- Built with Python, Tensorflow, Gstreamer and aiortc
- Hosted on a user home device: Raspberry Pi, Xbox, laptop or another machine.

Its amazing that across such a wide variety of frameworks and infrastructure, it is completely feasible to implement full cycle Continuous Integration and Deployment using 100% Open Source dependencies and 100% free cloud services generously supporting open source projects.

All three major Ambianic.ai modules follow the same flow, with the respective tools for each function:

Pull Request -> Continuous Integration -> Code Review -> Commit -> Semantic Release -> Graceful Auto Update & Deploy

Here is the list of Github Apps that make this possible.

Github Sponsors webhooks enables Open Source SaaS

Ivelin Ivanov — Fri, 17 Apr 2020 17:31:59 +0000

Github announced its new webhooks for sponsors API.

Offering perks to sponsors is a natural way to express gratitude. Until now that required keeping a list of sponsors based on email notification and makings sure that each sponsor is acknowledged according to their tier.

With the new sponsor webhooks, it is now possible to completely automate perks for sponsors. In fact it makes it relatively straightforward to build incentives into sponsor tiers such as premium feature access to hosted (SaaS) version of a project. This is fitting especially for end-user applications where open source code is an important part of building a dev community, but operating a live service adds real value to end users.

Open Source SaaS is a strong trend that more new gen open source SaaS products are adopting, including dev.to, MongoDB, Kafka/Confulent, Elastic, Wordpress and others.

Another nice touch of the new gitub sponsors update is labeling issues and PRs from sponsors. I suspect there is more to come along these lines to make it simpler (and fairly so) for project maintainers to attract and reward sponsors.

Great work Github team! Keep it up!

Trustless, privacy preserving surveillance system with AI capabilities

Ivelin Ivanov — Fri, 10 Apr 2020 01:56:55 +0000

Are you holding back on installing home automation in your house, because it requires cloud account sign up and you are not quite sure what happens with your data that goes to the cloud?

I was hit by the same hesitation after I played with Ring, Nest, Arlo and a few other plug-and-play cloud camera systems. The AI capabilities makes them very attractive, but the fact that I have to use a cloud API to download video footage recorded by cameras in my own home, does not sit well with me.

Why can't I get access to my local video camera stream directly on my local network, get a private and secure notification to my mobile app. Look at the AI detected object, pet or a person and make an informed decision what to do next? That should be possible without trusting a man-in-the-middle system to see, but not look at my data, right?

Absolutely! When I asked the question first a few months ago, it was not that obvious, but after a bit of tinkering it all worked out just as expected. The result is now available for everyone else who wants a trustless, privacy preserving surveillance system with AI capabilities. Check it out: https://github.com/ambianic

There is a 5 minute quick start guide to get you going, if you don't have time to browse the code now.

And if you'd like to read more how it works, follow this link:
https://blog.ambianic.ai/2020/02/05/pnp.html

Privacy Preserving Home Surveillance with AI, Pi and WebRTC

Ivelin Ivanov — Fri, 10 Apr 2020 01:39:36 +0000

Home surveillance systems are now mainstream. Easy to setup and easy to use. Mobile app to see who is at the door. Cloud backup for a subscription fee. All great. Except that privacy issues are getting out of control. In an article for WebRTCHacks, I cover the technical design and implementation of a solution that leverages WebRTC to prevent sensitive data storage on devices that aren’t directly controlled by the user.

https://webrtchacks.com/private-home-surveillance-with-the-webrtc-datachannel/

A small code contribution to the COVID19 fight

Ivelin Ivanov — Fri, 10 Apr 2020 01:33:54 +0000

On March 24, 2020, New York governor Andrew Cuomo sounded a dire alarm during a press conference amid an escalating COVID19 situation in his state. He then repeated on twitter: "My mother is not expendable. Your mother is not expendable. We will not put a dollar figure on human life."

COVID19 isolated many of us, but especially vulnerable are elderly and disabled family members who are left unattended for days or maybe even weeks in some states and countries with full military lockdown.

Remote home care is an emerging topic in the theme of healthy aging, but it is especially relevant in times of COVID19 isolation.

The holistic problem of home care is very complex and dynamically changing. We can’t solve it at once, but there are discrete steps we can take in the right direction. One specific area of home care is proactive and unobtrusive monitoring of the activities and the home environment of residents.

For example it is a well researched issue that falls can be a key indicator of health complications. In fact, falls are the leading cause of fatal and non-fatal injuries for older Americans. If there is no one present to observe a fall and take action, it may go unnoticed. Things can be even worse if the person is not able to move and recover on their own.

Other important health factors include eating patterns, exercise activities, medication compliance. Changes in voice may also mean trouble. Environmental risk factors include flooding in the house, gas leaks, smoke or fire.

When detected early, these risks can be mitigated with minimal intervention from family members or caregivers. Respectively if response is delayed, the severity of the problems escalates.

With all this said, an idea begins to form. If we are able to build an affordable, unintrusive system that detects early warning signs, it may improve healthy independent aging at home.

Fortunately there is a solid foundation of open source projects that provide the key building blocks for a secure, privacy preserving AI system that begins to solve these problems.

We have released a public Beta version that integrantes these building blocks and demonstrates feasibility. Given the positive community feedback, we are advancing the project further with participation in a series of COVID19 hackathons.

We need to work on these main areas:

Recruit volunteers in the home care community to test the system and provide feedback.

Select more ML models to address open use cases such as fall detection, gas leaks and others.
Work on implementing Federated Learning infrastructure to fine tune initial pre-trained models in a privacy preserving manner.
Improve UX of the Progressive Web App.

If this topic resonates with you and you have cycles to spare for social good, join the team at The Global Hack this weekend or COVIDathon next month. If you don't have time to contribute, you can also sponsor the project on github with a $1 or $5 donation.

We’re one of the 50 Austin Startups to Watch in 2020

Ivelin Ivanov — Tue, 07 Apr 2020 22:04:58 +0000

We’re one of the 50 Austin Startups to Watch in 2020 # BootcampsAustinhttps://bootcampsaustin.com/top-startups-to-watch-in-austin-2020/ …

#AI is now reading better than humans. (World Economic Forum)pic.twitter.com/EWnMV4GURY

Ivelin Ivanov — Mon, 06 Apr 2020 23:33:00 +0000

# AI is now reading better than humans. (World Economic Forum) pic.twitter.com/EWnMV4GURY