Disclaimer: All the views and technical details shared in this blog post are my personal opinion and its neither my Employer’s nor any particular vendor’s opinion or thoughts on this topic.
In the era of Industrial IOT 4.0 its becoming very common to deploy ML models to the IOT sensors /devices or to the IOT gateways which typically have lower memory/RAM footprint and lower CPU. Some of the important challenges in solving IOT usecases with ML or Deep learning at scale with any framework TinyML or others are as follows
Illustration of main barriers for the usage of TinyML in Industry projects
a) Selection of right set of toolsets, runtime and languages
b) Selection of right ML frameworks
c) The portability of such a ML/DL framework across a wide variety of MCU chipset from different vendors.
d) Cost of engineering resources and different persona of engineers involved in moving the experimental notebook into a production grade model serving at scale with low latency.
Here is an illustration of the full architecture of the Edge_AI /MLOps for IOT usecase with MQTT based publish events to specific topics used to collect the data in real time from different IOT sensors to trigger the automated data pipeline.
Architecture for TinyML based MLOps workflow for the IOT usecase of Predictive maintenance (OEE)
Based on my hands on work in the field of TinyML for IOT and the current trends seen in MLOps market I came across this tool called Neuton.ai through a presentation of Danilo Pietro Pau: https://youtu.be/-OdREYE-BX8?t=4247 who gives a detailed overview of TinyML solutions and their challenges. Neuton.ai is solving all the mentioned above problems at scale. Neuton.ai is based on a proprietary framework to solve IOT use cases in different verticals. The Neuton platform enables models to be built automatically, neuron by neuron, of optimal size and accuracy.
Key aspects of this platform is solving all the above mentioned problems at scale. Neuton.ai is based on the TinyML framework to solve IOT usecases in different verticals. One of the key aspects of TinyML is it negates the need to do NAS (Neural architecture search) in a subspace which improves the model training time as it can reach convergence along with a lowest number of coefficients/weights required for the model. The size of the trained model along with dependent artifacts packaged comes to be around ~ 4–5 Kb which is really good for deployment in IOT sensors without any model compression. Generally an ML model could be evaluated with the following criteria as per “PLASTER” terminology which stands for P- Programmability of model, L- latency of Model serving , A- accuracy of the Model, S- Size of the model and Security aspect of model serving ,T- Throughput , E- Energy efficiency/Energy consumption of the model while serving and R- Rate of learning of the model. Neuton.ai is meeting almost all the above evaluation criteria with good set of metrics is extremely energy efficient as it takes about few mW to make the predictions in real world deployment. Based on atleast my personal experimentation with one IOT usecase for a predictive health monitoring for OEE (operational equipment efficiency) with Neuton.ai platform I see its performance is better than training with QKeras tinyML as illustrated below with model metrics.
End to End ML and DL frameworks for training in Constrained IOT sensor environments
QKeras framework based 2D Convolution with Keras Quantizer
Trained Model accuracy and Training hours for different TinyML approaches across MCU’s
Experiments show that all these issues are resolved by the new framework. You can see that the model footprint is very low as per illustration below including all dependent artifacts.
Different Model Size’s for different IOT usecases along with the ML techniques
The current ML/DL toolsets, framework and ecosystem involved in Constrained ML for IOT is illustrated below (from the Webinar). As illustrated there are number of challenges in NAS, hyper parameter tuning/HP and complex layers , number of Coefficients derived et al. All the above factors attribute to cost and timeliness of the delivery of the trained ML model to production. All the work experiments show that all these issues are resolved by the new framework on which i am working with the IOT dataset from real world IOT gateways. As you can see from the below illustration with training from Neuton.ai platform that the trained ML model footprint is very low ( ~4–5 KB maximum including all artifacts for deployment). The existing work is done on GCP GKE platform and in my future blog post, I am planning to cover the code aspects, share github repo along with Kubeflow related workflow orchestration to leverage TinyML based MLops platform for IOT usecases.
To summarize according to
Andrew Ng
both corporate projects and future academia research trends in ML and DL for the future should be more inclined towards “Data driven AI” modeling and not “Model driven AI” to drive the industry, ecosystem to move forward in accelerated manner. Engineering time and resources should be spent wisely on data cleaning, data verification, data pre-processing, data security, data observability task related aspects rather than spending time towards experimenting with different ML algorithms and frameworks to get the job done which will again become a technical debt for the product over a period of time which increases the product sustenance overhead cost. Hence I suggest all ML/DL engineers to do utmost due diligence in all the aspects discussed above in the selection of toolset, platform for your projects.
Feel free to share your thoughts and comments below on this interesting topic which is an intersection of IOT,Data Science, AI,ML/DL and embedded architectures for Product definition, Product Strategy and MLOps Solution architecture .
Top comments (0)