Running ML and AI on constraint IoT embedded devices — A game changer

Being an embedded developer, i have always envisaged “Machine Learning” (ML) and “Artificial Intelligence” (AI) technologies requires humongous amount of computing power to perform their expected behavior. So to attempt any kind of AI or ML on any embedded device like raspberry pi or microcontrollers like arduino had always been daunting. Interestingly, “Tiny ML / Tensorflow Lite”, focuses on taking ML to constraint IoT devices operating at low power or ultra low power range.

What is TinyML / Tensorflow lite ?

According to the TinyML Summit it is defined as “Tiny machine learning is broadly defined as a fast growing field of machine learning technologies and applications including hardware (dedicated integrated circuits), algorithms and software capable of performing on-device sensor (vision, audio, IMU, biomedical, etc.) data analytics at extremely low power, typically in the mW range and below, and hence enabling a variety of always-on use-cases and targeting battery operated devices.”

According to tensorflow, “TensorFlow Lite for Microcontrollers is an experimental port of TensorFlow Lite designed to run machine learning models on microcontrollers and other devices with only kilobytes of memory. It doesn’t require operating system support, any standard C or C++ libraries, or dynamic memory allocation”

Why TinyML/Tensorflow Lite ?

There are approximately 200+ billion microcontrollers in the world today. The market study shows, 28.1 billion units were sold in 2018 alone, and IC Insights forecasts annual shipment volume to grow to 38.2 billion by 2023. Global AI in embedded IoT devices market will approach $26.2B USD by 2023 as quoted by prnewswire. These will start enabling the future of embedded IoT devices on the so called “true edge” that can do really great things which we thought were not possible.

Though edge computing has been gaining momentum, it cannot solve all problems most importantly data privacy. Data privacy concerns are ruling the world today and especially in scenarios like smart home where consumers are truly excited about home automation and at the same time equally concerned for data leaving the premises. TinyML/Tensorflow Lite, if implemented, can confine the processing of data on the IoT device itself there by ensuring privacy

Another interesting problem that can be addressed is in areas of limited or no connectivity such as rural areas, sea etc., is reliability, where either cloud or edge can do no help in connectivity. TinyML/Tensorflow Lite can enable the ability to perform certain ML and AI operations locally, there by producing several advantages

In mission critical applications, where a millisecond delay could also cost, “Latency” is an important parameter. Though edge computing reduces the network latency to a significant extent, executing certain ML operations on the device itself could save further.

In addition to above, factors like energy consumption, network bandwidth etc., could also prove beneficial in terms of using ML and AI operations on the device itself.

The above are many potential advantages of using TinyML/Tensorflow Lite than edge computing

Books & Resources

https://www.tinyml.org/ has plethora of information on the concept and their summits etc.,

“TensorFlow Lite” — https://www.tensorflow.org/lite/microcontrollers

A blog on tinyML and technology aspects in detail was an interesting read https://www.ericsson.com/en/blog/2019/12/tinyml-as-a-service

https://venturebeat.com/2020/01/11/why-tinyml-is-a-giant-opportunity/ is another good read on why ML on IoT embedded devices could be game changers and a giant opportunity as well

https://www.enterpriseai.news/2020/06/08/qeexo-takes-tinyml-to-aws-cloud/ takes tinyML to AWS cloud is another interesting offering

A book from the creators seems to be interesting to understand TinyML

Conclusion

With more maturity and adoption, running ML and AI algorithms on memory, power constraint IoT embedded devices seems to be a clear game changer in the field of IoT and bring plethora of opportunities in the same. It will be interesting to watch this field and try few algorithms on either a raspberry pi device or a microcontroller