AIoT is the combination of Artificial Intelligence and the Internet of Things. Combining the power of embedded systems and Ai to solve industry problems efficiently at a low cost. 

Standard IT solutions – AIoT research and development facility hosts tools and equipment to design and develop Ai devices that are small in size and accurate when it comes to solving problems. We use a combination of custom hardware design and industry-standard frameworks to solve problems.

Technology – Deep Learning on the edge

Objective

The objective was to create an infrastructure and a knowledge hub for creating Edge Ai devices that are small in size, accurate, power-efficient, and can handle deep learning inference at a low cost.

Keywords

TinyML, Tensorflow Lite Micro, Computer Vision, Deep Learning, Ai at Edge,  Anomaly detection

Methods used

For creating custom, small AI engines that can run for days on a Lithium-Ion battery, we had to optimize the device for power consumption and speed while keeping the accuracy of inference high. The process of creating an AI of Things device starts with problem analysis and hypothesis testing, where we see if AIoT is absolutely necessary for this task or if we can solve the problem using a simpler solution. If the problem demands an AIoT solution, we analyze the solution and see how we can solve the problem using existing methods. The first phase of developing the solution is to create a dataset. The dataset depends on the problem. ​

                When working with a Computer Vision based deep learning problem, the data will mostly be images. These images are to be taken from different environments. The dataset will be cleaned, pre-processed and labelled for training. The training shall be done on a much powerful system and the models obtained after the training shall be converted to tensorflow lite. This model can be run on devices such as raspberry pi hardware, android, and IOS mobile phones. But for running this model on an edge device, we have to convert it to a C byte array using standard tools to store it in a read-only program memory on the device, which shall be used for the inference. After inference, the results can be pushed to the cloud. The hardware product shall have a central processing unit, a camera or sensor modules that get input from the environment, a power supply (battery), and an internet connection. 

Applications of TinyML in Industry applications

Industrial Predictive Maintenance

Machines are prone to fault. Using TinyML on low powered devices, it is possible to monitor the machine and predict faults ahead of time constantly. This predictive maintenance can lead to significant cost savings. Ping Services, an Australian startup, has introduced an IoT device that autonomously monitors wind turbines by magnetically attaching to the outside of the turbine and analyzing detailed data at the edge. This device can alert the authorities regarding potential issues even before it occurs.

Healthcare

 The Solar Scare Mosquito project uses TinyML to curb the spread of mosquito-borne diseases like Dengue, Malaria, Zika Virus, Chikungunya, etc. It works by detecting the mosquito breeding conditions and agitates the water to prevent mosquito breeding. It runs on solar power and can thus run indefinitely.

Agriculture

The Nuru app helps farmers detect diseases in plants just by taking a picture of it by running Machine Learning models on the device using TensorFlow Lite. Since it works on the device, there is no need for an internet connection. This is a crucial requirement for remote farmers since they might not have a proper internet connection in their place.

Ocean Life Conservation

Smart ML-powered devices are used to monitor whales in real-time in waterways around Seattle and Vancouver to avoid whale strikes in busy shipping lanes.

Proof Of Concept 1 : Deep Learning at Edge – Warehouse People Count 

Hardware – ESP-32, OV7670 640×480 VGA CMOS Camera Image Sensor Module

Software – C, Tensorflow Lite micro

Description –  The objective of the project was to build an edge device that can continuously monitor the number of people in a room. In the initial stages, we used Raspberry 4B 8GB to do the inference. After training a custom model, deployed the model in Raspberry Pi and later converted the model to tensorflow lite, quantized it and used it with  Arduino BLE Sense for deploying it. We saw 50% reduction in cost. Later, using ESP-32 WROOM module, we could further reduce the hardware cost and improve the accuracy and speed of detection

Results – The hardware was able to detect the number of people in a room with 91% accuracy in real time and send the data to the cloud. The device operated on a 2000 mAh Lithium-Ion battery and the device was able to run for 7 days without recharging the battery. The data sent to the cloud was within 3 KB in size so we were able to reduce the size, cost, latency and portability of the device.

Proof Of Concept 2 : Deep Learning at Edge – Face recognition 

Hardware – ESP-32, OV7670 640×480 VGA CMOS Camera Image Sensor Module

Software – C, Tensorflow Lite micro

Description –  The objective of the project was to build an edge device that keeps track of the attendance without using fingerprint. The solution was built in the light of the rising number of COVID cases and most of the attendance tracking systems use fingerprint scanners. The user interface for this application is a web server and the users can add/remove faces. When a face is detected, it checks for the faces to recognize the person. When a person is recognized, based on the time, the data is sent to the cloud and stored for record keeping. 

Results – The hardware was able to recognize faces with 93% accuracy in real time and send the data to the cloud. The device operated on a 2000 mAh Lithium-Ion battery and the device was able to run for 7 days without recharging the battery. The data sent to the cloud was within 5 KB in size so we were able to reduce the size, cost, latency and portability of the device.