Plumerai is a company established in 2017, which specialises in enabling deep learning on small, low-cost and low-power hardware. Plumerai believes that this is the most effective way towards a future where machine learning (ML) helps everyone in the everyday world. Moreover, Plumerai’s people-detection AI for edge devices is highly accurate, while consuming minimal resources.

To further improve its AI software, Plumerai combined it with Espressif’s ESP32-S3, an SoC with powerful AI acceleration and strong security features, which has been especially designed for demanding AIoT applications. With the help of ESP32-S3, then, the Plumerai software provides masterful deep-learning that enables inference on low-power hardware. More specifically, Plumerai’s people-detection model, which is trained on more than 30 million images, runs at 3.3 frames per second in just 166kB of RAM, in only one core, detecting up to 20 people at distances over 65 feet (≈ 20 metres). Thus, the second core of ESP32-S3 is available for additional tasks. This may prove particularly useful in various everyday scenarios, e.g. when tiny smart-home cameras based on ESP32-S3 send notifications of people arriving in someone’s property. Also, lights may turn on automatically when people arrive at home, and so can the air conditioning. Traffic lights can instantly detect pedestrians arriving at a junction.

The Plumerai people-detection software functions in both indoor and outdoor settings, in dimmed lights, and even against moving backgrounds. The Plumerai model runs completely at the edge and all computation is performed on ESP32-S3, which eliminates any costly Cloud computation. What is more, no internet connection is needed, while the captured images never leave the device, which enhances the reliability and privacy of this solution.

ESP32-S3 is a dual-core XTensa LX7 microcontroller unit, capable of running at 240 MHz, with 512 KB of internal SRAM. ESP32-S3 also has 2.4 GHz, 802.11 b/g/n Wi-Fi and Bluetooth 5 (LE) connectivity with long-range support. Plumerai extensively optimised its people-detection software to take advantage of the custom vector instructions in ESP32-S3, thus providing a significant speedup compared to using the standard CPU’s instruction set.

A demo that runs on the ESP32-S3-EYE board has just become available. The board includes a two-megapixel camera and a liquid crystal display (LCD) which provides a live camera feed showing where and when people are detected. The ESP32-S3-EYE board is readily available from many electronics distributors. Interested parties can contact the company to receive access to the ESP32-S3-EYE demo. More info on how to use the demo can be found in the relevant Plumerai Docs. Last, but not least, Plumerai’s people-detection software is available for licensing from Plumerai.