This project aims to create a small, energy-efficient, standalone module that can classify several types of flying vehicles.
An ESP32-CAM microcontroller running this code will take a photo from the on-board rear camera, run inference via a local pre-trained model, and display the classification result on its OLED display. This sequence is triggered by a manual button press. Currently, the project is configured specifically for pinouts on the LILYGO TTGO T-Camera ESP32 (~$15).
- Vehicle classification model is trained from scratch, based on the MobileNetV1 architecture, and further compressed with full integer quantization of weights.
- Completely local inference on the ESP32 platform.
- Results produced from the trained deep learning model via TensorFlow Lite Micro.
- Install the ESP-IDF toolchain v4.0 with their Get Started Guide.
- Activate the toolchain by running
get_idf(if installed normally). - From the project directory, run
idf.py build. - With the target controller plugged in, run
idf.py -p <your port> flash. If you are unsure which port to use, consult the guide linked in step 1.
See the training_model directory.
-
The output classes are limited to the 5 options I chose (compatible with dataset). Can be improved.
-
With the TTGO T-Camera ESP32 module, the OLED display driver seems to have I2C issues. I addressed this for now by repeatedly re-instantiating the connection. There are corresponding error messages in serial output, but functionality is fine.
-
Due to ESP32's limited DRAM, the model network cannot get any wider, even with an increased application partition size.
-
Accuracy is not great because of an inaccurate dataset (explained in training section).
Due to Tensorflow Lite Micro's general lack of documentation, I consulted and used some functions from the person_detection example they provide. I have included the original license (Apache 2.0) for that project with modifications by me listed.