The car can follow circuits over floors not seen before with only a 320x240 resolution ~$10 camera 📸. It does it locally, without accessing the internet, employing a Raspberry Pi and doing the model inference in an Edge TPU. The work is based on DeepPiCar by dctian.
On top of Dctian's work, it adds the following capabilites:
- Achieve autonomy with the SunFounder kit camera (lower quality)
- Keyboard training: drive the car with the keyboard & collect labelled training images
- Autonomy over different floors: thanks to more quantity & diversity of training data
- Small tweaks to improve efficiency, like multi-threading for preprocessing
- Updated some deprecated functions (2024)
Due to the low-quality camera, no reliable object detection was achieved.
Autonomous.driving.demo.mp4
Sun Founder PiCar-V kit
Raspberry Pi Model 3B+
Coral Edge TPU Accelerator
Accessories:
- 2 18650 batteries and a charger
- Micro SD card
The code is programmed in Python, using the car manufacturer's libraries and various machine learning libraries such as Tensorflow, Keras and OpenCV. The OS of the car is the Raspberry Pi OS.
As a summary, the steps to build the SmartPiCar were the following:
- Formatting of the micro SD card in FAT32 format and installation of the Raspberry Pi operating system.
- Assembly of the car, following the manufacturer's instructions.
- Installation of the necessary libraries and their dependencies.
- Development of the deep learning model.
- Development of the code to collect training data.
- Collection and cleaning of the training data.
- Training of the model.
- Deployment of the model in the accelerator, following the manufacturer's instructions.
- Repetition of steps 4-7 until a good solution was achieved.
- Hardware malfunctioning: both the micro SD card reader and one car servo failed frequently
- Low quality camera: this was the bottleneck of the project. The angle of vision was so low that the lane lines had to be very close to the car and sudden turn were not possible.
- Camera positioning: since the camera is not fixed to the car & 2 degrees of variation can considerably affect results, there needs to be a methodology to ensure the position of the camera is always the same.
- Processor's speed: The Raspberry Pi struggled to handle remote development and execution of the code.
- driver: stores the code for driving the car
- autonomous_driver.py: performs deep learning model inference and returns the steering angle of the car for a given image.
- hand_coded_lane_follower.py: drives the car autonomously but programmed explicitly. It also returns the steering angle.
- save_training_data.py: from a driving video, saves an image of each frame with its respective steering angle, calculated by hand_coded_lane_follower.py.
- smart_pi_car.py: Drive the car in the way chosen by the user: manually or using one of the previous programs. It also allows you to save tagged images during manual driving.
- models: contains the code for the machine learning models and several trained models
- code/cnn-nvidia: best performing model. It is based on an architecture proposed by Nvidia.
- models/lane-navigation-best-model: the best model obtained, used in autonomous_driver.py.
This project was made by Andrés Martínez Martínez for the Final Year Project of Computer Engineering in Polytechnic University of Valencia.