Koalafinder

A YOLOv4 based solution for detecting koalas in thermal imaging

Setup

Ubuntu 18.04

Git will need to be pre-installed and configured for access. Ask Tim to be added to the repository user list.

sudo apt update
sudo apt install python3
python3 -m pip install --upgrade pip
git clone https://github.com/realtimshady1/Koalafinder.git
cd Koalafinder
python3 -m pip install --upgrade -r requirements.txt

Google Cloud

Google cloud has pre-configured image called Debian 10 based Deep Learning VM with CUDA 11.3 M89 to save time. Cloud environments should be reserved for training models.

# Install packages
sudo apt update
sudo apt install -y python-pip3 apt-file libopencv-dev
pip3 install -U setuptools
pip3 install opencv-python
sudo apt-file update

# grab unrar
wget https://www.rarlab.com/rar/rarlinux-x64-610.tar.gz
tar -xvf rarlinux-x64-610.tar.gz
sudo cp rar/unrar /usr/bin/

# Clone private repository using Github Token
TOKEN = ###Token ID### Ask Tim for access
git clone -b dev https://$TOKEN@github.com/realtimshady1/Koalafinder.git
cd Koalafinder/

Copying files from the compute engine to a local machine requires gcloud's CLI. Once configured, the following code is an example to copy files across in the local terminal:

gcloud compute scp biodiversityuav@instance-2:/home/biodiversityuav/Koalafinder/test.csv C:\Users\timot\Desktop\test.csv

Dataset

The derivation of koala data is slightly complicated can be downloaded from the Google Drive. To download everything, use the following command:

cp tools/download_data.sh data/
cd data/
./download_data.sh

Image data is stored as the original videos into a new obj folder the data/ folder along with the annotations in the following structure

└───data
    └───20210512_DJI_0026
        ├───00000.jpg
        ├───00000.txt
        ├───00001.jpg
        ├───00001.txt
        ├───00002.jpg
        ├───00002.txt
        └───...

The training data split files [test.txt train.txt valid.txt] are generated using the write_dataset.py script

python3 write_datasets.py 70 15 data .

YOLOv4

GPU

Ideally there is a GPU provided for YOLO to run on but it doesn't necessarily need one. The following sections continue with the assumption that there is a GPU to train with. In the future, a section will be made for how to build without the use of a GPU.

Make sure that the correct GPU toolkit are installed. Run the following commands to verify that the versions are correct.

nvidia-smi
/usr/local/cuda/bin/nvcc --version

Build

To build YOLOv4 to run as the primary object detector, we need to clone AlexyAB@github's darknet repository and build according to their instructions. This is necessary so that YOLOv4 can run efficiently on your specific machine.

For the sake of convenience, a pre-built YOLOv4 is available in the build/ folder according to the following specifications. Thus if your environment meets the following constraints, you may skip the Build step.

Component	Version
GPU	Tesla T4
CUDA	11.0
NVCC	11.0.221
cuDNN	8.0.5
OpenCV	3.2.0

To create a new build, use the following set of commands. The Makefile will also need to be updated to reflect the GPU that is being used.

# Build
sudo rm -r build
git clone https://github.com/AlexeyAB/darknet build
cd build
sed -i 's/OPENCV=0/OPENCV=1/' Makefile
sed -i 's/GPU=0/GPU=1/' Makefile
sed -i 's/CUDNN=0/CUDNN=1/' Makefile
sed -i 's/CUDNN_HALF=0/CUDNN_HALF=1/' Makefile
sed -i 's/LIBSO=0/LIBSO=1/' Makefile
sed -i 's/NVCC=nvcc/NVCC=\/usr\/local\/cuda\/bin\/nvcc/' Makefile
sudo make
cd ../

Build errors attributed to /bin/sh: 1: nvcc: not found can be fixed by directing NVCC=nvcc to the location of CUDA's NVCC location. Usually nvcc is stored at /usr/local/cuda/bin/nvcc

Build errors attributed to opencv2/core/version.hpp: No such file or directory occur because sudo apt install -y python-pip3 apt-file libopencv-dev failed

Copy the files to the working directory

# Copy files
cp build/darknet ./
cp build/darknet.py ./
cp build/libdarknet.so ./

Config

A config file is needed to run the YOLO model and the parameters can be modified to suit each different application

The following is a recommendation on how best to tune the YOLO model configuration

batch = 64
subdivisions = 1
width = 512 but anything divisible by 32 is fine
height = 512 the same as width
max_batches = (# of classes) * 2000 but no less than 4000
steps = (80% of max_batches), (90% of max_batches)
filters = (# of classes + 5) * 3

Anchor Points

Custom anchor points can be calculated using the following command

./darknet detector calc_anchors obj.data -num_of_clusters 9 -width 512 -height 512

num_of_clusters: refers to the number of bbox detection points which can be counted in each [yolo] tile in the config file.

Make sure the write_datasets.py datasets have been generated

Usage

Training

Training the neural network can be completed using

./darknet detector train obj.data cfg/yolov4-tiny.cfg backup/yolov4-tiny.conv.29 -dont_show -ext_output -map

Evaluate

Evaluate the neural network on the test dataset

./darknet detector map obj.data cfg/yolov4-tiny.cfg backup/yolov4-tiny_best.weights -iou_thresh 0.50

Test

Test the neural network on one image. This should be one from the test.txt dataset

./darknet detector test obj.data cfg/yolov4-tiny.cfg backup/yolov4-tiny_best.weights data/obj/DJI_0026_00001.jpg -ext_output

The neural network will generate a predictions.jpg file as the output

Inference

To perform inference on a test image

python3 yolov4_image.py cfg/yolov4-tiny.cfg obj.data backup/yolov4-tiny_best.weights data/obj/DJI_0026_00001.jpg

To perform inference on a test video

python3 yolov4_video.py cfg/yolov4-tiny.cfg obj.data backup/yolov4-tiny_best.weights data/DJI_0026.MP4 -post_process True

Labelling

The program that is used for labelling can be installed through the following repository DarkLabel

A pre-configured executable is also available in the tools/ folder

Progress

The most recent run example is shown here