Skip to content

Latest commit

 

History

History
364 lines (311 loc) · 13.8 KB

README.md

File metadata and controls

364 lines (311 loc) · 13.8 KB
Raw

Fish Detection

Application: Overview

The Fish detection application uses the advanced Tiny-YOLOv3/YOLOv3 algorithm to automatically detect fishes in real-time camera streams. Fish detection applications can be used in a variety of different settings, including:

  • On-board vessels: Fish detection applications can be installed on fishing vessels to help fishermen identify and track fish schools.
  • Underwater vehicles: Fish detection applications can be equipped on underwater vehicles to monitor fish populations and study fish behavior in their natural environment.
  • Aquaculture facilities: Fish detection applications can be used in aquaculture facilities to monitor fish health and track fish growth.
  • Research laboratories: Fish detection applications can be used in research laboratories to study fish behavior and identify new fish species.

Key Features

Here are some of the key features of the Fish Detection Application:

  • Automatic Detection: The application utilizes Tiny-yolov3/yolov3 model for detection, identifying and localizing people specified within the provided frame.
  • Customizable Settings: Users can adjust the detection parameters by using the config file provided in the repository

It has following camera input modes.

Mode RZ/V2L RZ/V2H
USB Camera Supported Supported
MIPI Camera Supported -

Supported Product

  • RZ/V2L Evaluation Board Kit (RZ/V2L EVK)
  • RZ/V2H Evaluation Board Kit (RZ/V2H EVK)

Demo

Following is the demo for RZ/V2H EVK.

Application: Requirements

Hardware Requirements

For Equipment Details
RZ/V2L RZ/V2L EVK Evaluation Board Kit for RZ/V2L.
Includes followings.
  • MIPI Camera Module(Google Coral Camera)
    Used as a camera input source.
  • MicroUSB to Serial Cable for serial communication.
AC Adapter USB Power Delivery adapter for the board power supply.
MicroHDMI Cable Used to connect the HDMI Monitor and the board.
RZ/V2L EVK has microHDMI port.
RZ/V2H RZ/V2H EVK Evaluation Board Kit for RZ/V2H.
AC Adapter USB Power Delivery adapter for the board power supply.
100W is required.
HDMI Cable Used to connect the HDMI Monitor and the board.
RZ/V2H EVK has HDMI port.
USB Camera Used as a camera input source.
Common USB Cable Type-C Connect AC adapter and the board.
HDMI Monitor Used to display the graphics of the board.
microSD card Used as the filesystem.
Must have over 4GB capacity of blank space.
Operating Environment: Transcend UHS-I microSD 300S 16GB
Linux PC Used to build application and setup microSD card.
Operating Environment: Ubuntu 20.04
SD card reader Used for setting up microSD card.
USB Hub Used to connect USB Keyboard and USB Mouse to the board.
USB Keyboard Used to type strings on the terminal of board.
USB Mouse Used to operate the mouse on the screen of board.

Note: All external devices will be attached to the board and does not require any driver installation (Plug n Play Type)

Connect the hardware as shown below.

RZ/V2L EVK RZ/V2H EVK

Note 1: When using the keyboard connected to RZ/V Evaluation Board, the keyboard layout and language are fixed to English.
Note 2: For RZ/V2H EVK, there are USB 2.0 and USB 3.0 ports.
USB camera needs to be connected to appropriate port based on its requirement.

Application: Build Stage

Note: User can skip to the next stage (deploy) if they do not want to build the application.
All pre-built binaries are provided.

Prerequisites

This section expects the user to have completed Step 5 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

  • AI SDK setup is done.

  • Following docker container is running on the host machine.

    Board Docker container
    RZ/V2L EVK rzv2l_ai_sdk_container
    RZ/V2H EVK rzv2h_ai_sdk_container

    Note: Docker environment is required for building the sample application.

Application File Generation

  1. On your host machine, copy the repository from the GitHub to the desired location.

    1. It is recommended to copy/clone the repository on the data folder, which is mounted on the Docker container.
    cd <path_to_data_folder_on_host>/data
git clone https://github.com/renesas-rz/rzv_ai_sdk.git

    Note: This command will download the whole repository, which include all other applications.
    If you have already downloaded the repository of the same version, you may not need to run this command.

  2. Run (or start) the docker container and open the bash terminal on the container.
    E.g., for RZ/V2L, use the rzv2l_ai_sdk_container as the name of container created from rzv2l_ai_sdk_image docker image.

    Note that all the build steps/commands listed below are executed on the docker container bash terminal.

  3. Set your clone directory to the environment variable.

    export PROJECT_PATH=/drp-ai_tvm/data/rzv_ai_sdk

  4. Go to the application source code directory.

    cd ${PROJECT_PATH}/Q11_fish_detection/src

  5. Create and move to the build directory.

    mkdir -p build && cd build

  6. Build the application by following the commands below.
    For RZ/V2L

    cmake -DCMAKE_TOOLCHAIN_FILE=./toolchain/runtime.cmake ..
make -j$(nproc)

    For RZ/V2H

    cmake -DCMAKE_TOOLCHAIN_FILE=./toolchain/runtime.cmake -DV2H=ON ..
make -j$(nproc)

  7. The following application file would be generated in the ${PROJECT_PATH}/Q11_fish_detection/src/build directory

    • fish_detector

Application: Deploy Stage

Prerequisites

This section expects the user to have completed Step 7-1 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

  • microSD card setup is done.

File Configuration

For the ease of deployment all the deployable files and folders are provided in following folders.

Board EXE_DIR
RZ/V2L EVK exe_v2l
RZ/V2H EVK exe_v2h

Each folder contains following items.

File Details
fish_detection_model Model object files for deployment.
fish_class.txt Label list for Fish Detection.
config.ini Configuration file for the application.
fish_detector application file.

Instruction

  1. [FOR RZ/V2H only] Run following commands to download the necessary file.

    cd <path_to_data_folder_on_host>/data/rzv_ai_sdk/Q11_fish_detection/exe_v2h/fish_detection_model
wget https://github.com/renesas-rz/rzv_ai_sdk/releases/download/v5.00/Q11_fish_detection_deploy_tvm_v2h-v230.so

  2. [FOR RZ/V2H only] Rename the Q11_fish_detection_deploy_*.so to deploy.so.

    mv Q11_fish_detection_deploy_*.so deploy.so

  3. Copy the following files to the /home/root/tvm directory of the rootfs (SD Card) for the board.

    File Details
    All files in EXE_DIR directory Including deploy.so file.
    fish_detector application file Generated the file according to Application File Generation

  4. Check if libtvm_runtime.so exists under /usr/lib64 directory of the rootfs (SD card) on the board.

  5. Folder structure in the rootfs (SD Card) would look like:

    |-- usr
|   `-- lib64
|       `-- libtvm_runtime.so
`-- home
    `-- root
        `-- tvm
            |-- fish_detection_model
            |   |-- deploy.json
            |   |-- deploy.params
            |   `-- deploy.so
            |-- config.ini
            |-- fish_class.txt
            `-- fish_detector

Note: The directory name could be anything instead of tvm. If you copy the whole EXE_DIR folder on the board, you are not required to rename it tvm.

Application: Run Stage

Prerequisites

This section expects the user to have completed Step 7-3 of Getting Started Guide provided by Renesas.

After completion of the guide, the user is expected of following things.

  • The board setup is done.
  • The board is booted with microSD card, which contains the application file.

Instruction

  1. On Board terminal, go to the tvm directory of the rootfs.

    cd /home/root/tvm

  2. Change the values in config.ini as per the requirements. Detailed explanation of the config.ini file is given at below section.

    vi config.ini

  3. Run the application.

    • For USB Camera Mode
    ./fish_detector USB
    • For MIPI Camera Mode (RZ/V2L only)
    ./fish_detector MIPI

    Note: MIPI Camera Mode is only supported by RZ/V2L EVK.

  4. Following window shows up on HDMI screen.

    RZ/V2L EVK RZ/V2H EVK

    On application window, following information is displayed.

    • Camera capture
    • Object Detection result (Bounding boxes, class name and score.)
    • Processing time
      • Total AI Time: Sum of all processing time below.
      • Inference: Processing time taken for AI inference.
      • PreProcess: Processing time taken for AI pre-processing.
      • PostProcess: Processing time taken for AI post-processing.
        (excluding the time for drawing on HDMI screen).

  5. To terminate the application, switch the application window to the terminal by using Super(windows key)+Tab and press ENTER key on the terminal of the board.

Application: Configuration

AI Model

  • RZ/V2L

    • Tiny YOLOv3: Darknet
      Dataset: Custom labelled dataset with classes listed here
      Input size: 1x3x416x416
      Output1 size: 1x13x13x57
      Output2 size: 1x26x26x57

  • RZ/V2H

    • YOLOv3: Darknet
      Dataset: Custom labelled dataset with classes listed here
      Input size: 1x3x416x416
      Output1 size: 1x13x13x57
      Output2 size: 1x26x26x57
      Output3 size: 1x52x52x57

AI inference time

Board AI model AI inference time
RZ/V2L EVK Tiny YOLOv3 Approximately 58 ms
RZ/V2H EVK YOLOv3 Approximately 26 ms

Processing

Processing Details
Pre-processing Processed by CPU.
Inference Processed by DRP-AI and CPU.
Post-processing Processed by CPU.

Explanation of the config.ini file

  • The config.ini file should contain two sections [path] & [detect].
  • The section [path] should contains two variables - 'model_path' & 'label_path'.
  • The model_path value is the path to the folder containing compiled model. The folder should also contains also contain preprocess folder.
  • The label_path value is the path to the label list the model supports.
  • The [detect] section contains three variables - 'conf', 'anchors' & 'objects'.
  • The conf value is the confidence threshold used for object detection.
  • The anchors are a set of predefined bounding boxes values of a certain height and width. These boxes are defined to capture the scale and aspect ratio of specific object classes you want to detect and are typically chosen based on object sizes in your training datasets.
  • The objects represents class and it can be changed to other classes present on the label list.
  • To modify the configuration settings, edit the values in this file using VI Editor, from the RZ/V2L or RZ/V2H Evaluation Board.

Image buffer size

Board Camera capture buffer size HDMI output buffer size
RZ/V2L EVK VGA (640x480) in YUYV format HD (1280x720) in BGRA format
RZ/V2H EVK VGA (640x480) in YUYV format FHD (1920x1080) in BGRA format

Reference

License

Apache License 2.0
<<<<<<< HEAD For third party OSS library, please see the source code file itself.

For third party OSS library, please see the source code file itself.

origin/fish_detection