Skip to content

Latest commit

 

History

History
70 lines (48 loc) · 2.9 KB

README.md

File metadata and controls

70 lines (48 loc) · 2.9 KB
Raw

Larval-Zebrafish-Tracking

The larval zebrafish tracking program for "All-optical interrogation of brain-wide activity in freely swimming larval zebrafish"

Introduction

This tracking system for larval zebrafish is a component of the whole system for whole brain imaging of neural activity in freely swimming larval zebrafish. It is built up in C++. The tracking program will detect the fish and control the motion stage to compensate the motion of fish. With U-net based detection model and model predictive control (MPC), our tracking system is highly reliable and can be used in a variety of behavioral experiments. The main program is main.cpp. Some codes for estimation of parameters are in setting-parameters. Some codes for labelling data to train the U-Net are in labelling-data-for-DL.

pipeline

Pipeline

Detection of the fish

We use a simple U-Net to detect fish head and yolk in real time. The model contains only two downsampling layers and two upsampling layers, which improves the detection speed. The heat map is used as the output of the model, showing the location of the target point and the confidence level. The codes for training and the trained network are in U-net.

Motion stage control

We adopted the model predictive control (MPC) method in (1) to control the X-Y motorized stage. We modeled the motion of the stage and the fsh, and then selected the optimal stage input by minimizing future tracking error. The main program of this part is MPC_main.cpp which will call MPC.cpp.

  1. Kim, D. H., Kim, J., Marques, J. C., Grama, A., Hildebrand, D. G., Gu, W., Li, J. M., and Robson, D. N. Pan-neuronal calcium imaging with cellular resolution in freely swimming zebrafsh. Nature methods, 14(11):1107–1114, 2017.

Hardware

Motorized stage Zolix SK25A-65SR
Tracking camera A high-speed camera (0.8 ms exposure time, 340 fps, Basler aca2000-340kmNIR, Germany) to capture the lateral motion of the fish.
Frame grabber BitFlow AXN-PC2-CL-1xE
GPU NVIDIA GeForce RTX 3080

Dependency

Libraries
  • TensorRT 7.2.2.3
  • CUDA 11.1
  • CUDNN 8.2.0
  • OpenCV 4.55
  • FLTK 1.4.0
  • Bitflow SDK 6.5

Contributors