The larval zebrafish tracking program for "All-optical interrogation of brain-wide activity in freely swimming larval zebrafish"
This tracking system for larval zebrafish is a component of the whole system for whole brain imaging of neural activity in freely swimming larval zebrafish. It is built up in C++. The tracking program will detect the fish and control the motion stage to compensate the motion of fish. With U-net based detection model and model predictive control (MPC), our tracking system is highly reliable and can be used in a variety of behavioral experiments. The main program is main.cpp. Some codes for estimation of parameters are in setting-parameters. Some codes for labelling data to train the U-Net are in labelling-data-for-DL.
Detection of the fish
We use a simple U-Net to detect fish head and yolk in real time. The model contains only two downsampling layers and two upsampling layers, which improves the detection speed. The heat map is used as the output of the model, showing the location of the target point and the confidence level. The codes for training and the trained network are in U-net.
Motion stage control
We adopted the model predictive control (MPC) method in (1) to control the X-Y motorized stage. We modeled the motion of the stage and the fsh, and then selected the optimal stage input by minimizing future tracking error. The main program of this part is MPC_main.cpp which will call MPC.cpp.
- Kim, D. H., Kim, J., Marques, J. C., Grama, A., Hildebrand, D. G., Gu, W., Li, J. M., and Robson, D. N. Pan-neuronal calcium imaging with cellular resolution in freely swimming zebrafsh. Nature methods, 14(11):1107–1114, 2017.
Motorized stage
Zolix SK25A-65SRTracking camera
A high-speed camera (0.8 ms exposure time, 340 fps, Basler aca2000-340kmNIR, Germany) to capture the lateral motion of the fish.Frame grabber
BitFlow AXN-PC2-CL-1xEGPU
NVIDIA GeForce RTX 3080Libraries
- TensorRT 7.2.2.3
- CUDA 11.1
- CUDNN 8.2.0
- OpenCV 4.55
- FLTK 1.4.0
- Bitflow SDK 6.5