Past and present of small object detection

Small object detection has long been a difficult and hot topic in computer vision. In order to promote the development of this field, I establish this repository to organize the papers related to small object detection. Any latest papers related to small object detection will be updated in this repository.
Chinese version: https://zhuanlan.zhihu.com/p/426047353

Updates

• 2023/8/31 The author has graduated and no longer studies small object detection, so I stop updating now, thank you for your attention. What's more, I will update an article related to small object detection in Zhihu. Good luck!
• 2022/12/14 add one Data-based paper: A full data augmentation pipeline for small object detection based on generative adversarial networks, PR 2023. Despite the lack of novelty, the proposed data augmentation framework is very complete.
• 2022/12/14 add one Special design in detection pipeline paper: RFLA: Gaussian Receptive Field based Label Assignment for Tiny Object Detection, ECCV 2022. Good paper, propose a new label assignment strategy for tiny object. This method is very suitable for detecting extremely small objects (like the object smaller than 8*8 pixels), but it is not suitable for scenes with multi-scale objects.
• 2021/12/24 add one Multi-scale feature learning paper: QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection, arxiv 2021. Good paper, the key contribution is the use of P2 feature for one stage detector without increasing the amount of calculation.
• 2021/12/12 paper: Addressing Scale Imbalance for Small Object Detection with Dense Detector, Neurocomputing 2021. Bad paper, so I decide not to add it to this repo.
• 2021/11/14
  • add one Context-based paper: Realize your surroundings: Exploiting context information for small object detection. Good paper, recommend to read.
  • add one Context-based paper: Intrinsic Relationship Reasoning for Small Object Detection.
• 2021/11/4
  • add one Context-based paper: Structure Inference Net.
  • add one Special design in detection pipeline paper: Dot Distance.
• 2021/11/2
  • create the repository.

Table of Contents

1. Multi-scale feature learning
2. Super resolution
3. Context-based
4. Data-based
5. Training strategy
6. Special design in detection pipeline
7. Loss reweight

1. Multi-scale feature learning

• QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection[Paper][Code]
  • Chenhongyi Yang*, Zehao Huang, Naiyan Wang, arxiv 2021.
• Effective fusion factor in fpn for tiny object detection[Paper][Code]
  • Yuqi Gong, Xuehui Yu, Yao Ding, Xiaoke Peng, Jian Zhao, Zhenjun Han, WACV 2021.
• Augfpn: Improving multi-scale feature learning for object detection[Paper][Code]
  • Chaoxu Guo, Bin Fan, Qian Zhang, Shiming Xiang, and Chunhong Pan, CVPR 2020.
• Path aggregation network for instance segmentation[Paper][Code]
  • Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, Jiaya Jia, YouTuLab Tencent, CVPR 2018.
• Feature Pyramid Networks for Object Detection[Paper]
  • Tsung-Yi Lin, Piotr Dollar, Ross Girshick, Kaiming He, Bharath Harihara, and Serge Belongie, Facebook AI, CVPR 2017.

2. Super resolution

• Better to Follow, Follow to Be Better: Towards Precise Supervision of Feature Super-Resolution for Small Object Detection[Paper]
  • Junhyug Noh, Wonho Bae, Wonhee Lee, Jinhwan Seo, Gunhee Kim, ICCV 2019.
• SOD-MTGAN: Small Object Detection via Multi-Task Generative Adversarial Network[Paper]
  • Yancheng Bai, Yongqiang Zhang, Mingli Ding, and Bernard Ghanem, ECCV 2018.
• Perceptual Generative Adversarial Networks for Small Object Detection[Paper]
  • Jianan Li, Xiaodan Liang, Yunchao Wei, Tingfa Xu, Jiashi Feng, Shuicheng Yan, CVPR 2017.

3. Context-based

• Realize your surroundings: Exploiting context information for small object detection[Paper]
  • Jiaxu Leng, Yihui Ren, Wen Jiang, Xiaoding Sun, Ye Wang, Neurocomputing 2021.
• Intrinsic Relationship Reasoning for Small Object Detection[Paper]
  • Kui Fu, Jia Li, Lin Ma, Kai Mu and Yonghong Tian, Tencent AI Laboratory, arxiv 2020.
• Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships[Paper][Code]
  • Yong Liu, Ruiping Wang, Shiguang Shan, Xilin Chen, VIPL, CVPR 2018.
• Relation Networks for Object Detection[Paper][Code]
  • Han Hu, Jiayuan Gu2, Zheng Zhang, Jifeng Dai, Yichen Wei, Microsoft Research Asia, CVPR 2018.
• PyramidBox: A Context-assisted Single Shot Face Detector[Paper][Code]
  • Xu Tang, Daniel K. Du, Zeqiang He, and Jingtuo Liu, Baidu Inc, ECCV 2018.
• Inside-Outside Net: Detecting objects in context with skip pooling and recurrent neural networks[Paper]
  • Sean Bell, C. Lawrence Zitnick, Kavita Bala, Ross Girshick, Microsoft Research, CVPR 2016.

4. Data-based

• A full data augmentation pipeline for small object detection based on generative adversarial networks[Paper]
  • Brais Bosquet, Daniel Cores, Lorenzo Seidenari, Víctor M. Brea, Manuel Mucientes, Alberto Del Bimbo, PR 2023.
• Stitcher: Feedback-driven Data Provider for Object Detection[Paper][Code]
  • Yukang Chen, Peizhen Zhang, Zeming Li, Yanwei Li, Xiangyu Zhang, Gaofeng Meng, Shiming Xiang, Jian Sun, Jiaya Jia, Megvii Technology, CVPR 2020.
• Augmentation for small object detection.[Paper][Code]
  • Mate Kisantal, Zbigniew Wojna, Jakub Murawski, Jacek Naruniec, Kyunghyun Cho, CVPR 2019.

5. Training strategy

• SNIPER: Efficient Multi-Scale Training[Paper][Code]
  • Bharat Singh, Mahyar Najibi, Larry S. Davis, NIPS 2018.
• An Analysis of Scale Invariance in Object Detection – SNIP[Paper]
  • Bharat Singh, Larry S. Davis, CVPR 2018.

6. Special design in detection pipeline

• RFLA: Gaussian Receptive Field based Label Assignment for Tiny Object Detection.[Paper][Code]
  • Chang Xu, Jinwang Wang, Wen Yang, Huai Yu, Lei Yu, and Gui-Song Xia, ECCV 2022.
• Dot Distance for Tiny Object Detection in Aerial Images[Paper]
  • Chang Xu, Jinwang Wang, Wen Yang, Lei Yu, CVPRW 2021.
• S3FD: Single Shot Scale-invariant Face Detector[Paper][Code]
  • Shifeng Zhang, Xiangyu Zhu, Zhen Lei∗, Hailin Shi, Xiaobo Wang, Stan Z. Li, ICCV 2017.
• FaceBoxes: A CPU Real-time Face Detector with High Accuracy[Paper][Code]
  • Shifeng Zhang, Xiangyu Zhu, Zhen Lei, Hailin Shi, Xiaobo Wang, Stan Z. Li, IJCB 2017.

7. Loss reweight

• Feedback-driven loss function for small object detection[Paper]
  • Gen Liu, Jin Han, Wenzhong Rong, Image Vison Computing 2021.