Loss Functions for Object Detection: An Overview

This repository provides an up-to-date list of loss functions proposed for solving the object detection problem, which includes both the 2D/3D, axis-aligned/rotated object detection tasks.

=============================

Table of contents

1. Ground Truth Label Assignment
   1.1 One-to-Many Assignment
   1.2 One-to-One Assignment
2. Classification Losses
   2.1 Classical Classification Losses
   2.2 Sampling-based Approaches
   2.3 Re-weighting-based Approaches
   2.4 Gradient-based Approaches
   2.5 Ranking-based Approaches
3. Regression Losses
   3.1 Scale Imbalance Issue
   3.2 Optimization Issue
   3.3 Outlier Problem
4. Loss Function for Oriented Object Detection
   4.1 Boundary Discontinuity Problem
   4.2 Complicated-IoU-Computation-Issue
5. Future Research Directions
   5.1 Unifying Classification and Localisation Tasks
   5.2 Automation Loss Function Searching
   5.3 End-to-end Object Detection
   5.4 Mulit-tasks Learning

---

1. Ground Truth Label Assignment

1.1 One-to-Many Assignment

• Many object detectors are designed with this strategy such as RCNN Series, One-stage, Two-stages Anchor-based Anchor-free, and many 3D detectors;

1.2 One-to-One Assignment

• DETR based object detectors, such as:
  -End-to-End Object Detection with Transformers. [Paper]
  -End-to-end object detection with fully convolutional network. [Paper]
  -Rethinking transformerbased set prediction for object detection. [Paper]
  -Sparse r-cnn: End-to-end object detection with learnable proposals. [Paper]
  -Deformable DETR: Deformable transformers for end-to-end object detection. [Paper]
  -Pnp-detr: towards efficient visual analysis with transformers. [Paper]
  -Conditional detr for fast training convergence. [Paper]

2. Classification Losses

2.1 Classical Classification Losses

• Boosting series，such as AdaBoost etc:
• Support Vector Manchine.

2.2 Sampling-based Approaches

• Offline Sampling-based Approaches.
  -Many object detectors are designed with this strategy such as RCNN Series, One-stage, Two-stages Anchor-based Anchor-free, and many 3D detectors;
  -CBGS: Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection. [Paper]
  -Copy-and-Paste: PointRCNN, CenterPoint, SECOND, PointPillars, etc.
  -Rendering-based Copy and Paste: LiDAR-Aug: A General Rendering-based Augmentation Framework for 3D Object Detection. [Paper]
• Online Sampling-based Approaches.
  -Training Region-based Object Detectors with Online Hard Example Mining. [Paper]
  -S-OHEM: Stratified Online Hard Example Mining for Object Detection. [Paper]
  -Libra R-CNN: Towards Balanced Learning for Object Detection. [Paper]
  -Prime Sample Attention in Object Detection. [Paper]
  -Generating Positive Bounding Boxes for Balanced Training of Object Detectors. [Paper]

2.3 Re-weighting-based Approaches

-Focal Loss for Dense Object Detection. [Paper]
-Focal Loss in 3D Object Detection. [Paper]
-Automated focal loss for image based object detection. [Paper]
-Spatial focal loss for pedestrian detection in fisheye imagery. [Paper]
-Focal text: an accurate text detection with focal loss. [Paper]
-Class-discriminative focal loss for extreme imbalanced multiclass object detection towards autonomous driving. [Paper]
-Dldenet: Deep local directional embeddings with increased foreground focal loss for object detection. [Paper]

2.4 Gradient-based Approaches

-Equalization loss for long-tailed object recognition. [Paper]
-Gradient harmonized single-stage detector. [Paper]
-Equalization loss v2: A new gradient balance approach for long-tailed object detection. [Paper]
-Droploss for longtail instance segmentation. [Paper]
-Distribution-balanced loss for multi-label classification in long-tailed datasets. [Paper]
-Distributional robustness loss for long-tail learning. [Paper]
-Adaptive class suppression loss for long-tail object detection. [Paper]
-Seesaw loss for long-tailed instance segmentation. [Paper]

2.5 Ranking-based Approaches

-Towards accurate one-stage object detection with ap-loss. [Paper]
-Ap-loss for accurate one-stage object detection. [Paper]
-Dr loss: Improving object detection by distributional ranking. [Paper]
-Rank & sort loss for object detection and instance segmentation. [Paper]
-A ranking-based, balanced loss function unifying classification and localisation in object detection. [Paper]
-Rankdetnet: Delving into ranking constraints for object detection. [Paper]
-Tackling class imbalance with ranking. [Paper]
-Combining ranking with traditional methods for ordinal class imbalance. [Paper]

3. Regression Losses

3.1. Scale Imbalance

-A scale balanced loss for bounding box regression. [Paper]
-Scaloss: Side and corner aligned loss for bounding box regression. [Paper]
-Scale-balanced loss for object detection. [Paper]

3.2. Optimization Issue

-Unitbox: An advanced object detection network. [Paper]
-Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression. [Paper]
-Distance-iou loss: Faster and better learning for bounding box regression. [Paper]
-Focal and efficient iou loss for accurate bounding box regression. [Paper]
-Alpha-iou:A family of power intersection over union losses for bounding box regression. [Paper] -IoU Loss for 2D/3D Object Detection. [Paper]

3.3. Outlier Problem

-Libra R-CNN: Towards Balanced Learning for Object Detection. [Paper]
-Iou-balanced loss functions for single-stage object detection. [Paper]
-Improving object localization with fitness nms and bounded iou loss. [Paper]

4. Loss Function for Oriented Object Detection

4.1. Boundary Discontinuity Problem

-Scrdet: Towards more robust detection for small, cluttered and rotated objects. [Paper]
-Bounding box projection for regression uncertainty in oriented object detection. [Paper] -Learning modulated loss for rotated object detection. [Paper] -Oriented object detection in aerial images with box boundary-aware vectors. [Paper]
-3d bounding box estimation using deep learning and geometry. [Paper] -Dense label encoding for boundary discontinuity free rotation detection. [Paper] -On the arbitrary-oriented object detection: Classification based approaches revisited. [Paper]

4.2. Complicated IoU Computation Issue

-IoU Loss for 2D/3D Object Detection. [Paper]
-Piou loss: Towards accurate oriented object detection in complex environments. [Paper]
-Rethinking rotated object detection with gaussian wasserstein distance loss. [Paper]
-Gaussian bounding boxes and probabilistic intersection-over-union for object detection. [Paper]
-The kfiou loss for rotated object detection. [Paper]
-A normalized gaussian wasserstein distance for tiny object detection. [Paper]
-Learning high-precision bounding box for rotated object detection via kullback-leibler divergence. [Paper]

5. Future Research Directions

5.1. Unifying Classification and Localisation Tasks

-Acquisition of localization confidence for accurate object detection. [Paper]
-Ap-loss for accurate one-stage object detection. [Paper]
-A ranking-based, balanced loss function unifying classification and localisation in object detection. [Paper]

5.2. Automation Loss Function Searching

-Amlfs: Automl for loss function search. [Paper]
-Loss function discovery for object detection via convergence-simulation driven search. [Paper]
-Autoloss-zero: Searching loss functions from scratch for generic tasks. [Paper]
-Searching parameterized ap loss for object detection. [Paper]

5.3. End-to-end Object Detection

• DETR based object detectors, such as:
  -End-to-End Object Detection with Transformers. [Paper]
  -End-to-end object detection with fully convolutional network. [Paper]
  -Rethinking transformerbased set prediction for object detection. [Paper]
  -Sparse r-cnn: End-to-end object detection with learnable proposals. [Paper]
  -Deformable DETR: Deformable transformers for end-to-end object detection. [Paper]
  -Pnp-detr: towards efficient visual analysis with transformers. [Paper]
  -Conditional detr for fast training convergence. [Paper]

5.4. Mulit-tasks Learning