Error Analysis Plots & Evaluation 🔥

[fcakyon]

✔️ Install sahi:

pip install -U sahi

✔️ Create COCO formatted prediction results using COCO formatted dataset:

sahi predict --source image_dir/ --dataset_json_path dataset.json --model_type mmdet --model_path weight.pt --model_config_path config.yaml --no_sliced_prediction

✔️ Calculate COCO evaluation metrics using the created result.json:

sahi coco evaluate --dataset_json_path dataset.json --result_json_path result.json

✔️ Create error analysis plots using the created result.json:

sahi coco analyse --dataset_json_path dataset.json --result_json_path result.json

🎯 Meaning of the metrics:
C75: Results at 0.75 IOU threshod
C50: Results at 0.75 IOU threshold
Loc: Results after ignoring localization errors
Sim: Results after ignoring supercategory false positives
Oth: Results after ignoring all category confusions
BG: Results after ignoring all false positives
FN: Results after ignoring all false negatives

📈 Possible model improvements:
C75-C50 and C50-Loc=Potential gain with more accurate bounding box prediction
Loc-Sim=Potential gain after fixing supercategory confusions
Loc-Oth=Potential gain after fixing category confusions
Oth-BG=Potential gain after fixing all false positives
BG-FN=Potential gain after fixing all false negatives

[bigrobinson]

Is there a workflow for using the analysis tools with yolov8?

[fcakyon]

It should work with yolov8 and yolo11, if you have any problems please post the full error traceback 👍🏻