Creating Multi-Class Labels for YOLO from RAMP masks

[GvdDool]

Hello,
I am currently investigating the use of Multi-Class training of a foundation model in YOLO and as input for this model converting the RAMP raster masks to YOLO vector format. The code base is on the branch: Feature/MultiMasks:
hot_fair_utilities/preprocessing/multimasks_from_polygons.py

This function gives the RAM masks, but YOLO needs this information in vector, so there is another script:
https://github.com/hotosm/fAIr-utilities/... /hot_fair_utilities/preprocessing/yolo_format.py (branch unknown) which is taking the masks and convert back to vector, but now having 3 classes:

• footprint (the inner buffer)
• boundary ( the original label buffered two sides)
• contact (where two buildings are touching)

The code was developed for binary masks, and should be working well for simple labels, but is not fit for purpose when handling multi-class labels because of the use of the selected function in the contour library. Working with contours is a good solution, and simple labels (binary masks) can be converted in this way, but complex labels (even binary masks) are misrepresented by the selected method.

Selected method:
contours, _ = cv2.findContours(x, mode=cv2.RETR_EXTERNAL, method=cv2.CHAIN_APPROX_TC89_KCOS)

Issue:
mode=cv2.RETR_EXTERNAL

This is not the correct mode for multi-class labels, as the linked medium post outlines:

The EXTERNAL option only captures the outer boundary in the contour, thus when a label is presented with a hole, this hole is ignored in the YOLO label, while for RAMP the raster mask would have this hole (creating a mismatch between the two methods). Using this method for multi-class labels will cause a problem because the boundary class is a doughnut, and by using the outer boundary only of the doughnut it is actually an extended footprint, and not the boundary of the building.

The more correct handling of multi-classes in YOLO is by using the mode=cv2.RETR_CCOMP, by calling this function the hierchal order of the contours are preserved, and doughnuts can be constructed by finding the nearest points of the two rings describing the "boundary" class in the multi-class label.

Links to the used resources in this issue:
Medium post: https://medium.com/ntust-aivc/digital-image-processing-contour-usage-77ab76918358
GitHub repo to work with doughnuts in YOLO: https://github.com/ryouchinsa/donut