-
Notifications
You must be signed in to change notification settings - Fork 644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add iou_complement as a configurable alternative distance function to center_distance (for detection and tracking evaluation) #1075
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this feature @sbarkby!
I've done an initial pass on PR (and will look deeper into your code in subsequent passes)
:return: 1 - IOU. | ||
""" | ||
# Do a cheaper first pass before calculating IOU i.e. check if the circles that enclose the two | ||
# boxes overlap |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I understand, shapely
's intersection
method is pretty fast - can all these calculations just to do this check really save much computation time?
If the savings are not significant, I would favor having simpler code here, e.g.:
def bev_iou_complement(...) -> float:
"""..."""
return 1.0 - bev_iou(...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this cheap first pass provides a considerable speedup e.g. putting this in decreases the run time of the nuscenes/eval/tracking/tests/test_evaluate.py script (on my local machine) from 17min18sec to 3min46sec.
Hi @whyekit-motional , thanks for the review, have made the changes you suggested, let me know if you need anything else. Cheers. :) |
Hi @whyekit-motional, just checking in, any updates? |
Hi @whyekit-motional, just checking in, any updates? Does this need to be transferred to a different reviewer? |
This PR introduces the Intersection Over Union complement (1.0 - IOU) as an alternative distance function for determining matches and the calculation of metrics, such as MOTP. The use of this distance function has been made configurable in the detection_cvpr_2019.json and tracking_nips_2019.json config files for detection and tracking evaluation respectively. Fortunately the codebase was already setup to cater for alternative distance functions, so the coding change proposed here is minimal.
Also note that this PR poses no change to the default operation of the evaluator, it will continue to use "center_distance" as it's distance function by default.