Evaluation, Reproducibility, Benchmarks Meeting 31

Minutes of Meeting 31

Date: 22nd January, 2025

Most members interested in working with both task forces
Most members are able to contribute between 1-10 hours per month
There is a good mix of types of involvement that each member is willing to contribute

Need to think about how we phrase the contribution of the work
- Not ready to provide guidelines at this point
- More about raising awareness
What should we target?
- MICCAI is probably the best venue -- could initiate discussions in a way a journal article can't
Possible future directions
- Looking at more metrics (e.g., Hausdorff)
- Looking beyond segmentation (classification, regression, time to event)
- More feedback from statistical experts in our field -- would be important for consensus/recommendations
If anyone has suggestions for statisticians who might be interested in getting involved, please send them to Olivier
Python vs R?
- Ideally Python, will make it easier to integrate into MONAI
- In some cases, R's implementation is more correct. Ideally pull requests to Scipy could be made, but this may take too long

We are now more than halfway through reconciling disagreement between raters
- Already there are some interesting findings regarding reporting and license usage
How can we identify someone with a legal background to provide an expert perspective?
- Perhaps someone from Creative Commons would be willing to weigh in?

Copyright (c) MONAI Consortium