Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix F1 score computation #37

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

ClementCaporal
Copy link

@ClementCaporal ClementCaporal commented Jun 19, 2023

Partially? close #27

Hello,

I hope you are going well.

This PR proposes to update :

  • The distance matrix label plot to make it more self-explanatory
  • The compute_true_positives
    • change cle.maximum_y_projection to cle.sum_y_projection to correctly count ambiguous_matches
    • update to the way false positives are detected to match the new distance_matrix shape
    • Following https://arxiv.org/pdf/2206.01653.pdf recommandation (Fig 14), add ambiguous match to false positive in the F1 score
  • Update the discussion about how to choose the threshold and its figure.
    image
    --> I think the previous graph (in red) had errors as there are 15 annotated cells, and all detected center point should be ambiguous with a big enough threshold?

Thank you,

Clément

@ClementCaporal
Copy link
Author

ClementCaporal commented Jun 19, 2023

If it's useful, I can also add to the PR :

  1. manage image anisotropy similar to : https://haesleinhuepf.github.io/BioImageAnalysisNotebooks/19_spatial_transforms/scaling.html (plus it seems to me that the anisotropy is already relatively corrected in the recorded image)

  2. Show the evolution of the F1 score depending on the threshold

  3. propose a function that handles ambiguous cases using the greedy assignment or Hungarian assignment algorithm shown in figure DG9.1 (https://arxiv.org/abs/2206.01653) that are, if I understood correctly, preferred to validate center point detection?

  4. replace the term "spot" with "center point" to match the naming from: https://arxiv.org/abs/2206.01653

What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Questions on page /29_algorithm_validation/validate-spot-counting.html
1 participant