[caprieval] comparisons against multiple references
#1136
Conversation
VGPReys
added
enhancement
| [topoaa] | ||
|
|
||
| [caprieval] | ||
| # Comparison agains first reference | ||
| reference_fname = "data/ensemble_4G6M.pdb" |
There was a problem hiding this comment.
I find this example confusing for users: why using the same input ensemble as a reference in caprieval? the fact that we support this does not mean we should encourage this..this could become an integration test!
There was a problem hiding this comment.
Yes, indeed !
At least we see that we obtain 1 in dockq and 0 in rmsd.
And clearly highlighting the behavior of this approach.
But I think that indeed this is a very bad example that we do not want to showcase.
I will find an other reference file !
There was a problem hiding this comment.
you can also remove the example and we only have this as integration test..depends on how much we want to advertise this feature
| multiple_references: | ||
| default: false | ||
| type: boolean | ||
| title: Allow the use of multiple reference conformations. | ||
| short: Performs caprieval computation on multiple reference conformations. | ||
| long: Performs caprieval computation on multiple reference conformations. | ||
| Final values will contain the best performing metrics with respect to | ||
| all reference conformations. Usefull when evaluating docking poses against | ||
| NMR structures, for example. | ||
| group: analysis | ||
| explevel: expert |
There was a problem hiding this comment.
why this parameter? isn't it easier to evaluate the quality of whatever is provided as reference_fname in caprieval? maybe limiting the number of reference structures to 10 or 20
There was a problem hiding this comment.
To make sure the user is willing to compare against multiple references.
Could be removed and always applied.
There was a problem hiding this comment.
I would remove it..I think the logic gets too complicated otherwise
Checklist
Summary of the Pull Request
This PR in made to handle comparisons with multiple references.
This is performed by turning on the parameter
multiple_references = true(default false).In this scenario, reported values in
capri_ss.tsvare the ones obtained against the best reference.If
multiple_references = false(default), the first reference will be used as reference.I had to add the
__eq__and__lt__methods to the CAPRI class, so comparisons could be performed by functions/methods such assorted([CAPRI, CAPRI])or[CAPRI, CAPRI].sort().Related Issue
#1133