You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A user on the TAC mailing list wished to evaluate a mix of 2014 and pre-2014 EL tasks, such that precision should indicate precision of linking/clustering, ignoring spurious mentions.
While it is not hard to remove spurious mentions using grep, this may be worth facilitating in one of the following ways:
a command to output the subset of a dataset that aligns to the gold standard.
an --ignore-spurious flag to evaluate, significance and confidence.
an is_aligned attribute on Annotations that is True for all gold annotations and set with respect to some gold standard when loading annotations from a system output.
(3.) would appear to be most flexible and in line with current design.
The text was updated successfully, but these errors were encountered:
A user on the TAC mailing list wished to evaluate a mix of 2014 and pre-2014 EL tasks, such that precision should indicate precision of linking/clustering, ignoring spurious mentions.
While it is not hard to remove spurious mentions using
grep
, this may be worth facilitating in one of the following ways:--ignore-spurious
flag toevaluate
,significance
andconfidence
.is_aligned
attribute onAnnotation
s that is True for all gold annotations and set with respect to some gold standard when loading annotations from a system output.(3.) would appear to be most flexible and in line with current design.
The text was updated successfully, but these errors were encountered: