Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Literals in the evaluation #15

Open
yhshu opened this issue Aug 4, 2021 · 1 comment
Open

Literals in the evaluation #15

yhshu opened this issue Aug 4, 2021 · 1 comment

Comments

@yhshu
Copy link

yhshu commented Aug 4, 2021

I’m working on a KBQA system on LC-QuAD 1.0. I would like to ask some questions about the evaluation.
Should I include the literal in golden answers when I execute the golden SPARQL against DBpedia 1604? I noticed that all lambda variables in golden SPARQLs are ‘?uri’. Since there’s no FILTER in SPARQL for literals, does it mean answers are just URIs?
Thanks.
@saist1993
Copy link
Member

In my opinion, the best way to evaluate your system would be to execute the gold SPARQL query (the one given in the dataset) and compare it with the answers retrieved by your system.

Purely from the structure/template of the SPARQL, there might be literals as answers. However, we removed all SPARQL and its associated question while creating this dataset, which returned a literal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@saist1993 @yhshu