Skip to content

Commit

Permalink
Update sharedtasks.html
Browse files Browse the repository at this point in the history
update SciVQA
  • Loading branch information
esborisova authored Jan 30, 2025
1 parent b9038b9 commit b0ad69c
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion 2025/sharedtasks.html
Original file line number Diff line number Diff line change
Expand Up @@ -306,10 +306,13 @@ <h2 id="SciVQA">SciVQA: Scientific Visual Question Answering </h2>

<p>
Scholarly articles convey valuable information not only through unstructured text but also via (semi-)structured figures such as charts and diagrams. Automatically interpreting the semantics of knowledge encoded in these figures can be beneficial for downstream tasks such as question answering (QA).
</p>

<p>
In the SciVQA challenge, participants will develop multimodal QA systems using a dataset of scientific figures from ACL Anthology and arXiv papers. Each figure is annotated with seven QA pairs and includes metadata such as caption, ID, type (e.g., compound, line graph, bar chart, scatter plot), publication title, DOI and URL, QA pair type. This shared task specifically focuses on closed-ended visual (i.e., addressing visual attributes such as colour, shape, size, height, etc.) and non-visual (not addressing figure visual attributes) questions.
Systems will be evaluated using Accuracy, BLEU, METEOR, and ROUGE scores. Automated evaluations of submitted systems will be done through the Codabench platform (link will be provided soon).

</p>
</p>
<h3>Important Dates</h3>

<ul>
Expand Down

0 comments on commit b0ad69c

Please sign in to comment.