Metadata correction for 2024.findings-emnlp.198

[shramay-palta]

JSON data block

{
  "anthology_id": "2024.findings-emnlp.198",
  "abstract": "Questions involving commonsense reasoning about everyday situations often admit many possible or plausible answers. In contrast, multiple-choice question (MCQ) benchmarks for commonsense reasoning require a hard selection of a single correct answer, which, in principle, should represent the most plausible answer choice. On 250 MCQ items sampled from two commonsense reasoning benchmarks, we collect 5,000 independent plausibility judgments on answer choices. We find that for over 20% of the sampled MCQS, the answer choice rated most plausible does not match the benchmark gold answers; upon manual inspection, we confirm that this subset exhibits higher rates of problems like ambiguity or semantic mismatch between question and answer choices. Experiments with LLMs reveal low accuracy\nand high variation in performance on the subset, suggesting our plausibility criterion may be helpful in identifying more reliable benchmark items for commonsense evaluation.",
  "authors": [
    {
      "first": "Shramay",
      "last": "Palta",
      "id": "shramay-palta",
      "affiliation": "University of Maryland"
    },
    {
      "first": "Nishant",
      "last": "Balepur",
      "id": "nishant-balepur",
      "affiliation": "University of Maryland"
    },
    {
      "first": "Peter",
      "last": "Rankel",
      "id": "peter-a-rankel",
      "affiliation": "University of Maryland"
    },
    {
      "first": "Sarah",
      "last": "Wiegreffe",
      "id": "sarah-wiegreffe",
      "affiliation": "Allen Institute for AI"
    },
    {
      "first": "Marine",
      "last": "Carpuat",
      "id": "marine-carpuat",
      "affiliation": "University of Maryland"
    },
    {
      "first": "Rachel",
      "last": "Rudinger",
      "id": "rachel-rudinger",
      "affiliation": "University of Maryland"
    }
  ]
}

[github-actions]

Found ACL Anthology entry: https://aclanthology.org/2024.findings-emnlp.198

[nschneid]

abstract, author affiliations. I changed "Peter A." to "Peter" to match the PDF.

[shramay-palta]

Has this been resolved? I still do not see the abstract on the aclanthology page for the paper here: https://aclanthology.org/2024.findings-emnlp.198/

[nschneid]

These approved fixes will be processed on a weekly basis. This issue will be closed at that time.

[mbollmann]

@nschneid The issue is two weeks old though. It seems we last processed on Jan 9. I guess @mjpost is still triggering them manually for now since this is a new process.

[mjpost]

Yes, sorry, I failed to set a reminder for the new schedule. See #4454.