Paper Metadata: 2023.findings-emnlp.512

[elangovana]

Confirm that this is a metadata correction

• I want to file corrections to make the metadata match the PDF file hosted on the ACL Anthology.

Anthology ID

2023.findings-emnlp.512

Type of Paper Metadata Correction

• Paper Title
• Paper Abstract
• Author Name(s)

Correction to Paper Title

Effects of Human Adversarial and Affable Samples on BERT Generalization

Correction to Paper Abstract

BERT-based models have had strong performance on leaderboards, yet have been demonstrably worse in real-world settings requiring generalization. Limited quantities of training data is considered a key impediment to achieving generalizability in machine learning. In this paper, we examine the impact of training data quality, not quantity, on a model’s generalizability. We consider two characteristics of training data: the portion of human-adversarial (h- adversarial), i.e. sample pairs with seemingly minor differences but different ground-truth labels, and human-affable (h-affable) training samples, i.e. sample pairs with minor differences but the same ground-truth label. We find that for a fixed size of training samples, as a rule of thumb, having 10-30% h-adversarial instances improves the precision, and therefore F1, by up to 20 points in the tasks of text classification and relation extraction. Increasing h-adversarials beyond this range can result in performance plateaus or even degradation. In contrast, h-affables may not contribute to a model’s generalizability and may even degrade generalization performance

Correction to Author Name(s)

No response