Minor data error in PAWS-X

[jvamvas]

Thanks for creating this great resource!

I noticed that a small number of samples in the translated dev and tests contain the placeholder "NS". Here's an example from the dev set.

id	label	sentence1	sentence2
684	1	The road continues west through 5th Street as Delphos .	The road continues as Delphos further west through 5th Street .
684	0	Die Straße verläuft weiter nach Westen über die 5. Straße nach Delphos.	NS

Incidentally, the sample counts reported in Table 2 of the PAWS-X paper and on https://github.com/google-research-datasets/paws/tree/master/pawsx#data-format-and-statistics are lower than the actual size of the splits. The difference nearly corresponds to the number of "NS" samples.

So it seems that a cleaning step was planned to be made, but was omitted from the published dataset.

[yuanzh]

Good catch. Thanks for reporting this!