Unable to reproduce the result on STS tasks

[saharghannay]

Hi,

I evaluate the InferSent 1 model on STS transfer tasks (STS12-16 and STSBenchmark) using the SentEval tool.
However I could not reproduce the same results.

2019-04-28 17:35:35,511 : ***** Transfer task : STS12 *****
Found 8104(/8122) words with w2v vectors
Vocab size : 8104
2019-04-28 17:35:37,836 : ALL (weighted average) : Pearson = 0.5893, Spearman = 0.5896
2019-04-28 17:35:37,836 : ALL (average) : Pearson = 0.5792, Spearman = 0.5798

2019-04-28 17:35:37,836 : ***** Transfer task : STS13 (-SMT) *****
Found 5106(/5116) words with w2v vectors
Vocab size : 5106
2019-04-28 17:35:38,933 : ALL (weighted average) : Pearson = 0.6399, Spearman = 0.6371
2019-04-28 17:35:38,933 : ALL (average) : Pearson = 0.5553, Spearman = 0.5541

2019-04-28 17:35:38,933 : ***** Transfer task : STS14 *****
Found 9046(/9060) words with w2v vectors
Vocab size : 9046
2019-04-28 17:35:41,391 : ALL (weighted average) : Pearson = 0.6545, Spearman = 0.6377
2019-04-28 17:35:41,391 : ALL (average) : Pearson = 0.6411, Spearman = 0.6258

2019-04-28 17:35:41,391 : ***** Transfer task : STS15 ****
Found 7313(/7335) words with w2v vectors
Vocab size : 7313
2019-04-28 17:35:43,532 : ALL (weighted average) : Pearson = 0.6879, Spearman = 0.6988
2019-04-28 17:35:43,533 : ALL (average) : Pearson = 0.6600, Spearman = 0.6731

2019-04-28 17:35:43,533 : ***** Transfer task : STS16 *****
Found 3930(/3971) words with w2v vectors
Vocab size : 3930
2019-04-28 17:35:44,572 : ALL (weighted average) : Pearson = 0.6287, Spearman = 0.6465
2019-04-28 17:35:44,572 : ALL (average) : Pearson = 0.6321, Spearman = 0.6494

***** Transfer task : STSBenchmark*****
Found 15718(/15755) words with w2v vectors
Vocab size : 15718
2019-04-28 17:36:19,249 : Test : Pearson 0.68976421922 Spearman 0.683016070529

PS: I used /dataset/get_data.bash to download the data,
for word embeddings I was using CBOW trained on wiki data.

Is there something wrong with the default setting?

Thanks for your help!

[saharghannay]

Hi, I want to correct something :
I download the data with get_transfer_data.bash from SenEval.