run the preprocess.py ， ValueError: invalid literal for int() with base 10: ','

[zhailiang041219]

Thank you for sharing this wonderful tutorial. I downloaded the glove.42B.300d file and run the preprocess.py code，but occurd an error: ValueError: invalid literal for int() with base 10: ',' have you ever come across this error? how can i slove it ？I am looking forward for your reply. Thank you .

[AlexYangLi]

Hi, @zhailiang041219 . Did this error occur when you ran process_raw.py? Cause i only use int() in process_raw.py, not in preprocess.py. But it works fine for me.

[zhailiang041219]

Thanks for your reply . I find that the glove.42B.300d file download from the website cannot be used directly. I have solved the problem . thank you.

[AlexYangLi]

Hi, @zhailiang041219 . I might know what cause the error. I use gensim.models.KeyedVectors to load glove embeddings, it requires to provide vocab_size and embedding_dim at the first line of the embedding file. Howerver the glove.42B.300d file dosen't contain such information, use gensim.models.KeyedVectors to load it directly will cause eror.

So I explictly add this information to the head of glove file after I download it from the website. You can see the command in process.sh. But, of course you can load the embedding file in a way whatever you like :)

[zhailiang041219]

yeah, I've seen process.sh and solved the problem with the method you provided.Thank you very much.

…

------------------ 原始邮件 ------------------ 发件人: "Li Yang"<[email protected]>; 发送时间: 2019年5月5日(星期天) 下午3:15 收件人: "AlexYangLi/ABSA_Keras"<[email protected]>; 抄送: "*亮"<[email protected]>; "Mention"<[email protected]>; 主题: Re: [AlexYangLi/ABSA_Keras] run the preprocess.py ， ValueError: invalid literal for int() with base 10: ',' (#2) Hi, @zhailiang041219 . I might know what cause the error. I use gensim.models.KeyedVectors to load glove embeddings, it requires to provide vocab_size and embedding_dim at the first line of the embedding file. Howerver the glove.42B.300d file dosen't contain such information, use gensim.models.KeyedVectors to load it directly will cause eror. So I explictly add this information to the head of glove file after I download it from the website. You can see the command in process.sh. But, of course you can load the embedding file in a way whatever you like :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.