You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems the code cannot handle those Persian hashtags that contain ZWNJ. For example #نرمافزار_آزاد gets converted into افزار_آزاد
I have investigated and found that the issue is related to preprocessor package which doesn't recognize ZWNJ as a valid character in hashtag. I have reported the issue (s/preprocessor#30) and also sent a PR to the project (s/preprocessor#31)
Though solving that problem is good, but it will harm this project as it results in complete removal of hashtag. I suggest to keep hashtags in the word cloud by removing # character manually before passing it into preprocessor.
If you agree, I send PR for it
The text was updated successfully, but these errors were encountered:
It seems the code cannot handle those Persian hashtags that contain ZWNJ. For example #نرمافزار_آزاد gets converted into افزار_آزاد
I have investigated and found that the issue is related to
preprocessor
package which doesn't recognize ZWNJ as a valid character in hashtag. I have reported the issue (s/preprocessor#30) and also sent a PR to the project (s/preprocessor#31)Though solving that problem is good, but it will harm this project as it results in complete removal of hashtag. I suggest to keep hashtags in the word cloud by removing # character manually before passing it into preprocessor.
If you agree, I send PR for it
The text was updated successfully, but these errors were encountered: