From 9e2fbad99f38a6924c8a1caf4083ad58915a59af Mon Sep 17 00:00:00 2001 From: Sa Phyo Thu Htet Date: Tue, 6 Aug 2024 12:52:27 +0630 Subject: [PATCH] Update README.md --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index e89f1db..1cfe769 100644 --- a/README.md +++ b/README.md @@ -7,11 +7,11 @@ 1|Character Break| Used Regular Expression |O(n), O(n)|Can be used for any language. 2|Syllable Tokenization (Unicode)| Regular Expression |O(n), O(n)| Can be used for Unicode data of Myanmar (Burmese), Rakhine, Pali, and Paoh Languages. 3|Syllable Tokenization (Zawgyi)| Regular Expression |O(n), O(n)| Can be used for Zawgyi Encoding Myanmar (Burmese) Language. -4|Multilingual Semi-syllable Tokenization (Unicode)|Regular Expression|O(n), O(n)|Can be used for Unicode Encoding Lao, Kannada, Oriya, Gujarati, Malayalam, Khmer, Bengali, Sinhala, Tamil, Mon, Pali and Sanskrit, Sagaw Karen, Western Poh Karen, Eastern Poh Karen, Geba Karen, Kayah, Rumai Palaung, Khamathi Shan, Aiton and Phake, Burmese (Myanmar), Paoh, Rakhine Languages). Can also be used as a word break for English and charcter break for any other languages.|I got this new idea while working in keywords detection in burmese and other two languages. Regarding keywords detection, the word like "ဘောမ" can be found in the sentence like "သင်္ဘောမျိုး" and the scanerio is irrelevant. And luckily I found an alternative that would be helpful for three languages. Here, semi-syllable does not refer to the minor syllable in phonology. Instead, it is new tokenization that does not break into a full syllable mode. Now I found that it is useful in keyword detection to reduce False Positive errors. (I may explain why keywords detection later)The beauty of this tokenization would be you don't need to know much about the nature of the specific language. It will especially work for a similar script like Brahmic Script. Since it is in the initial state, it may have some errors +4|Multilingual Semi-syllable Tokenization (Unicode)|Regular Expression|O(n), O(n)|Can be used for Unicode Encoding Lao, Kannada, Oriya, Gujarati, Malayalam, Khmer, Bengali, Sinhala, Tamil, Mon, Pali and Sanskrit, Sagaw Karen, Western Poh Karen, Eastern Poh Karen, Geba Karen, Kayah, Rumai Palaung, Khamathi Shan, Aiton and Phake, Burmese (Myanmar), Paoh, Rakhine Languages). It can also be used as a word break for English and a character break for any other language.|I got this new idea while working on keyword detection in Burmese and other two languages. Regarding keywords detection, the words like "ဘောမ" can be found in sentences like "သင်္ဘောမျိုး" and the scenario is irrelevant. And luckily I found an alternative that would be helpful for three languages. Here, semi-syllable does not refer to the minor syllable in phonology. Instead, it is a new tokenization that does not break into a full syllable mode. Now I found that it is useful in keyword detection to reduce False Positive errors. (I may explain why keywords detection later)The beauty of this tokenization would be you don't need to know much about the nature of the specific language. It will especially work for a similar script like Brahmic Script. Since it is in the initial state, it may have some errors 5|Burmese Sentence Level Zawgyi Unicode Detection|Machine Learning|| -6|Burmese to Braille (Muu Haung) Converter|Regular Expression|O(n), O(n)|Can be used to change from burmese to burmese braille (Muu Haung)| The brialle to burmese dictonary may need to be updated. The data for the dicitonary is prepared by Phyo Thu Htet, Naing Linn Phyo and Thiha Nyein. +6|Burmese to Braille (Muu Haung) Converter|Regular Expression|O(n), O(n)|Can be used to change from Burmese to Burmese braille (Muu Haung)| The braille to Burmese dictionary may need to be updated. The data for the dictionary is prepared by Phyo Thu Htet, Naing Linn Phyo, and Thiha Nyein. 7|Keywords Detection|Regular Expression|| -8|Email Detection|Regular Expression|O(n), O(1)|Can be used to detect emails in the text
|E.g. Input: ဒီနေ့တော့ phyothuhtet39@gmail.com ဆီကို mail ပို့ရမယ်။ နေဉီး သူက Microsoft Mail phyothuhtet@studentambassadors.com ကို သုံးတာလားမေးကြည့်ပါဦး။ ငါ ayethida89.young@utycc.edu.mm ကနေ ပို့လိုက်မယ်။, Output: ayethida89.young@utycc.edu.mm;phyothuhtet39@gmail.com;phyothuhtet@studentambassadors.com +8|Email Detection|Regular Expression|O(n), O(n)|Can be used to detect emails in the text
|E.g. Input: ဒီနေ့တော့ phyothuhtet39@gmail.com ဆီကို mail ပို့ရမယ်။ နေဉီး သူက Microsoft Mail phyothuhtet@studentambassadors.com ကို သုံးတာလားမေးကြည့်ပါဦး။ ငါ ayethida89.young@utycc.edu.mm ကနေ ပို့လိုက်မယ်။, Output: ayethida89.young@utycc.edu.mm;phyothuhtet39@gmail.com;phyothuhtet@studentambassadors.com ## Streamlit @@ -25,13 +25,13 @@ $ pip3 install requirements.txt $ streamlit run nlptools.py ``` ## gdg 2022 -Text Classification with Zero Shot and Few Shot Learning pdf: https://github.com/SaPhyoThuHtet/nlp-tool/tree/main/gdg-2022 +Text Classification with Zero-Shot and Few Shot Learning pdf: https://github.com/SaPhyoThuHtet/nlp-tool/tree/main/gdg-2022 -Zero Shot Example Notebook: https://colab.research.google.com/drive/1jocViLorbwWIkTXKwxCOV9HLTaDDgCaw?usp=sharing (This is the original notebook provided by hugging face) +Zero-Shot Example Notebook: https://colab.research.google.com/drive/1jocViLorbwWIkTXKwxCOV9HLTaDDgCaw?usp=sharing (This is the original notebook provided by hugging face) ## Acknowledgment -I would like to thank Dr. Ye Kyaw Thu, Dr. Hnin Aye Thant, Ma Aye Hninn Khine, ​and Ma Yi Yi Chan Myae Win Shein for their guidance, support, and suggestions. The skills acquired from Dr. Ye Kyaw Thu's NLP Class helped me a lot in order to develop new ideas in NLP Field and this repo. And a shoutout to the creators of Rabbit Converter and jrgraphix.net's Unicode Character Table. These tools were super helpful to develop nlp-concepts especially for Burmese Language. Thanks. +I would like to thank Dr. Ye Kyaw Thu, Dr. Hnin Aye Thant, Ma Aye Hninn Khine, ​and Ma Yi Yi Chan Myae Win Shein for their guidance, support, and suggestions. The skills acquired from Dr. Ye Kyaw Thu's NLP Class helped me a lot in order to develop new ideas in the NLP Field and this repo. And a shoutout to the creators of Rabbit Converter and jrgraphix.net's Unicode Character Table. These tools were super helpful in developing concepts, especially for the Burmese Language. Thanks. ## License MIT License