translaTHOR

A language translation project

(still under progress)

Techonology used:

• 🤗 Transformers library

• Pytorch

For English to Bengali Traslation

Dataset from: SUPARA-BENCHMARK

Resources:

• https://huggingface.co/docs/transformers/en/model_doc/bert
• https://huggingface.co/transformers/v3.0.2/model_doc/bert.html
• https://huggingface.co/sagorsarker/bangla-bert-base
• https://huggingface.co/Helsinki-NLP/opus-mt-bn-en

Files & Folders:

• English_to_Bengali_1.ipynb is the Notebook for the whole ML process - v1
• English_to_Bengali_0.ipynb is the Notebook for the whole ML process - v0
• english_to_bengali_0.py is the raw python file for the whole ML process - v0
• saved_model is the folder containing saved model for translating English to Bengali - v0
• Translate_EnglishToBangla.ipynb is the Jupyter Notebook for initial working (incomplete)
• translate_englishtobangla.py is the raw python file for initial working (incomplete)

Machine Learning Process:

• Import libraries
• Import datasets and Preprocess data
• Tokenize the data
• Load the model
• Train the model
• Evaluate the model
• Save the model
• Output for User input
• Convert the saved folder into zip