Machine Translation with Attention Mechanism (From Spanish to English translation)

The research project was associated with "[INF-DSAM1B] Advanced Machine Learning B: Deep Learning for NLP", Summer Semester 2021, for my Masters of Science: Data Science, University of Potsdam, Germany.

You can find the Technical Report on ResearchGate.

Version:

Pytroch version : 1.7.1+cu101
torchtext version: 0.8.0
spacy version: 3.1.1

To install Pytorch:

Steps

1. Create an environment
   conda create -n pytorch python=3.7

2. Activate the environment
   conda activate pytorch

3. Install jupyter
   conda install -c anaconda jupyter

4. Install the ipykernel
   pip install ipykernel

5. Register your environment
   python -m ipykernel install --user --name pytorch --display-name "pytorch"

6. Install pytorch
   GPU Version: conda install pytorch cudatoolkit -c pytorch
   CPU Version: conda install pytorch cpuonly -c pytorch

Model was trained for 20 Epochs and achieved a BLEU score of 25.37 (with model size of roughly 510 mb).

Different version of pytorch, torchtext and spacy might cause the program to break.

The structure of the files should looks like this:

.
├── Data                    
│   ├──                     # initialy this would be empty.
│   └──                     # initialy this would be empty.
├── config.yml
├── data_utils.py
├── download_data.py
├── mymodel.py
├── README.txt
└── run.py

Make sure to have an empty folder, "Data"

The files can be run from cmd prompt, using the the following cmd lines:

1. cd to "/Machine_translation_with_attention_DL4NLP". Then run the below code line in cmd.
2. To train and validate the model, type:
   python run.py --RUN_MODE train_val
3. To evaluate the loss on test data, type:
   python run.py --RUN_MODE test --MODEL _number_ (_number_ is the model number until which you trained your model)
4. To calculate the BLEU score, type:
   python run.py --RUN_MODE bleu --MODEL _number_ (_number_ is the model number until which you trained your model)