GitHub - mdschramm/Machine-Translation

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 16 Commits
es-en		es-en
.translater.py.swp		.translater.py.swp
CustomLanguageModel.py		CustomLanguageModel.py
CustomLanguageModel.pyc		CustomLanguageModel.pyc
Datum.py		Datum.py
Datum.pyc		Datum.pyc
HolbrookCorpus.py		HolbrookCorpus.py
HolbrookCorpus.pyc		HolbrookCorpus.pyc
LaplaceBigramLanguageModel.py		LaplaceBigramLanguageModel.py
LaplaceBigramLanguageModel.pyc		LaplaceBigramLanguageModel.pyc
Output.txt		Output.txt
Output0.txt		Output0.txt
OutputDev.txt		OutputDev.txt
OutputDev0.txt		OutputDev0.txt
OutputTest.txt		OutputTest.txt
README		README
Sentence.py		Sentence.py
Sentence.pyc		Sentence.pyc
bleu_score.py		bleu_score.py
holbrook-tagged-train.dat		holbrook-tagged-train.dat
translater.py		translater.py

Repository files navigation

This is the data/script package for Homework 6 of CS 124 at Stanford.

This data package contains bitext data for two language pairs, French-English (fr-en) and Spanish-English (es-en). Training data are selected from the Europarl-v7 corpus, and dev/test data are from its corresponding development set.

All data have been tokenized such that each line is a sentence, and all tokens are separated by spaces. Special characters such as apostrophes are escaped in a form similar to HTML special characters (&apos;). Lines with the same line number from the beginning of the files of corresponding languages are source/target sentence pairs (i.e. source sentence and its translation).
baseline test:
BLEU-1 score: 43.107084
BLEU-2 score: 6.324437
basline dev:
BLEU-1 score: 42.554177
BLEU-2 score: 6.322566