word-segmentation

Usage

1. Make sure you have installed neural transducer.
2. Run predict.py:

# output folder must exist ("." for current folder)
python predict.py --model-folder model --output PATH_TO_OUTPUT_DIR --test PATH_TO_FILE

Segmentation token

Note that we used a different SINGLE segmentation token to decrease the complexity (as opposed to @@ in the orginial shared task data), so check if this token is contained in your test data (if so, change it manually in the loaded vocabulary instance).