Investigating the Robustness of Natural Language Generation from Logical Forms via Counterfactual Samples
This repository contains the data and code for the paper Investigating the Robustness of Natural Language Generation from Logical Forms via Counterfactual Samples.
Chengyuan Liu, Leilei Gan, Kun Kuang, Fei Wu
- Python == 3.7
pip install -r requirements.txt
We also rely on some external resources, you can manually download them and put them into corresponding directories.
- Download pretrained Code-T5 checkpoints, and put it into the
t5_backbone/codet5-base
directory. You can also runt5_backbone/codet5-base/download.sh
. - Download pretrained GPT-2 checkpoints, and put it into the
gpt_backbone/gpt2
directory. You can also rungpt_backbone/gpt2/download.sh
.
prepare general codes
cd t5_backbone
cp -r ../BLEC ./
cp -r ../DataAugment ./
cp -r ../utils ./
cp -r ../multi-bleu.perl ./
prepare data
cp -r ../CD ./
cp -r ../logic2text ./
train from scratch
python main.py --mode=train --edit-strategy=mix
test on logic2text
python main.py --mode=test --load-ckpt=$load_ckpt
where $load_ckpt
is the checkpoint chosen from the training log.
test on LCD
python main.py --mode=test --load-ckpt=$load_ckpt --data-path=CD/data
prepare general codes
cd gpt_backbone
cp -r ../BLEC ./
cp -r ../DataAugment ./
cp -r ../utils ./
cp -r ../multi-bleu.perl ./
prepare data
cp -r ../CD ./
cp -r ../logic2text ./
train from scratch
python main.py --mode=train --edit-strategy=mix
test on logic2text
python main.py --mode=test --load-ckpt=$load_ckpt
where $load_ckpt
is the checkpoint chosen from the training log.
test on LCD
python main.py --mode=test --load-ckpt=$load_ckpt --data-path=CD/data
If you have any issues or questions about this repo, feel free to contact [email protected].
Please cite the following paper if you found our work useful. Thanks!
@inproceedings{liu-etal-2022-investigating,
title = "Investigating the Robustness of Natural Language Generation from Logical Forms via Counterfactual Samples",
author = "Liu, Chengyuan and
Gan, Leilei and
Kuang, Kun and
Wu, Fei",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.emnlp-main.370",
pages = "5499--5512",
}