Code to generate adversarial examples for Mozilla DeepSpeech. Began as a modified version of Carlini and Wagner's attack. Has slowly turned into a bit of a behemoth.
This is the build repo. If you want to see the package in action, grab a docker image or install the package using the steps outlined below.
Docker images are available on DockerHub. Each docker image contains the necessary audio examples, transcripts and model checkpoints etc. to get up and running with minimal fussing about.
To start running some experiments with docker:
- Install the latest version of docker (at least version
19.03
). - Install and configure the NVIDIA container runtime.
- Run the container (the image itself will be pulled automatically):
docker run \
-it \
--rm \
--name cleverspeech \
--gpus all \
dijksterhuis/cleverspeech:latest
- Run one of the scripts from
./cleverspeech/scripts
python3 ./cleverspeech/scripts/ctc_attacks.py \
--max_examples 1 \ # number of adversarial examples to generate
--attack_graph cgd \ # clipped gradient descent
--loss ctc # tensorflow provides two ctc loss implementations
If you want to run the container as your user and group ID you'll need to some extra arguments so that the container can change the deepspeech checkpoint, language model, trie etc. file permissions:
# You will need to wait a few minutes for file permissions to propagate
docker run \
-it \
--rm \
--name cleverspeech \
--gpus all \
-e LOCAL_UID=$(id -u ${USER}) \
-e LOCAL_GID=$(id -g ${USER}) \
-v path/to/output/dir:/home/cleverspeech/cleverSpeech/adv:rw \
dijksterhuis/cleverspeech:latest
Note: Using --user
with docker run
will not work as the container must start as root then
switch users after start up (otherwise you can't chown
the model checkpoints/scorer files).
Run:
git clone --recurse-submodules --shallow-submodules \
https://github.com/dijksterhuis/cleverSpeech.git \
&& cd ./cleverSpeech/ \
&& ./bin/downloads.sh \
&& python3 -m pip install -e .
Then run an experiment as before
and have fun fixing undocumented/unexpected dependency errors
(there shouldn't be any on ubuntu >= 18.04
).