Skip to content

Quantized Neural Networks - networks trained for inference at arbitrary low precision.

License

Notifications You must be signed in to change notification settings

razor1179/QuantizedNeuralNetworks-Keras-Tensorflow

This branch is up to date with BertMoons/QuantizedNeuralNetworks-Keras-Tensorflow:master.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

2eb9f8f · Nov 28, 2017

History

25 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Training Quantized Neural Networks

Introduction

Train your own Quantized Neural Networks (QNN) - networks trained with quantized weights and activations - in Keras / Tensorflow. If you use this code, please cite "B.Moons et al. "Minimum Energy Quantized Neural Networks", Asilomar Conference on Signals, Systems and Computers, 2017". Take a look at our presentation or at the paper on arxiv.

This code is based on a lasagne/theano and a Keras/Tensorflow version of BinaryNet.

Preliminaries

Running this code requires:

  1. Tensorflow
  2. Keras 2.0
  3. pylearn2 + the correct PYLEARN2_DATA_PATH in ./personal_config/shell_source.sh
  4. A GPU with recent versions of CUDA and CUDNN
  5. Correct paths in ./personal_config/shell_source.sh

Make sure your backend='tensorflow' and image_data_format='channels_last' in the ~/.keras/keras.json file.

Training your own QNN

This repo includes toy examples for CIFAR-10 and MNIST. Training can be done by running the following:

./train.sh <config_file> -o <override_parameters>

-o overrides parameters in the <config_file>.

The following parameters are crucial:

Examples

  • This is how to train a 4-bit full qnn on CIFAR-10:

    ./train.sh config_CIFAR-10 -o lr=0.01 wbits=4 abits=4 network_type='full-qnn'

  • This is how to train a qnn with 4-bit weights and floating point activations on CIFAR-10:

    ./train.sh config_CIFAR-10 -o lr=0.01 wbits=4 network_type='qnn'

  • This is how to train a BinaryNet on CIFAR-10:

    ./train.sh config_CIFAR-10 -o lr=0.01 network_type='full-bnn'


The included networks have parametrized sizes and are split into three blocks (A-B-C), each with a number of layers (nl) and a number of filters per layer (nf).

  • This is how to train a small 2-bit network on MNIST:

    ./train.sh config_MNIST -o nla=1 nfa=64 nlb=1 nfb=64 nlc=1 nfc=64 wbits=2 abits=2 network_type='full-qnn'

  • This is how to train a large 8-bit network on CIFAR-10:

    ./train.sh config_CIFAR-10 -o nla=3 nfa=256 nlb=3 nfb=256 nlc=3 nfc=256 wbits=8 abits=8 network_type='full-qnn'

About

Quantized Neural Networks - networks trained for inference at arbitrary low precision.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 99.1%
  • Shell 0.9%