Probabilistic Inference for Learning Control (PILCO)

A modern & clean implementation of the PILCO Algorithm in TensorFlow.

Unlike PILCO's original implementation which was written as a self-contained package of MATLAB, this repository aims to provide a clean implementation by heavy use of modern machine learning libraries.

In particular, we use TensorFlow to avoid the need for hardcoded gradients and scale to GPU architectures. Moreover, we use GPflow for Gaussian Process Regression.

The core functionality is tested against the original MATLAB implementation.

Example of usage

Before using, or installing, PILCO, you need to have Tensorflow 1.13.1 installed (either the gpu or the cpu version). It is recommended to install everything in a fresh conda environment with python>=3.7. Given Tensorflow, PILCO can be installed as follows

git clone https://github.com/nrontsis/PILCO && cd PILCO
python setup.py develop

The examples included in this repo use OpenAI gym 0.15.3 and mujoco-py 2.0.2.7. Once these dependencies are installed, you can run one of the examples as follows

python examples/inverted_pendulum.py

While running an example, Tensorflow might print a lot of warnings, some of which are deprecated. If necessary, you can suppress them by running

tf.logging.set_verbosity(tf.logging.ERROR)

right after including TensorFlow in Python.

Credits:

The following people have been involved in the development of this package:

• Nikitas Rontsis
• Kyriakos Polymenakos

References

See the following publications for a description of the algorithm: 1, 2, 3