Causal RL

Repo for my master thesis on the intersection between causality and reinforcement learning

FIRST

pip install -r requirements.txt

Training

The entry point for the code is train.py at the root of the repository. There are scripts in scripts/ that call the file using the correct arguments. These are:

--intervention_value 5 # value to use when intervening on the enviroment
--n_iters 10000 \ # number of training iterations
--use_random True \ # whether to use a random or active intervention policy
--log_iters 1000 \ # frequency of logging statistics
--lr 0.001 \ # Learning rate for the Predictor
--noise_dist bernoulli 0.5 \ # distribution on the noise variables: options ('gaussian', sigma) and ('bernoulli', p)
--reg_lambda 0 \ # regularization coefficient for causal weights
--dag_name random \ # name of model (can be used to import pre-defined models found in `causal_rl.environments.causal_models`)
--random_dag 4 0.6 \ # parameters for random dag: (dimension, sparsity)
--seed 0 \ # seed for reproducibility
--random_weights True \ # initialize model with random weights or keep pre-defined weights
--output_dir None # if specified, location to save config file and statistics

By default, statistics will be saved in experiments/inbox/<uuid>

Plotting

The stats.pkl file that is generated by training can be used to plot the data by importing it into notebooks/plotting.ipynb