Evidently: Simulate Evidence Accumulation Models in Python

Evidently is a python package for working with evidence accumulation models.

It provides

• Efficient functions for simulating data from a range of models.
• Classes that make it easier to tweak model parameters and manage simulated data.
• A consistent way to implement new models.
• Visualisation, including interactive widgets for Jupyter.
• Kernel density-based methods for estimating the likelihood of real data under a given model/set of parameters, allowing parameter estimation and model comparision.

To see some of the features of Evidently in action, click the link below to launch a notebook packed full of interactive visualisations.

Installation

Evidently isn't on PyPI yet, but you can install it directly from GitHub:

pip install git+https://github.com/EoinTravers/Evidently

Basic Use

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import evidently

Set up a model and provide parameters

model = evidently.models.Diffusion(pars=[1., .5, -.25, .8, .4], max_time=5., dt=.001)
model

Classic Drift Diffusion Model
Parameters: [t0 = 1.00, v = 0.50, z = -0.25, a = 0.80, c = 0.40]

model.describe_parameters()

Parameters for Classic Drift Diffusion Model:
- t0   : 1.00  ~ Non-decision time
- v    : 0.50  ~ Drift rate
- z    : -0.25 ~ Starting point
- a    : 0.80  ~ Threshold (±)
- c    : 0.40  ~ Noise SD

Simulate data

X, responses, rts = model.do_dataset(n=1000)

X.head()

	0.000	0.001	0.002	0.003	0.004	0.005	0.006	0.007	0.008	0.009	...	4.990	4.991	4.992	4.993	4.994	4.995	4.996	4.997	4.998	4.999
sim
0	-0.207292	-0.195330	-0.189456	-0.207362	-0.203131	-0.209652	-0.201883	-0.216559	-0.224473	-0.211417	...	2.886206	2.886012	2.871632	2.870360	2.846827	2.854295	2.854028	2.899442	2.906381	2.917107
1	-0.201057	-0.194829	-0.190369	-0.204401	-0.211715	-0.223547	-0.223436	-0.229830	-0.228947	-0.200351	...	0.563363	0.546944	0.535339	0.526217	0.523513	0.525713	0.529461	0.544851	0.539877	0.538094
2	-0.199180	-0.192633	-0.185509	-0.168731	-0.174054	-0.173817	-0.182057	-0.181962	-0.192889	-0.157841	...	2.501043	2.500121	2.487202	2.492311	2.480312	2.482806	2.489521	2.481254	2.479215	2.464364
3	-0.207049	-0.230524	-0.223886	-0.228257	-0.221087	-0.224099	-0.236907	-0.238688	-0.231143	-0.248498	...	2.188082	2.181968	2.201805	2.205892	2.204378	2.222228	2.250239	2.251298	2.273741	2.270209
4	-0.218699	-0.202019	-0.205839	-0.192008	-0.179896	-0.181002	-0.198876	-0.190790	-0.175592	-0.190891	...	3.363507	3.372508	3.387018	3.409127	3.388889	3.363482	3.363401	3.364192	3.370832	3.392171

5 rows × 5000 columns

print(responses[:5]) 
print(rts[:5])

[1. 1. 1. 1. 1.]
[1.622 3.958 1.902 3.537 1.974]

Visualise

The evidently.viz submodule contains a collection of matplotlib-based functions for visualising model simulations. Here are a few examples.

ax = evidently.viz.setup_ddm_plot(model) # Uses model info to draw bounds.
evidently.viz.plot_trace_mean(model, X, ax=ax); # Plots simulations

ax = evidently.viz.setup_ddm_plot(model)
evidently.viz.plot_traces(model, X, responses, rts, ax=ax, 
                          terminate=True, show_mean=True); # Show raw data

/home/eoin/miniconda3/lib/python3.7/site-packages/evidently/viz.py:162: RuntimeWarning: invalid value encountered in greater
  X.iloc[i, t > rt] = np.nan

ax = evidently.viz.setup_ddm_plot(model)
for resp in [1, -1]:
    mask = (responses == resp) # Split by response
    evidently.viz.plot_trace_mean(model, X[mask], ax=ax, label='Response: %i' % resp)
plt.legend();

mX = evidently.utils.lock_to_movement(X, rts, duration=2) # Time-lock to threshold crossing
ax = evidently.viz.setup_ddm_plot(model, time_range=(-2, 0))
evidently.viz.plot_traces(model, mX, responses, rts, ax=ax, show_mean=True);

ax = evidently.viz.setup_ddm_plot(model, time_range=(-2, 0))
for resp in [1, -1]:
    mask = responses == resp
    resp_mX = evidently.utils.lock_to_movement(X[mask], rts[mask])
    evidently.viz.plot_trace_mean(model, resp_mX, ax=ax, label='Response: %i' % resp)
plt.legend();

There high-level functions can create multi-axis figures.

evidently.viz.visualise_model(model, model_type='ddm', measure='means');

Interactive Visualisation

Using the ipywidgets package, we can wrap high level visualisation functions like accum.viz.visualise_ddm in a call to ipywidgets to make them interactive.

To try the interactive plots, download this repository to your own computer, or run the code in the cloud by visiting this Binder notebook.

from ipywidgets import interact, FloatSlider
def fs(v, low, high, step, desc=''):
    return FloatSlider(value=v, min=low, max=high, step=step, description=desc, continuous_update=False)

def ddm_simulation_plot(t0=1., v=.5, z=0., a=.5, c=.1):
    model = evidently.Diffusion(pars=[t0, v, z, a, c])
    evidently.viz.visualise_model(model)
    title = 't0 = %.1f, Drift = %.1f, Bias = %.1f, Threshold = %.1f; Noise SD = %.1f' % (t0, v, z, a, c)
    plt.suptitle(title, y=1.01)

interact(ddm_simulation_plot,
         t0  = fs(1., 0, 2., .1,   't0'),
         v   = fs(.5, 0, 2., .1,   'Drift'),
         z   = fs(0., -1., 1., .1,  'Bias'),
         a     = fs(.5, 0., 2., .1,   'Threshold'),
         c   = fs(.1, 0., 1., .1,   'Noise SD'));

Other Models

The following model classes are currently available:

• Diffusion
• Wald
• HDiffision (Hierarchical Diffusion)
• HWald (Hierarchical Wald)
• Race

See the API for more details.

Road Map

More Models!

I have already implemented several of these models, but have to integrate them with the rest of the package.

• Leaky Competing Accumulator model.
• LCA/Race models with > 2 options.
• Leaky/unstable Diffusion.
• Time-varying parameters, including
  • Collapsing decision bounds
  • Time-varying evidence
• Hierarchical models with regressors that differ across trials.

Reparameterisation

Ideally, parameterisation with other packages used for fitting accumulator models such as HDDM and PyDDM, (for Python) and rtdists and DMC (for R). This would make it possible to efficiently fit models using those packages, then explore their dynamics here.

Model probably should also specify default parameters.

Visualisation

There's no shortage of ways to visualise accumulator models. Future versions will include both more low-level plotting functions and high-level wrappers.

I'll also be implementing vector field plots, e.g. Figure 2 of Bogacz et al. (2007).

Likelihood

The evidently.likelihood model contains functions for estimating the likelihood of data $x$ under parameters $\theta$ and model $M$, based on the "likelihood-free" technique introduced by Turner and Sederberg (2007). These functions aren't properly tested yet, and haven't been documented.

Support

Development of Evidently was in part supported by a research project grant from The Leverhulme Trust (RPG-2016-378)