docs/ch03_models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 3: Models\n",
    "In this chapter, we explain the ideas behind model and agent interfaces and the example model/agent implementations provided by `cognibench`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Capabilities\n",
    "Similar to `sciunit`, `cognibench` test classes require models to have certain capabilities. For example, `InteractiveTest` requires the models to implement the requirements specified by `cognibench.capabilities.Interactive` capability. In addition to test classes, the score class used for testing can also require capabilities. If you want to use `cognibench.scores.AICScore`, then the model should also have `cognibench.capabilities.PredictsLogpdf` and `cognibench.capabilities.ReturnsNumParams` capabilities. We can see these requirements as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(<class 'cognibench.capabilities.Interactive'>,)\n",
      "(<class 'cognibench.capabilities.PredictsLogpdf'>, <class 'cognibench.capabilities.ReturnsNumParams'>)\n"
     ]
    }
   ],
   "source": [
    "from cognibench.testing import InteractiveTest\n",
    "from cognibench.scores import AICScore\n",
    "print(InteractiveTest.required_capabilities)\n",
    "print(AICScore.required_capabilities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Interface\n",
    "In `cognibench` terms, a model represents a way of predicting the actions of some agent and fitting parameters to data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CNBModel Documentation\n",
      "----------------------\n",
      "\n",
      "    Base class for CogniBench models.\n",
      "\n",
      "    In `cognibench`, a model is a way of representing a continuum of an agent and corresponding parameters. Models are\n",
      "    expected to provide fitting and action prediction functionalities, while leaving the tasks of acting on environments\n",
      "    and updating hidden state variables to agents. However, this distinction is not strictly enforced, and users may\n",
      "    choose to use models to represent both of these concepts together.\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "from cognibench.models import CNBModel\n",
    "print('CNBModel Documentation')\n",
    "print('----------------------')\n",
    "print(CNBModel.__doc__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Agent Interface\n",
    "An agent represents a way of interacting with an environment through `act` and `update` methods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CNBAgent Documentation\n",
      "----------------------\n",
      "\n",
      "    Base class for CogniBench agents.\n",
      "\n",
      "    In `cognibench`, an agent is a way of interacting with environments through `act` and `update` methods while possibly\n",
      "    storing some hidden state.\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "from cognibench.models import CNBAgent\n",
    "print('CNBAgent Documentation')\n",
    "print('----------------------')\n",
    "print(CNBAgent.__doc__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example Agents and Models\n",
    "In this section, we go over example model and agent implementations provided by cognibench:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import Rescorla-Wagner Choice Kernel model\n",
    "from cognibench.models.decision_making import RWCKModel, RWCKAgent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we look at the superclasses of `RWCKAgent`, we see that it inherits from"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(cognibench.models.base.CNBAgent,\n",
       " cognibench.capabilities.ProducesPolicy,\n",
       " cognibench.capabilities.DiscreteAction,\n",
       " cognibench.capabilities.DiscreteObservation)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RWCKAgent.__bases__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, the last two define the action and observation space this agent can interact with. This is important for methods that require environment and agent interaction such as `cognibench.simulation.simulate` function. Then, we see that `RWCKAgent` inherits from `ProducesPolicy` capability. This means that `RWCKAgent` provides `eval_policy` method that returns a probability distribution over the action space. Although this is not directly required for interacting with environments, it allows a very straightforward model implementation. In `cognibench`, this functionality is in `cognibench.models.policy_model.PolicyModel` class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "    PolicyModel provides a model implementation that can be created from agents satisfying\n",
      "    :class:`cognibench.capabilities.ProducesPolicy` capability.\n",
      "\n",
      "    If you already have an agent implementation that can provide a probability distribution over the action space\n",
      "    (`eval_policy` method), you can create a model of that agent that uses `eval_policy` to make predictions and to fit\n",
      "    model parameters (using maximum likelihood) by simply deriving from this class. For examples of such usage, refer to\n",
      "    decision making or associative learning model implementations provided by `cognibench`.\n",
      "    \n"
     ]
    }
   ],
   "source": [
    "from cognibench.models.policy_model import PolicyModel\n",
    "print(PolicyModel.__doc__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Any model that subclass from `PolicyModel` automatically gets `fit` and `predict` method implementations by using the `eval_policy` method of the underlying agent. `RWCKModel` is an example of such implementation; it subclasses from `PolicyModel` and uses an `RWCKAgent` for the underlying update equations and probability distribution generation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(cognibench.models.policy_model.PolicyModel,\n",
       " cognibench.capabilities.DiscreteAction,\n",
       " cognibench.capabilities.DiscreteObservation)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RWCKModel.__bases__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementing Your Own Agents or Models\n",
    "You can implement models by taking any of the examples as a starting point. For interactive tests, satisfying `cognibench.capabilities.Interactive` is a requirement. Apart from this, models can continuously satisfy new capabilities in order to take new tests."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:ldmunit]",
   "language": "python",
   "name": "conda-env-ldmunit-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}