Demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c63127fb-6496-4ce5-ab89-5f0183bea676",
   "metadata": {
    "tags": []
   },
   "source": [
    "<h1 style=\"font-size: 74px\">SFLKit: <br>A Workbench for Statistical Fault Localization</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "904f5cad-8d7c-41ea-980f-8aa9abc2787d",
   "metadata": {},
   "source": [
    "<div style=\"float: left; font-size:30px\"><b>Marius Smytzek</b><br>marius.smytzek@cispa.de</div><div style=\"float: left;margin-left: 100px; font-size:30px\"><b>Andreas Zeller</b><br>zeller@cispa.de</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5ba28ad-32c3-411d-b30f-6f8309fb8bb8",
   "metadata": {},
   "source": [
    "[start](#Statistical-Fault-Localization)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e2346efa-affd-40cb-be6e-12a49f391a88",
   "metadata": {},
   "source": [
    "from IPython.display import IFrame\n",
    "src = f\"https://www.youtube-nocookie.com/embed/qcnmi6PgrKg\"\n",
    "IFrame(src, 640, 360)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "0b55e972-81c4-4fc0-9da4-416f1c93723f",
   "metadata": {},
   "source": [
    "## What is SFLKit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa52fe9e-abd2-4fc1-9c61-8d05f3c52521",
   "metadata": {},
   "source": [
    "SFLKit is a workbench for statistical fault localization. It comes with the fundamental concepts of statistical debugging and spectrum-based fault localization.\n",
    "\n",
    "You can use SFLKit out-of-the-box by integrating its command-line interface `sfl.py` or as a library, as we do in this demonstration. We designed SFLKit to be highly configurable and expandable with novel concepts.\n",
    "\n",
    "To install SFLKit execute"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b48fd713-7f8f-4c00-aba6-a8052aa5ed36",
   "metadata": {},
   "source": [
    "!pip install ."
   ],
   "outputs": []
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e8333c71-28ed-44ba-8e4e-52a72f8f96e8",
   "metadata": {},
   "source": [
    "import enum\n",
    "import importlib\n",
    "import inspect\n",
    "import os\n",
    "import shutil\n",
    "\n",
    "from IPython.display import HTML\n",
    "\n",
    "from sflkit.color import ColorCode\n",
    "from sflkit import instrument_config, analyze_config\n",
    "from sflkit.config import Config"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "7be07d73-70af-4d38-9bc9-2d2f027987f0",
   "metadata": {},
   "source": [
    "## A faulty Program"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "742bbc76-5fd4-46a9-ac78-f132118ec072",
   "metadata": {},
   "source": [
    "First, we need a faulty program. We chose an implementation of the `middle(x, y, z)` function that returns the *middle* number of its three arguments. For example, `middle(1, 3, 2)` should return 2 because `1 < 2` and `2 < 3`. We introduced a fault in this implementation of `middle` that occurs in line 7 `m = y`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "97ceacc2-3d6f-4d8f-95b4-11d45e805760",
   "metadata": {},
   "source": [
    "def middle(x, y, z):\n",
    "    m = z\n",
    "    if y < z:\n",
    "        if x < y:\n",
    "            m = y\n",
    "        elif x < z:\n",
    "            m = y  # bug\n",
    "    else:\n",
    "        if x > y:\n",
    "            m = y\n",
    "        elif x > z:\n",
    "            m = x\n",
    "    return m"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "3669e4eb-0643-488e-9859-9290dc88eea5",
   "metadata": {},
   "source": [
    "Next, we introduce a class to capture test runs' results efficiently. The `TestResult` is an enum with two possible values, `PASS`and `FAIL`. `PASS` donates a passing test case and `FAIL` a failing one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "374e93f7-e48b-46fd-997d-54dea2b67e12",
   "metadata": {},
   "source": [
    "class TestResult(enum.Enum):\n",
    "    \n",
    "    def __repr__(self):\n",
    "        return self.value\n",
    "    \n",
    "    PASS = 'PASS'\n",
    "    FAIL = 'FAIL'"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "89df6ffe-a6b3-4a56-bd07-cd74a0fb438e",
   "metadata": {},
   "source": [
    "Now we implement a test function that takes the three arguments of `middle(x, y, z)` and an expected result. This test function compares the return of `middle(x, y, z)` with the desired value and returns `PASS` if they match and `FAIL` otherwise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "eea55fff-0efa-48ba-8112-ed6f5698b645",
   "metadata": {},
   "source": [
    "def test(function, x, y, z, expected):\n",
    "    try:\n",
    "        if function(x, y, z) == expected:\n",
    "            return TestResult.PASS\n",
    "        else:\n",
    "            return TestResult.FAIL\n",
    "    except BaseException:\n",
    "        return TestResult.FAIL\n",
    "\n",
    "def test_middle(x, y, z, expected):\n",
    "    return test(middle, x, y, z, expected)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "dfe80e8c-0bac-4f7e-9919-51b2befb7452",
   "metadata": {},
   "source": [
    "Let's check the results for some combinations of the numbers 1, 2, and 3. The expected value is in all cases 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "549d8951-4d04-4d38-baf6-f561e5d72888",
   "metadata": {},
   "source": [
    "test_middle(3, 2, 1, expected=2)"
   ],
   "outputs": []
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "45baa555-b8a8-48d1-bb82-02614bcd7c60",
   "metadata": {},
   "source": [
    "test_middle(3, 1, 2, expected=2)"
   ],
   "outputs": []
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "1b35abe6-1ee9-48cc-a327-2db6c05a1784",
   "metadata": {},
   "source": [
    "test_middle(2, 1, 3, expected=2)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "8296a6c8-413e-4d4e-aa6d-ed43c8b51c7d",
   "metadata": {},
   "source": [
    "As you can see, the result of `middle(2, 1, 3)` does not match the expected value 2. Hence, we found a failing test case"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0b0fe1c-d040-4fa0-8fd9-d04eea5a6413",
   "metadata": {},
   "source": [
    "## Statistical Fault Localization\n",
    "\n",
    "Statistical fault localization aims at detecting execution features that correlate with failures, such as whether individual lines are part of the execution.\n",
    "\n",
    "With the example from above and the three test cases, we can run a statistical fault localization by marking which test case executes which lines:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f25bc4c-fa08-4889-a182-f3d83c569e77",
   "metadata": {
    "tags": []
   },
   "source": [
    "<table>\n",
    "<tr>\n",
    "<td></td>\n",
    "<td></td>\n",
    "<td>\n",
    "\n",
    "```python\n",
    "middle(3, 2, 1)\n",
    "```\n",
    "            \n",
    "</td>\n",
    "<td>\n",
    "\n",
    "```python\n",
    "middle(3, 1, 2)\n",
    "```\n",
    "            \n",
    "</td>\n",
    "<td>\n",
    "\n",
    "```python\n",
    "middle(2, 1, 3)\n",
    "```\n",
    "            \n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>\n",
    "\n",
    "```\n",
    "1\n",
    "2\n",
    "3\n",
    "4\n",
    "5\n",
    "6\n",
    "7\n",
    "8\n",
    "9\n",
    "10\n",
    "11\n",
    "12\n",
    "13\n",
    "```\n",
    "            \n",
    "</td>\n",
    "<td>\n",
    "\n",
    "```python\n",
    "def middle(x, y, z):\n",
    "    m = z\n",
    "    if y < z:\n",
    "        if x < y:\n",
    "            m = y\n",
    "        elif x < z:\n",
    "            m = y  # bug\n",
    "    else:\n",
    "        if x > y:\n",
    "            m = y\n",
    "        elif x > z:\n",
    "            m = x\n",
    "    return m\n",
    "```\n",
    "            \n",
    "</td>\n",
    "<td>\n",
    "\n",
    "```\n",
    "\n",
    "       X\n",
    "       X\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "       X\n",
    "       X\n",
    "\n",
    "\n",
    "       X\n",
    "```\n",
    "            \n",
    "</td>\n",
    "<td>\n",
    "\n",
    "```\n",
    "\n",
    "       X\n",
    "       X\n",
    "       X\n",
    "       \n",
    "       X\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "       X\n",
    "```\n",
    "         \n",
    "</td>\n",
    "<td>\n",
    "\n",
    "```\n",
    "\n",
    "       X\n",
    "       X\n",
    "       X\n",
    "       \n",
    "       X\n",
    "       X\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "       X\n",
    "```\n",
    "            \n",
    "</td>\n",
    "</tr>\n",
    "  \n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b64a07c-09c8-4a16-ad9c-6b6d2004e899",
   "metadata": {},
   "source": [
    "We can see here that line 7 gets only executed by the failing test and not by the passing tests, making it the most likely line to contain the fault, which we already know is true."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "beb7a2ea-1d08-41b2-90b5-bc83c2e22edd",
   "metadata": {},
   "source": [
    "[next](#Configuring-SFLKit)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebf75f32-f4c3-4d96-a258-569da3b882e0",
   "metadata": {},
   "source": [
    "## Instrument the Program"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29f1e1e5-3b88-4f0c-a72c-b15215c0ad5f",
   "metadata": {},
   "source": [
    "Subsequently, we want to leverage SFLKit to find the location in the code that is most likely to include the fault.\n",
    "\n",
    "Let us first get the source of our function and write it to a file so we have something to perform our instrumentation and analysis.\n",
    "\n",
    "We leverage Python's `inspect` to get the source code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d395a695-ba1d-47e9-b21f-ef2d6c95fb37",
   "metadata": {},
   "source": [
    "source = inspect.getsource(middle)\n",
    "print(source)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "06b9faaf-25ca-4b33-aad6-b7566da54913",
   "metadata": {},
   "source": [
    "We also define the file we write the source to and the python file we will work on, namely `middle.py` and `tmp.py`, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "76d8e864-1983-4d28-9403-141d20477d75",
   "metadata": {},
   "source": [
    "middle_py = 'middle.py'\n",
    "tmp_py = 'tmp.py'"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "5c3cfffa-93b4-45a6-bfc6-2906da72d538",
   "metadata": {},
   "source": [
    "We write the source code to `middle.py`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1cac1b2a-a3f4-45ca-a852-897dc1b754d7",
   "metadata": {},
   "source": [
    "with open(middle_py, 'w') as fp:\n",
    "    fp.write(source)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "d417ea19-6938-44b3-999c-06d625c10206",
   "metadata": {},
   "source": [
    "Let's update our test function to import the correct module and run the `middle(x, y, z)` from this module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "085526f9-fadb-4b73-a5f6-df6e6f82a267",
   "metadata": {},
   "source": [
    "def test_middle_import(x, y, z, expected):\n",
    "    from middle import middle\n",
    "    return test(middle, x, y, z, expected)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c88260b4-24b9-4381-a361-a4da931bb1e7",
   "metadata": {},
   "source": [
    "We repeat the tests to check that our setup works with the import."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "7eb154e9-e4c4-4be6-a8ee-ee1c3f1d1fa9",
   "metadata": {},
   "source": [
    "test_middle_import(3, 2, 1, expected=2), test_middle_import(3, 1, 2, expected=2), test_middle_import(2, 1, 3, expected=2)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "6c623cfd-eb78-4fa3-afc8-85a82d48002b",
   "metadata": {},
   "source": [
    "We produced the same results for the test cases, so it seems to work."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12423b92-7543-4017-9891-6bad2a2994e1",
   "metadata": {},
   "source": [
    "### Configuring SFLKit\n",
    "\n",
    "The `Config` class provides comfortable access to `SFLKit` by defining the fundamental concepts we want to investigate.\n",
    "\n",
    "We give some information for the config that we need to define. First, we need the path to the source we want to investigate, which we already have in `middle_py`. Next, we need an out, `tmp_py`. We also need:\n",
    "\n",
    "The language of our subject is `'python'`.\n",
    "Let's start with `'line'` as the predicates we want to investigate.\n",
    "We define `'tarantula'` as our evaluation metric for the predicates, i.e., the similarity coefficient.\n",
    "We also need a list of passing and failing tests used during the analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "e43c061a-807b-4bc3-b707-7e1c89799f99",
   "metadata": {},
   "source": [
    "language='python'\n",
    "predicates='line'\n",
    "metrics='Tarantula'\n",
    "passing='event-files/0,event-files/1'\n",
    "failing='event-files/2'"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "6bc586ae-8d1d-4bb0-987c-cdd608f6ea20",
   "metadata": {},
   "source": [
    "We define a function that gives as a `Config` object, so we do not need to create it manually every time we change something."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "b0cd0637-7b4a-4736-80a3-6211b7e529c9",
   "metadata": {},
   "source": [
    "def get_config():\n",
    "    return Config.create(path=middle_py, working=tmp_py, language=language, predicates=predicates, metrics=metrics, passing=passing, failing=failing)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "a905e276-c475-4fc4-b23c-936dd5fea818",
   "metadata": {},
   "source": [
    "Now we can define a function that instruments our subject. We leverage `SFLKit`'s `instrument_config()`, which takes a config we create with our defined `get_config()` and instruments the subject. We can also show the content of the instrumented python file with this function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d0a87958-fbf9-4ad8-91c5-c4f9526ebcc5",
   "metadata": {},
   "source": [
    "def instrument(out=True):\n",
    "    instrument_config(get_config())\n",
    "    if out:\n",
    "        with open(tmp_py, 'r') as fp:\n",
    "            print(fp.read())"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "65524f0a-7c30-41db-be30-a108261e5592",
   "metadata": {},
   "source": [
    "Now we instrument our `middle.py` subject and check the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "c19129e7-cf49-4028-8997-76afeafc9324",
   "metadata": {},
   "source": [
    "instrument()"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "ba9c03c5-419d-49a0-accb-53b190d200dd",
   "metadata": {},
   "source": [
    "As you can see, the instrumentation added an import at the beginning to a lib that comes with `SFLKit`, cluing the execution of files together. Moreover, the instrumentation added a function call function of the lib in front of each executable line that tracks the executed lines."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "690cf88f-0965-4ee7-9ea3-84e0a536e08d",
   "metadata": {},
   "source": [
    "## Get and Analyze Events"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4fe045c-cb86-4c05-90e6-730ab1123601",
   "metadata": {},
   "source": [
    "Now, we want to extract the events from the execution of tests. Therefore, we need to adjust our test execution function again because the shared library for tracking the events does not know when to start and stop. We need to reset this library before entering our `middle.py` and tell the library to dump the events after the function finishes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "a5daf879-9158-402f-8b5f-bab67f4736da",
   "metadata": {},
   "source": [
    "def test_tmp(x, y, z, expected): \n",
    "    import tmp\n",
    "    importlib.reload(tmp)\n",
    "    tmp.sflkitlib.lib.reset()\n",
    "    try:\n",
    "        return test(tmp.middle, x, y, z, expected)\n",
    "    finally:\n",
    "        tmp.sflkitlib.lib.dump_events()\n",
    "        del tmp"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "bb741ed6-bb58-4b7a-9c56-8e58804651d3",
   "metadata": {},
   "source": [
    "We define a path to write the generated event logs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "823303be-b13f-4405-9186-857625aec1b1",
   "metadata": {},
   "source": [
    "event_files = 'event-files'"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "8f8453ef-c2ba-4b77-800a-0c44b72bfd5a",
   "metadata": {},
   "source": [
    "Then, we need a function to generate the event log from the previous test cases. We change the environment variable `EVENTS_PATH` to the output path of the event log file before running each test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "e084ac5b-b15e-47d0-8cda-4dc14adbb2e9",
   "metadata": {},
   "source": [
    "def run_tests():\n",
    "    if os.path.exists(event_files):\n",
    "        shutil.rmtree(event_files)\n",
    "    os.mkdir(event_files)\n",
    "    os.environ['EVENTS_PATH'] = os.path.join(event_files, '0')\n",
    "    test_tmp(3, 2, 1, expected=2)\n",
    "    os.environ['EVENTS_PATH'] = os.path.join(event_files, '1')\n",
    "    test_tmp(3, 1, 2, expected=2)\n",
    "    os.environ['EVENTS_PATH'] = os.path.join(event_files, '2')\n",
    "    test_tmp(2, 1, 3, expected=2)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "bf9f60a3-c565-404d-8d61-7b804fb65907",
   "metadata": {},
   "source": [
    "With this, we can execute the tests and analyze the result with the help of `analyze_config()` from SFLKit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "d922df2e-39a9-4aea-a926-b9275bdd9197",
   "metadata": {},
   "source": [
    "def analyze():\n",
    "    run_tests()\n",
    "    return analyze_config(get_config())"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "726382fb-4b55-4952-b105-e0afd88dd36f",
   "metadata": {},
   "source": [
    "Let's execute the tests and analyze the event logs for lines and the Tarantula metric."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "f52db0b7-4423-49a2-b79c-3bff82e9f084",
   "metadata": {},
   "source": [
    "results = analyze()"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "09b4a254-9059-4e18-87d2-a8a9c01a26d4",
   "metadata": {},
   "source": [
    "The results look something like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "cd538827-c2b6-45fe-a3bf-1bae6e213223",
   "metadata": {},
   "source": [
    "results"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "7ee136ed-b595-448a-b2e6-89e9d1b16d0b",
   "metadata": {},
   "source": [
    "This structure maps analysis objects and metrics to a list of sorted suggestions where the fault occurs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29fee667-ff00-426a-ab91-254a89f09bb8",
   "metadata": {},
   "source": [
    "Now, we can put all this together and produce a pretty output that shows us where the fault originates by leveraging `SFLKit`'s `ColorCode` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "d2e2b4ed-5c28-45c9-bb12-bd746e7bc966",
   "metadata": {},
   "source": [
    "def sfl():\n",
    "    instrument(out=False)\n",
    "    results = analyze()\n",
    "    code = ColorCode(results[predicates.upper()][metrics])\n",
    "    return HTML(code.code(middle_py, source, color=True, suspiciousness=True))"
   ],
   "outputs": []
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "a7a490d9-7622-4e46-968e-aeb0d7b368b0",
   "metadata": {},
   "source": [
    "sfl()"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "a35f1a31-eba0-4efc-8c44-545f8ecf95ca",
   "metadata": {},
   "source": [
    "As you can see, the analysis indeed suggests the buggy line 7 as the most suspicious."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c9e89ce-815f-4d84-aa6f-f43d41030c96",
   "metadata": {},
   "source": [
    "But what if lines are not enough to show the fault?\n",
    "\n",
    "What if the metric we have chosen for evaluation is insufficient?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21426bec-6997-490d-8bd8-f836ea28f4f1",
   "metadata": {},
   "source": [
    "## Change the Analysis Object"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1bae9ae4-abf8-4c34-8175-cedc30e66401",
   "metadata": {},
   "source": [
    "Say you want to investigate different code elements, for instance, def-use pairs. We can do so by simply adjusting the `predicates` of the `Config` object. Keep in mind that `get_config()` automatically updates the `Config` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "bf3f66c0-1131-4017-bbd2-a4d68eec4c22",
   "metadata": {},
   "source": [
    "predicates='def_use'\n",
    "sfl()"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "40ec3073-553c-4556-8f1f-a6e217323981",
   "metadata": {},
   "source": [
    "Now we have an analysis for def-use pairs. The bug occurs when `y` gets defined in line 1, used in line 7, and `m` gets defined in line 7 and used in line 13. So we found the data flow leading to the fault."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbc111ff-ea08-46d5-ad10-4ea87253a9ef",
   "metadata": {},
   "source": [
    "## Change the Metric"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "727a99ec-cd07-4add-8e6e-6740910bec22",
   "metadata": {},
   "source": [
    "We can apply the same to the metric, for instance, if we want to investigate the Jaccard similarity coefficient."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "416d8d9f-2636-4e79-8dd4-83d5c3a3e68f",
   "metadata": {},
   "source": [
    "metrics='Jaccard'\n",
    "sfl()"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "2b5a5c6d-2396-4255-a2cb-a08460de602b",
   "metadata": {},
   "source": [
    "We have an adjusted def-use pair analysis to the Jaccard coefficient with this result. You can compare it with the previous one and see the differences in the suspiciousness."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea11fde8-0ce5-4f43-8e0c-89cf94623523",
   "metadata": {},
   "source": [
    "## From Spectra to Predicates"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8681643-27c6-4e7b-b7bf-fe52d6499c2b",
   "metadata": {},
   "source": [
    "Up to this point, we have investigated spectra, but we could also examine predicates that need to hold, for instance, the conditions of branches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "0c9dcfc0-98db-4151-a0bd-44862985a3b4",
   "metadata": {},
   "source": [
    "predicates='branch'\n",
    "metrics='IncreaseTrue'\n",
    "sfl()"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "875b493b-79f2-4ec0-a74a-23dcd3a2d426",
   "metadata": {},
   "source": [
    "We can map these predicates to code locations and see what branch causes the fault."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6447144c-5fa8-4004-91c7-dd86ca4761b4",
   "metadata": {},
   "source": [
    "# Thank You"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c14eb751-f214-4447-9b17-7066e3e6aaf5",
   "metadata": {},
   "source": [
    "## GitHub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "501b70f5-3276-45e8-9e48-7331964004d7",
   "metadata": {},
   "source": [
    "https://github.com/uds-se/sflkit"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}