examples/finding_pairs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matching profiles based on metadata columns\n",
    "\n",
    "This example demostrates how to use `copairs` to group profiles based on their metadata properties.\n",
    "\n",
    "Specifically, this is used in calculation of mAP for profile strength and similarity assesement.\n",
    "\n",
    "Citation:\n",
    "> Kalinin, A. A. et al. A versatile information retrieval framework for evaluating profile strength and similarity. bioRxiv, 2024-04, (2024)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import pandas as pd\n",
    "\n",
    "from copairs import Matcher, MatcherMultilabel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data\n",
    "\n",
    "Let's assume you have a dataset with 20 samples taken in 3 plates `p1, p2, p3`,\n",
    "each plate is composed of 5 wells `w1, w2, w3, w4, w5`, and each well \n",
    "has one or more labels (`t1, t2, t3, t4`) assigned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "random.seed(0)\n",
    "n_samples = 20\n",
    "dframe = pd.DataFrame(\n",
    "    {\n",
    "        \"plate\": [random.choice([\"p1\", \"p2\", \"p3\"]) for _ in range(n_samples)],\n",
    "        \"well\": [\n",
    "            random.choice([\"w1\", \"w2\", \"w3\", \"w4\", \"w5\"]) for _ in range(n_samples)\n",
    "        ],\n",
    "        \"label\": [random.choice([\"t1\", \"t2\", \"t3\", \"t4\"]) for _ in range(n_samples)],\n",
    "    }\n",
    ")\n",
    "dframe = dframe.drop_duplicates()\n",
    "dframe = dframe.sort_values(by=[\"plate\", \"well\", \"label\"])\n",
    "dframe = dframe.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting valid pairs\n",
    "\n",
    "To get pairs of samples that share the same `label` but comes from different\n",
    "`plate`s at different `well` positions: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'t1': [(3, 11), (3, 5), (3, 6), (3, 7)],\n",
       " 't2': [(1, 16), (1, 10), (1, 15), (8, 16), (8, 15), (10, 16)],\n",
       " 't3': [(9, 4), (9, 13), (13, 4), (13, 12), (4, 12)],\n",
       " 't4': [(0, 17), (0, 14), (17, 2), (2, 14)]}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "matcher = Matcher(dframe, [\"plate\", \"well\", \"label\"], seed=0)\n",
    "pairs_dict = matcher.get_all_pairs(sameby=[\"label\"], diffby=[\"plate\", \"well\"])\n",
    "pairs_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting valid pairs from a multilabel column\n",
    "\n",
    "For eficiency reasons, you may not want to have duplicated rows. You can\n",
    "group all the labels in a single row and use `MatcherMultilabel` to find the\n",
    "corresponding pairs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>plate</th>\n",
       "      <th>well</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>p1</td>\n",
       "      <td>w2</td>\n",
       "      <td>[t4]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>p1</td>\n",
       "      <td>w3</td>\n",
       "      <td>[t2, t4]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>p1</td>\n",
       "      <td>w4</td>\n",
       "      <td>[t1, t3]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>p2</td>\n",
       "      <td>w1</td>\n",
       "      <td>[t1]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>p2</td>\n",
       "      <td>w2</td>\n",
       "      <td>[t1]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>p2</td>\n",
       "      <td>w3</td>\n",
       "      <td>[t1, t2, t3]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>p2</td>\n",
       "      <td>w4</td>\n",
       "      <td>[t2]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>p2</td>\n",
       "      <td>w5</td>\n",
       "      <td>[t1, t3]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>p3</td>\n",
       "      <td>w1</td>\n",
       "      <td>[t3, t4]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>p3</td>\n",
       "      <td>w4</td>\n",
       "      <td>[t2]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>p3</td>\n",
       "      <td>w5</td>\n",
       "      <td>[t2, t4]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   plate well         label\n",
       "0     p1   w2          [t4]\n",
       "1     p1   w3      [t2, t4]\n",
       "2     p1   w4      [t1, t3]\n",
       "3     p2   w1          [t1]\n",
       "4     p2   w2          [t1]\n",
       "5     p2   w3  [t1, t2, t3]\n",
       "6     p2   w4          [t2]\n",
       "7     p2   w5      [t1, t3]\n",
       "8     p3   w1      [t3, t4]\n",
       "9     p3   w4          [t2]\n",
       "10    p3   w5      [t2, t4]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dframe_multi = dframe.groupby([\"plate\", \"well\"])[\"label\"].unique().reset_index()\n",
    "dframe_multi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "matcher_multi = MatcherMultilabel(\n",
    "    dframe_multi, columns=[\"plate\", \"well\", \"label\"], multilabel_col=\"label\", seed=0\n",
    ")\n",
    "pairs_multi = matcher_multi.get_all_pairs(sameby=[\"label\"], diffby=[\"plate\", \"well\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`pairs_multi` is also a `label_id: pairs` dictionary with the same\n",
    "structure discussed before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'t1': [(2, 7), (2, 3), (2, 4), (2, 5)],\n",
       " 't2': [(1, 10), (1, 6), (1, 9), (5, 10), (5, 9), (6, 10)],\n",
       " 't3': [(5, 2), (5, 8), (8, 2), (8, 7), (2, 7)],\n",
       " 't4': [(0, 10), (0, 8), (10, 1), (1, 8)]}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pairs_multi"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "map_benchmark",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}