notebooks/bellingcat/wayback-google-analytics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Wayback Google Analytics\n",
    "\n",
    "<a href=\"https://www.bellingcat.com\"><img alt=\"Bellingcat logo: Discover Bellingcat\" src=\"https://img.shields.io/badge/Discover%20Bellingcat-%20?style=for-the-badge&logo=data%3Aimage%2Fpng%3Bbase64%2CiVBORw0KGgoAAAANSUhEUgAAAA4AAAAYCAYAAADKx8xXAAABhGlDQ1BJQ0MgcHJvZmlsZQAAKJF9kT1Iw0AcxV9TS0UqDnZQEcxQneyiIo6likWwUNoKrTqYXPoFTRqSFBdHwbXg4Mdi1cHFWVcHV0EQ%2FABxdnBSdJES%2F5cUWsR4cNyPd%2Fced%2B8AoVllqtkTA1TNMtKJuJjLr4rBVwQwhhBEDEvM1JOZxSw8x9c9fHy9i%2FIs73N%2Fjn6lYDLAJxLHmG5YxBvEs5uWznmfOMzKkkJ8Tjxp0AWJH7kuu%2FzGueSwwDPDRjY9TxwmFktdLHcxKxsq8QxxRFE1yhdyLiuctzir1Tpr35O%2FMFTQVjJcpzmKBJaQRIo6klFHBVVYiNKqkWIiTftxD%2F%2BI40%2BRSyZXBYwcC6hBheT4wf%2Fgd7dmcXrKTQrFgcCLbX%2BMA8FdoNWw7e9j226dAP5n4Err%2BGtNYO6T9EZHixwBA9vAxXVHk%2FeAyx1g6EmXDMmR%2FDSFYhF4P6NvygODt0Dfmttbex%2BnD0CWulq%2BAQ4OgYkSZa97vLu3u7d%2Fz7T7%2BwHEU3LHAa%2FQ6gAAAAZiS0dEAAAAAAAA%2BUO7fwAAAAlwSFlzAAAuIwAALiMBeKU%2FdgAAAAd0SU1FB%2BgFHwwiMH4odB4AAAAZdEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIEdJTVBXgQ4XAAAA50lEQVQ4y82SvWpCQRCFz25ERSJiCNqlUiS1b5AuEEiZIq1NOsGXCKms0wXSp9T6dskDiFikyiPc%2FrMZyf3FXSGQ0%2BzuzPl2ZoeVKgQ0gQ2wBVpVHlcDkjM5V%2FJ5nag6sJ%2FZX%2Bh%2FC7gEhqeAFKf7p1M9aB3b5oN1OomB7g1axUBPBr3GQHODHmOgqUF3MZAzKI2d4LWBV4H%2BMXDuJd1a7Cew1k7SwksaHC4LqNaw7aeX9GWHXkC1G1sTAS17Y3Kk2lnp4wNLiz0DrgLq8qt2MfmSSabAO%2FBBXp26dtrADPjOmN%2BAUdG7B3cE61l5hOZiAAAAAElFTkSuQmCC&logoColor=%23fff&color=%23000\"></a><!--\n",
    "--><a href=\"https://discord.gg/bellingcat\"><img alt=\"Discord logo: Join our community\" src=\"https://img.shields.io/badge/Join%20our%20community-%20?style=for-the-badge&logo=discord&logoColor=%23fff&color=%235865F2\"></a><!--\n",
    "--><a href=\"https://www.bellingcat.com/resources/2024/01/09/using-the-wayback-machine-and-google-analytics-to-uncover-disinformation-networks/\"><img alt=\"Book icon: explore the guide\" src=\"https://img.shields.io/badge/Explore%20the%20guide-%20?style=for-the-badge&logo=ReadMe&logoColor=fff&color=C41E3D\"></a>\n",
    "\n",
    "Gather historic Google analytics data (UA, GA, and GTM codes) from a collection of website urls.\n",
    "\n",
    "Read [more about the tool and google analytics codes](https://github.com/bellingcat/wayback-google-analytics/blob/main/README.md#about-the-project).\n",
    "\n",
    "#### [Read the article on bellingcat.com](https://www.bellingcat.com/resources/2024/01/09/using-the-wayback-machine-and-google-analytics-to-uncover-disinformation-networks/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1 - install the python package "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "bat"
    }
   },
   "outputs": [],
   "source": [
    "!pip install wayback-google-analytics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2 - Get a full command line description by calling `help` \n",
    "This will show all the command line options we can use and what each of them does"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "bat"
    }
   },
   "outputs": [],
   "source": [
    "!wayback-google-analytics --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3 - call the tool\n",
    "Using the options from above, let's call the CLI to get all google analytics ids from 4 different websites:\n",
    "- `https://yapatriot.ru`\n",
    "- `https://zanogu.com`\n",
    "- `https://whoswho.com.ua`\n",
    "- `https://adamants.ru`\n",
    "\n",
    "Starting in Jan 1st 2015 (`-s 01/01/2015`) by checking changes on a yearly frequency (`-f yearly`) and save the results into an excel file (`-o xlsx`). \n",
    "\n",
    "The progress will be visible in the console and the results in the last row of the console. The excel file (or csv,txt,json if you change the command to that output) will be written to the `output/` folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "bat"
    }
   },
   "outputs": [],
   "source": [
    "!wayback-google-analytics -u https://yapatriot.ru https://zanogu.com https://whoswho.com.ua https://adamants.ru -s 01/01/2015 -f yearly -o xlsx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an example output for one of the sites (`yapatriot.ru`) that has 2 historical UA codes (`UA-65087228-1` that is the 1st assigned id to the analytics account `UA-65087228` and `UA-53176102-14` which is the 14th assigned id to the `UA-53176102` analytics account):\n",
    "```\n",
    "[{\n",
    "\t'https://yapatriot.ru': {\n",
    "\t\t'archived_UA_codes': {\n",
    "\t\t\t'UA-65087228-1': {\n",
    "\t\t\t\t'first_seen': '20/01/2017:03:55',\n",
    "\t\t\t\t'last_seen': '30/06/2019:05:32'\n",
    "\t\t\t},\n",
    "\t\t\t'UA-53176102-14': {\n",
    "\t\t\t\t'first_seen': '15/06/2015:19:36',\n",
    "\t\t\t\t'last_seen': '15/06/2015:19:36'\n",
    "\t\t\t}\n",
    "\t\t},\n",
    "\t\t'archived_GA_codes': {},\n",
    "\t\t'archived_GTM_codes': {}\n",
    "\t}\n",
    "}\n",
    "...\n",
    "]\n",
    "```\n",
    "You can jump into other tools such as https://spyonweb.com/ and see that [`UA-53176102`](https://spyonweb.com/ua-53176102) is used on at least 4 other sites (`material-evidence.com` `news-region.ru` `syriainform.com` `whoswho.com.ua`) very likely belonging to the same owner. To note that between 2016-2017 this page started using `UA-65087228` instead (this would be the code in the live website), a fresh UA id which would not reveal the connection we uncovered with the historical UA code.\n",
    "\n",
    "That's a good example of how this tool can help investigations, by digging through historical ids. If you want to be even more thorough you can update the frequency parameter to `-f daily` or even `-f hourly` but that will make running the tool much slower."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "open-source-research-notebooks-4sg58OrJ",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}