notebooks/ts-1-fetching-metrics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fetching metrics\n",
    "\n",
    "In this notebook, we will learn how to use the Prometheus API client for fetching and formatting the raw metrics obtained from a [Prometheus](https://prometheus.io/) host to drive better data science analysis on these metrics. \n",
    "\n",
    "You can find more information about the functions of the [API client here](https://prometheus-api-client-python.readthedocs.io/source/prometheus_api_client.html).\n",
    "\n",
    "## Installing the library:\n",
    "To begin any exploratory analysis, we need to first install all the required packages. \n",
    "\n",
    "For this notebook in particular, the [prometheus api client library](https://pypi.org/project/prometheus-api-client/) needs to be installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip3 install prometheus-api-client\n",
    "# !pip3 install matplotlib pandas\n",
    "\n",
    "from prometheus_api_client import PrometheusConnect\n",
    "from prometheus_api_client.metric_range_df import MetricRangeDataFrame\n",
    "from prometheus_api_client.utils import parse_datetime\n",
    "from datetime import timedelta\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connecting to Prometheus\n",
    "The `PrometheusConnect` module of the library can be used to connect to a Prometheus host.\n",
    "\n",
    "To know more about this module, you can refer to the [documentation](https://prometheus-api-client-python.readthedocs.io/en/master/source/prometheus_api_client.html#module-prometheus_api_client.prometheus_connect)\n",
    "\n",
    "We will connect to a public prometheus instance: http://demo.robustperception.io:9090"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prometheus uri:  http://demo.robustperception.io:9090\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>metrics</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ALERTS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ALERTS_FOR_STATE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>alertmanager_alerts_invalid_total</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>alertmanager_alerts_received_total</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>scrape_duration_seconds</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>502</th>\n",
       "      <td>scrape_samples_post_metric_relabeling</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>503</th>\n",
       "      <td>scrape_samples_scraped</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>504</th>\n",
       "      <td>scrape_series_added</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>up</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>506 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   metrics\n",
       "0                                   ALERTS\n",
       "1                         ALERTS_FOR_STATE\n",
       "2                      alertmanager_alerts\n",
       "3        alertmanager_alerts_invalid_total\n",
       "4       alertmanager_alerts_received_total\n",
       "..                                     ...\n",
       "501                scrape_duration_seconds\n",
       "502  scrape_samples_post_metric_relabeling\n",
       "503                 scrape_samples_scraped\n",
       "504                    scrape_series_added\n",
       "505                                     up\n",
       "\n",
       "[506 rows x 1 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prom_url = \"http://demo.robustperception.io:9090\"\n",
    "print(\"Prometheus uri: \", prom_url)\n",
    "\n",
    "# Creating the prometheus connect object with the required parameters\n",
    "pc = PrometheusConnect(url=prom_url, disable_ssl=True)\n",
    "\n",
    "# Fetching a list of all metrics scraped by the Prometheus host.\n",
    "pd.DataFrame(pc.all_metrics(), columns={\"metrics\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fetching Metrics from Prometheus\n",
    "Every metric in Prometheus is stored as **time series** data: streams of timestamped values belonging to the same metric and the same set of labeled dimensions. Each of these time series is uniquely identified by:\n",
    "* **metric name** - Specifies the general feature of a system that is measured. \n",
    "E.g. `http_requests_total` - the total number of HTTP requests received.\n",
    "* **labels** - Provides more details to identify a particular dimensional instantiation of the metric. E.g. `http_requests_total{method=\"POST\", handler=\"/api/tracks\"}`: all HTTP requests that used the method POST to the /api/tracks handler\n",
    "\n",
    "Prometheus provides a functional query language called **PromQL (Prometheus Query Language)** that lets the user select and aggregate time series data in real time. \n",
    "\n",
    "The `get_current_metric_value()` method in the library can be used to fetch metrics according to this PromQL format.\n",
    "\n",
    "Parameters:\t\n",
    "* **metric_name** – (str) This is a PromQL query, a few examples can be found at https://prometheus.io/docs/prometheus/latest/querying/examples/\n",
    "* **params** – (dict) Optional dictionary containing GET parameters to be sent along with the API request, such as “time”\n",
    "\n",
    "\n",
    "### Examples of fetching metrics using different queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'metric': {'__name__': 'alertmanager_alerts',\n",
       "   'instance': 'demo.robustperception.io:9093',\n",
       "   'job': 'alertmanager',\n",
       "   'state': 'active'},\n",
       "  'value': [1606158844.546, '4']},\n",
       " {'metric': {'__name__': 'alertmanager_alerts',\n",
       "   'instance': 'demo.robustperception.io:9093',\n",
       "   'job': 'alertmanager',\n",
       "   'state': 'suppressed'},\n",
       "  'value': [1606158844.546, '0']}]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Here, we are fetching the current values of a particular metric name\n",
    "pc.get_current_metric_value(\"alertmanager_alerts\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'metric': {'__name__': 'alertmanager_alerts',\n",
       "   'instance': 'demo.robustperception.io:9093',\n",
       "   'job': 'alertmanager',\n",
       "   'state': 'active'},\n",
       "  'value': [1606158848.546, '4']}]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now, let's see if we can fetch a particular label configuration of this metric\n",
    "pc.get_current_metric_value(\"alertmanager_alerts{state='active'}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'metric': {}, 'value': [1606158849.597, '0.042080264']}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sum of all the values of a metric\n",
    "# You can also try methods such as rate, and count\n",
    "# More functions here: https://prometheus.io/docs/prometheus/latest/querying/examples/\n",
    "pc.get_current_metric_value(\"sum(scrape_duration_seconds)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collecting Historical Data\n",
    "Suppose we want to fetch historical data instead of just the current value, say the past few days of data, we can do so by using the `get_metric_range_data()` method. This method will fetch the data for the specifed metric label configuration within the time range specified. \n",
    "Along with specifying the start_time and end_time for the metric data collection, we need to specify the chunk size or the amount of data requested in one go. A large chunk size can result in non responsive code if the data is huge. If the end goal is to create a data frame, the chunk size will not affect the final frame size. Therefore it is a good practice to make sure the chunk size is balanced, i.e, not too high or not too low."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'__name__': 'alertmanager_alerts',\n",
       " 'instance': 'demo.robustperception.io:9093',\n",
       " 'job': 'alertmanager',\n",
       " 'state': 'active'}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metric_data = pc.get_metric_range_data(\n",
    "    \"alertmanager_alerts{job='alertmanager'}\",  # metric name and label config\n",
    "    start_time=parse_datetime(\n",
    "        \"2d\"\n",
    "    ),  # datetime object for metric range start time\n",
    "    end_time=parse_datetime(\n",
    "        \"now\"\n",
    "    ),  # datetime object for metric range end time\n",
    "    chunk_size=timedelta(\n",
    "        hours=12\n",
    "    ),  # timedelta object for duration of metric data downloaded in one request\n",
    ")\n",
    "metric_data[0][\"metric\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating pandas Data Frame\n",
    "We can easily create a pandas data frame from the json response using the `MetricRangeDataFrame` class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "metric_df = MetricRangeDataFrame(metric_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>__name__</th>\n",
       "      <th>instance</th>\n",
       "      <th>job</th>\n",
       "      <th>state</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>timestamp</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1.605986e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.605986e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.605986e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.605986e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.605986e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.606159e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.606159e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.606159e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.606159e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.606159e+09</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>34558 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                         __name__                       instance  \\\n",
       "timestamp                                                          \n",
       "1.605986e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "1.605986e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "1.605986e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "1.605986e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "1.605986e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "...                           ...                            ...   \n",
       "1.606159e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "1.606159e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "1.606159e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "1.606159e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "1.606159e+09  alertmanager_alerts  demo.robustperception.io:9093   \n",
       "\n",
       "                       job       state value  \n",
       "timestamp                                     \n",
       "1.605986e+09  alertmanager      active     4  \n",
       "1.605986e+09  alertmanager      active     4  \n",
       "1.605986e+09  alertmanager      active     4  \n",
       "1.605986e+09  alertmanager      active     4  \n",
       "1.605986e+09  alertmanager      active     4  \n",
       "...                    ...         ...   ...  \n",
       "1.606159e+09  alertmanager  suppressed     0  \n",
       "1.606159e+09  alertmanager  suppressed     0  \n",
       "1.606159e+09  alertmanager  suppressed     0  \n",
       "1.606159e+09  alertmanager  suppressed     0  \n",
       "1.606159e+09  alertmanager  suppressed     0  \n",
       "\n",
       "[34558 rows x 5 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metric_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Converting utc timestamp to python datetime object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "metric_df.index = pd.to_datetime(metric_df.index, unit=\"s\", utc=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>__name__</th>\n",
       "      <th>instance</th>\n",
       "      <th>job</th>\n",
       "      <th>state</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>timestamp</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-11-21 19:14:16.102999926+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-21 19:14:26.095999956+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-21 19:14:36.095999956+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-21 19:14:46.098000050+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-21 19:14:56.105000019+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>active</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-23 19:13:26.095999956+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-23 19:13:36.105000019+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-23 19:13:46.114000082+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-23 19:13:56.096999884+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-11-23 19:14:06.095999956+00:00</th>\n",
       "      <td>alertmanager_alerts</td>\n",
       "      <td>demo.robustperception.io:9093</td>\n",
       "      <td>alertmanager</td>\n",
       "      <td>suppressed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>34558 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                __name__  \\\n",
       "timestamp                                                  \n",
       "2020-11-21 19:14:16.102999926+00:00  alertmanager_alerts   \n",
       "2020-11-21 19:14:26.095999956+00:00  alertmanager_alerts   \n",
       "2020-11-21 19:14:36.095999956+00:00  alertmanager_alerts   \n",
       "2020-11-21 19:14:46.098000050+00:00  alertmanager_alerts   \n",
       "2020-11-21 19:14:56.105000019+00:00  alertmanager_alerts   \n",
       "...                                                  ...   \n",
       "2020-11-23 19:13:26.095999956+00:00  alertmanager_alerts   \n",
       "2020-11-23 19:13:36.105000019+00:00  alertmanager_alerts   \n",
       "2020-11-23 19:13:46.114000082+00:00  alertmanager_alerts   \n",
       "2020-11-23 19:13:56.096999884+00:00  alertmanager_alerts   \n",
       "2020-11-23 19:14:06.095999956+00:00  alertmanager_alerts   \n",
       "\n",
       "                                                          instance  \\\n",
       "timestamp                                                            \n",
       "2020-11-21 19:14:16.102999926+00:00  demo.robustperception.io:9093   \n",
       "2020-11-21 19:14:26.095999956+00:00  demo.robustperception.io:9093   \n",
       "2020-11-21 19:14:36.095999956+00:00  demo.robustperception.io:9093   \n",
       "2020-11-21 19:14:46.098000050+00:00  demo.robustperception.io:9093   \n",
       "2020-11-21 19:14:56.105000019+00:00  demo.robustperception.io:9093   \n",
       "...                                                            ...   \n",
       "2020-11-23 19:13:26.095999956+00:00  demo.robustperception.io:9093   \n",
       "2020-11-23 19:13:36.105000019+00:00  demo.robustperception.io:9093   \n",
       "2020-11-23 19:13:46.114000082+00:00  demo.robustperception.io:9093   \n",
       "2020-11-23 19:13:56.096999884+00:00  demo.robustperception.io:9093   \n",
       "2020-11-23 19:14:06.095999956+00:00  demo.robustperception.io:9093   \n",
       "\n",
       "                                              job       state value  \n",
       "timestamp                                                            \n",
       "2020-11-21 19:14:16.102999926+00:00  alertmanager      active     4  \n",
       "2020-11-21 19:14:26.095999956+00:00  alertmanager      active     4  \n",
       "2020-11-21 19:14:36.095999956+00:00  alertmanager      active     4  \n",
       "2020-11-21 19:14:46.098000050+00:00  alertmanager      active     4  \n",
       "2020-11-21 19:14:56.105000019+00:00  alertmanager      active     4  \n",
       "...                                           ...         ...   ...  \n",
       "2020-11-23 19:13:26.095999956+00:00  alertmanager  suppressed     0  \n",
       "2020-11-23 19:13:36.105000019+00:00  alertmanager  suppressed     0  \n",
       "2020-11-23 19:13:46.114000082+00:00  alertmanager  suppressed     0  \n",
       "2020-11-23 19:13:56.096999884+00:00  alertmanager  suppressed     0  \n",
       "2020-11-23 19:14:06.095999956+00:00  alertmanager  suppressed     0  \n",
       "\n",
       "[34558 rows x 5 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metric_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "Great! In this notebook we saw how to use prometheus api client to create a pandas time series dataframe that can be used for analysis. In the next post, we will learn how to manipulate and visualize this dataframe.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}