QED transformer demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e9c3d576",
   "metadata": {},
   "source": [
    "# SYMBA Model for QED:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ff828b0",
   "metadata": {},
   "source": [
    "We train this model to map the (amplitude) of particles interactions in quantum electrdynamics theory QED to the (squared amplitude), which is the key element in calculating the cross section."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "677bea40",
   "metadata": {},
   "source": [
    "# "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8f74e1ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras.layers import TextVectorization\n",
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a6404f2",
   "metadata": {},
   "source": [
    "## Preparing the data:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d25cb46",
   "metadata": {},
   "source": [
    "To prepape the dataset for training, we use the symbolic computation program MARTY [7] to generate expressions for possible interactions in QED. We restrict the scope 2-to-2 and 2-to-3 particle tree-level processes. All interactions involving off-shell and on-shell particles, anti-particles and gauge bosons are included. Since it is possible for different amplitudes to yield the same squared expressions, we include such amplitudes in our dataset.  All output expressions (squared amplitudes) are simplified with the Python symbolic mathematics module SymPy [25].\n",
    "\n",
    "You can find the data here:\n",
    "https://drive.google.com/file/d/1SRsg0dPh0O2JLop2OsEfuml6qZVbaOFP/view?usp=share_link"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ab14b4a2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data size:  134802\n"
     ]
    }
   ],
   "source": [
    "with open('qed_data.txt', 'r', encoding='utf-8') as f:\n",
    "    lines = f.read().split('\\n')\n",
    "    \n",
    "text_pairs=[]\n",
    "for line in lines[: min(len(lines), len(lines)-1)]:\n",
    "    amplitude, squared_amplitude = line.split('>')\n",
    "    text_pairs.append((amplitude, squared_amplitude ))\n",
    "\n",
    "\n",
    "text_pairs = list(set(text_pairs))\n",
    "print('data size: ', len(text_pairs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "68149a40",
   "metadata": {},
   "outputs": [],
   "source": [
    "def sort(pairs):\n",
    "    dic  = dict()\n",
    "    for i,j in pairs:\n",
    "        dic[(i,j)]=len(j)\n",
    "    pairs_= list(sorted(dic.items(), key=lambda x: x[1]))\n",
    "    \n",
    "    pairs =[]\n",
    "    for i,j in pairs_:\n",
    "        pairs.append(i)\n",
    "    return pairs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a05db585",
   "metadata": {},
   "outputs": [],
   "source": [
    "text_pairs = sort(text_pairs)[:40000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6a780a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "len(text_pairs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce5a434c",
   "metadata": {},
   "source": [
    "#### Examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "550bec71",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Amplitude:   \n",
      " i* e^3* (m_t* gamma_{%\\lambda,%eta,%eta} * gamma_{+%\\lambda,%eps,%del} * gamma_{%\\lambda,%eta,%del} * A_{l,+%\\lambda} (p_1)* e_{j,%del} (p_3)* e_{k,%eps} (p_4)^(*)* t_{i,%del} (p_2)* t_{l,%eta} (p_5)^(*) + -p_1_%\\sigma* gamma_{+%\\sigma,%eta,%eta} * gamma_{%\\lambda,%eta,%eta} * gamma_{+%\\lambda,%eps,%del} * gamma_{%\\lambda,%eta,%del} * A_{l,+%\\lambda} (p_1)* e_{j,%del} (p_3)* e_{k,%eps} (p_4)^(*)* t_{i,%del} (p_2)* t_{l,%eta} (p_5)^(*) + p_5_%\\sigma* gamma_{+%\\sigma,%eta,%eta} * gamma_{%\\lambda,%eta,%eta} * gamma_{+%\\lambda,%eps,%del} * gamma_{%\\lambda,%eta,%del} * A_{l,+%\\lambda} (p_1)* e_{j,%del} (p_3)* e_{k,%eps} (p_4)^(*)* t_{i,%del} (p_2)* t_{l,%eta} (p_5)^(*)) /((s_33 + 2* s_34 + s_44)* (m_t^2 + -s_11 + 2* s_15 + -s_55)) \n",
      "\n",
      "Squared Amplitude:   \n",
      " [ m_t^4 , m_t^2 , 1] , [32 *( 2*m_e^2 + s_34) , 16 *( m_e^2 *( 4*s_11 + 4*s_12 - 4*s_15 - 3*s_25) + 2*s_11*s_34 + 4*s_13*s_24 + 4*s_14*s_23 - 2*s_15*s_34 - 3*s_23*s_45 - 3*s_24*s_35) , -16 *( m_e^2 *( s_11*s_25 - 2*s_12*s_15 + 2*s_12*s_55 - s_25*s_55) + s_11*s_23*s_45 + s_11*s_24*s_35 - 2*s_13*s_15*s_24 + 2*s_13*s_24*s_55 - 2*s_14*s_15*s_23 + 2*s_14*s_23*s_55 - s_23*s_45*s_55 - s_24*s_35*s_55 ] , [(s_33 + 2*s_34 + s_44)^2 *( m_t^2 - s_11 + 2*s_15 - s_55)^2])\n"
     ]
    }
   ],
   "source": [
    "print('Amplitude:  ' , '\\n', text_pairs[1][0], '\\n' )\n",
    "print('Squared Amplitude:  ', '\\n' ,text_pairs[1][1] )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a53c810c",
   "metadata": {},
   "source": [
    "### Data preprocessing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6413a9af",
   "metadata": {},
   "outputs": [],
   "source": [
    "#preprocessing for the amplitudes:\n",
    "\n",
    "def prepro_ampl(data):\n",
    "\n",
    "    for r in (('}', '} '),('{', '{'), (' + ',' + ' ), (' - ', ' - ') ,('*', '* '), ('(* )', '(*)'),('^', '^') , ('(', '('),(')', ')'),('/', ' /')  ,('  ', ' ') ) :  #,('{', ' {')\n",
    "        data = data.replace(*r) \n",
    "        \n",
    "    return data\n",
    "\n",
    "#preprocessing for the squared amplitudes:\n",
    "\n",
    "def prepro_squared_ampl(data):\n",
    "\n",
    "    for r in (('*', '*'), (',', ' , '), ('*(', ' *(') , ('([', '[ '), ('])', ' ]'), ('[', '[ '), (']', ' ]'), ('[ start ]', '[start]'), ('[ end ]', '[end]') ,('/', ' / ') ,('  ', ' ')) :\n",
    "        data = data.replace(*r) \n",
    "    data = re.sub(r\"(\\s\\d+|\\s-\\d+|\\s\\+\\d+)\\*(s_\\d+\\*s_\\d+)\", r\"\\1* \\2\", data)\n",
    "    data = re.sub(r\"(\\s\\d+|\\s-\\d+|\\s\\+\\d+)\\*(m_\\w+\\^\\d+)\", r\"\\1* \\2 \", data)\n",
    "    data = re.sub(r\"(m_\\w+\\^\\d+)\\(\", r\"\\1 )\", data)\n",
    "    data = data.replace('  ', ' ')\n",
    "    \n",
    "    \n",
    "    return data\n",
    "\n",
    "\n",
    "\n",
    "text_pairs_prep = []\n",
    "for i in range(len(text_pairs)):\n",
    "    text_pairs_prep.append((prepro_ampl(text_pairs[i][0]), prepro_squared_ampl(text_pairs[i][1])))\n",
    "\n",
    "text_pairs = text_pairs_prep"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2d8f3e5",
   "metadata": {},
   "source": [
    "###  Maximum sequence length:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4f38dfea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maximum sequence length of amplitudes        : 180\n",
      "Maximum sequence length of squared amplitudes: 152\n"
     ]
    }
   ],
   "source": [
    "def max_len(sq_data):\n",
    "    l = len(sq_data[sq_data.index(max(sq_data, key=len))].split())\n",
    "    return l\n",
    "\n",
    "\n",
    "ampl = [pair[0] for pair in text_pairs]\n",
    "sq_ampl= [pair[1] for pair in text_pairs]\n",
    "\n",
    "print( 'Maximum sequence length of amplitudes        :' ,max_len(ampl))\n",
    "print( 'Maximum sequence length of squared amplitudes:' ,max_len(sq_ampl))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b42fb7da",
   "metadata": {},
   "source": [
    "\n",
    "# Tokenization:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bf843f6",
   "metadata": {},
   "source": [
    "### Split the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ffe743a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "num_val_samples = int(0.15 * len(text_pairs))\n",
    "num_train_samples = len(text_pairs) - 2 * num_val_samples\n",
    "train_pairs = text_pairs[:num_train_samples]\n",
    "val_pairs  = text_pairs[num_train_samples : num_train_samples + num_val_samples]\n",
    "test_pairs = text_pairs[num_train_samples + num_val_samples :]\n",
    "\n",
    "train_input_texts = [pair[0] for pair in train_pairs]\n",
    "train_output_texts = [pair[1] for pair in train_pairs]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f1a8ccd",
   "metadata": {},
   "source": [
    "### Tokeniaztion:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c53d1191",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-11-23 12:43:15.237006: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "number of input (amplitude) tokens:   788\n",
      "number of target (squared amplitude) tokens:  1424\n"
     ]
    }
   ],
   "source": [
    "sequence_length = max(max_len(ampl), max_len(sq_ampl))\n",
    "\n",
    "input_vectorization = TextVectorization(\n",
    "    max_tokens=None, output_mode=\"int\", output_sequence_length=sequence_length, standardize=None, )\n",
    "\n",
    "output_vectorization = TextVectorization(\n",
    "    max_tokens=None,\n",
    "    output_mode=\"int\",\n",
    "    output_sequence_length=sequence_length + 1, standardize=None)\n",
    "\n",
    "input_vectorization.adapt(train_input_texts)\n",
    "output_vectorization.adapt(train_output_texts)\n",
    "\n",
    "target_tokens = output_vectorization.get_vocabulary()\n",
    "input_tokens = input_vectorization.get_vocabulary()\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "print('number of input (amplitude) tokens:  ', len(input_tokens))\n",
    "print('number of target (squared amplitude) tokens: ', len(target_tokens))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6ed8d80d",
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 64\n",
    "vocab_size= max(len(input_tokens), len(target_tokens))\n",
    "\n",
    "def format_dataset(input_exp, target_exp):\n",
    "    input_exp = input_vectorization(input_exp)\n",
    "    target_exp = output_vectorization(target_exp)\n",
    "    return ({\"encoder_inputs\": input_exp, \"decoder_inputs\": target_exp[:, :-1],}, target_exp[:, 1:])\n",
    "\n",
    "\n",
    "def make_dataset(pairs):\n",
    "    ampl_texts, sqampl_texts = zip(*pairs)\n",
    "    ampl_texts = list(ampl_texts)\n",
    "    sqampl_texts = list(sqampl_texts)\n",
    "    dataset = tf.data.Dataset.from_tensor_slices((ampl_texts, sqampl_texts))\n",
    "    dataset = dataset.batch(batch_size)\n",
    "    dataset = dataset.map(format_dataset)\n",
    "    return dataset.shuffle(2048).prefetch(16).cache()\n",
    "\n",
    "\n",
    "\n",
    "train_ds = make_dataset(train_pairs)\n",
    "val_ds = make_dataset(val_pairs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f439937",
   "metadata": {},
   "source": [
    "## Transformer Model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "410fbb01",
   "metadata": {},
   "outputs": [],
   "source": [
    "class TransformerEncoder(layers.Layer):\n",
    "    def __init__(self, embed_dim, dense_dim, num_heads, **kwargs):\n",
    "        super(TransformerEncoder, self).__init__(**kwargs)\n",
    "        self.embed_dim = embed_dim\n",
    "        self.dense_dim = dense_dim\n",
    "        self.num_heads = num_heads\n",
    "        self.attention = layers.MultiHeadAttention(\n",
    "            num_heads=num_heads, key_dim=embed_dim\n",
    "        )\n",
    "        self.dense_proj = keras.Sequential(\n",
    "            [layers.Dense(dense_dim, activation=\"relu\"), layers.Dense(embed_dim),]\n",
    "        )\n",
    "        self.layernorm_1 = layers.LayerNormalization()\n",
    "        self.layernorm_2 = layers.LayerNormalization()\n",
    "        self.supports_masking = True\n",
    "\n",
    "    def call(self, inputs, mask=None):\n",
    "        if mask is not None:\n",
    "            padding_mask = tf.cast(mask[:, tf.newaxis, tf.newaxis, :], dtype=\"int64\")\n",
    "        attention_output = self.attention(\n",
    "            query=inputs, value=inputs, key=inputs, attention_mask=padding_mask\n",
    "        )\n",
    "        proj_input = self.layernorm_1(inputs + attention_output)\n",
    "        proj_output = self.dense_proj(proj_input)\n",
    "        return self.layernorm_2(proj_input + proj_output)\n",
    "\n",
    "\n",
    "class PositionalEmbedding(layers.Layer):\n",
    "    def __init__(self, sequence_length, vocab_size, embed_dim, **kwargs):\n",
    "        super(PositionalEmbedding, self).__init__(**kwargs)\n",
    "        self.token_embeddings = layers.Embedding(\n",
    "            input_dim=vocab_size, output_dim=embed_dim\n",
    "        )\n",
    "        self.position_embeddings = layers.Embedding(\n",
    "            input_dim=sequence_length, output_dim=embed_dim\n",
    "        )\n",
    "        self.sequence_length = sequence_length\n",
    "        self.vocab_size = vocab_size\n",
    "        self.embed_dim = embed_dim\n",
    "\n",
    "    def call(self, inputs):\n",
    "        length = tf.shape(inputs)[-1]\n",
    "        positions = tf.range(start=0, limit=length, delta=1)\n",
    "        embedded_tokens = self.token_embeddings(inputs)\n",
    "        embedded_positions = self.position_embeddings(positions)\n",
    "        return embedded_tokens + embedded_positions\n",
    "\n",
    "    def compute_mask(self, inputs, mask=None):\n",
    "        return tf.math.not_equal(inputs, 0)\n",
    "\n",
    "\n",
    "class TransformerDecoder(layers.Layer):\n",
    "    def __init__(self, embed_dim, latent_dim, num_heads, **kwargs):\n",
    "        super(TransformerDecoder, self).__init__(**kwargs)\n",
    "        self.embed_dim = embed_dim\n",
    "        self.latent_dim = latent_dim\n",
    "        self.num_heads = num_heads\n",
    "        self.attention_1 = layers.MultiHeadAttention(\n",
    "            num_heads=num_heads, key_dim=embed_dim\n",
    "        )\n",
    "        self.attention_2 = layers.MultiHeadAttention(\n",
    "            num_heads=num_heads, key_dim=embed_dim\n",
    "        )\n",
    "        self.dense_proj = keras.Sequential(\n",
    "            [layers.Dense(latent_dim, activation=\"relu\"), layers.Dense(embed_dim),]\n",
    "        )\n",
    "        self.layernorm_1 = layers.LayerNormalization()\n",
    "        self.layernorm_2 = layers.LayerNormalization()\n",
    "        self.layernorm_3 = layers.LayerNormalization()\n",
    "        self.supports_masking = True\n",
    "\n",
    "    def call(self, inputs, encoder_outputs, mask=None):\n",
    "        causal_mask = self.get_causal_attention_mask(inputs)\n",
    "        if mask is not None:\n",
    "            padding_mask = tf.cast(mask[:, tf.newaxis, :], dtype=\"int64\")\n",
    "            padding_mask = tf.minimum(padding_mask, causal_mask)\n",
    "\n",
    "        attention_output_1 = self.attention_1(\n",
    "            query=inputs, value=inputs, key=inputs, attention_mask=causal_mask\n",
    "        )\n",
    "        out_1 = self.layernorm_1(inputs + attention_output_1)\n",
    "\n",
    "        attention_output_2 = self.attention_2(\n",
    "            query=out_1,\n",
    "            value=encoder_outputs,\n",
    "            key=encoder_outputs,\n",
    "            attention_mask=padding_mask,\n",
    "        )\n",
    "        out_2 = self.layernorm_2(out_1 + attention_output_2)\n",
    "\n",
    "        proj_output = self.dense_proj(out_2)\n",
    "        return self.layernorm_3(out_2 + proj_output)\n",
    "\n",
    "    def get_causal_attention_mask(self, inputs):\n",
    "        input_shape = tf.shape(inputs)\n",
    "        batch_size, sequence_length = input_shape[0], input_shape[1]\n",
    "        i = tf.range(sequence_length)[:, tf.newaxis]\n",
    "        j = tf.range(sequence_length)\n",
    "        mask = tf.cast(i >= j, dtype=\"int64\")\n",
    "        mask = tf.reshape(mask, (1, input_shape[1], input_shape[1]))\n",
    "        mult = tf.concat(\n",
    "            [tf.expand_dims(batch_size, -1), tf.constant([1, 1], dtype=tf.int32)],\n",
    "            axis=0,\n",
    "        )\n",
    "        return tf.tile(mask, mult)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96d1c4b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "    \n",
    "embed_dim = 256\n",
    "latent_dim = 512\n",
    "num_heads = 8\n",
    "\n",
    "encoder_inputs = keras.Input(shape=(None,), dtype=\"int64\", name=\"encoder_inputs\")\n",
    "x = PositionalEmbedding(sequence_length, vocab_size, embed_dim)(encoder_inputs)\n",
    "encoder_outputs = TransformerEncoder(embed_dim, latent_dim, num_heads)(x)\n",
    "encoder = keras.Model(encoder_inputs, encoder_outputs)\n",
    "\n",
    "decoder_inputs = keras.Input(shape=(None,), dtype=\"int64\", name=\"decoder_inputs\")\n",
    "encoded_seq_inputs = keras.Input(shape=(None, embed_dim), name=\"decoder_state_inputs\")\n",
    "x = PositionalEmbedding(sequence_length, vocab_size, embed_dim)(decoder_inputs)\n",
    "x = TransformerDecoder(embed_dim, latent_dim, num_heads)(x, encoded_seq_inputs)\n",
    "x = layers.Dropout(0.5)(x)\n",
    "decoder_outputs = layers.Dense(vocab_size, activation=\"softmax\")(x)\n",
    "decoder = keras.Model([decoder_inputs, encoded_seq_inputs], decoder_outputs)\n",
    "\n",
    "decoder_outputs = decoder([decoder_inputs, encoder_outputs])\n",
    "transformer = keras.Model(\n",
    "    [encoder_inputs, decoder_inputs], decoder_outputs, name=\"transformer\"\n",
    ")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94c9b0a2",
   "metadata": {},
   "source": [
    "## Training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1e738b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 5  \n",
    "learning_rate=0.001\n",
    "opt = keras.optimizers.Adam(learning_rate=learning_rate)\n",
    "\n",
    "transformer.summary()\n",
    "transformer.compile(loss=\"sparse_categorical_crossentropy\", metrics=[\"accuracy\"], optimizer=opt)\n",
    "transformer.fit(train_ds, epochs=epochs, validation_data=val_ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3375bed",
   "metadata": {},
   "source": [
    "## Inference:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42503048",
   "metadata": {},
   "outputs": [],
   "source": [
    "target_tokens = output_vectorization.get_vocabulary()\n",
    "target_index_lookup = dict(zip(range(len(target_tokens)), target_tokens))\n",
    "max_decoded_sentence_length = max_sequence_length\n",
    "\n",
    "\n",
    "def decode_sequence(input_sentence):\n",
    "    tokenized_input_sentence = input_vectorization([input_sentence])\n",
    "    decoded_sentence = \"[start]\"\n",
    "    for i in range(max_decoded_sentence_length):\n",
    "        tokenized_target_sentence = output_vectorization([decoded_sentence])[:, :-1]\n",
    "        predictions = transformer([tokenized_input_sentence, tokenized_target_sentence])\n",
    "\n",
    "        sampled_token_index = np.argmax(predictions[0, i, :])\n",
    "        sampled_token = target_index_lookup[sampled_token_index]\n",
    "        decoded_sentence += \" \" + sampled_token\n",
    "\n",
    "        if sampled_token == \"[end]\":\n",
    "            break\n",
    "    return decoded_sentence\n",
    "\n",
    "test_input_texts = [pair[0] for pair in test_pairs]\n",
    "test_output_texts = [pair[1] for pair in test_pairs]\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02b105c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in random.sample(range(0,len(test_input_texts)), 5):\n",
    "    input_sentence = test_input_texts[i]\n",
    "    translated = decode_sequence(input_sentence)\n",
    "    print('Actual:    ', test_output_texts[i], '\\n')\n",
    "    print('Predicted: ', translated, '\\n')\n",
    "    print(test_output_texts[i]==translated, '\\n')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}