fixing the absence of a vqa model (#150)

* fixing the absence of a VQA model
ssciwr · Sep 26, 2023 · 32d1321 · 32d1321
1 parent a57e67b
commit 32d1321
Show file tree

Hide file tree

Showing 2 changed files with 5 additions and 4 deletions.
diff --git a/ammico/notebooks/image_summary.ipynb b/ammico/notebooks/image_summary.ipynb
@@ -84,7 +84,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Here you can choose between two models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
+    "Here you can choose between models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
     "\n",
     "The results are written in the columns: \n",
     "- `const_image_summary` - the permanent summaries, which do not change from run to run (analyse_image).\n",
@@ -201,7 +201,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Set the list of questions as a list of strings `list_of_questions`, load the models to the memory and pass them to the function"
+    "Load the model to the memory through object creation with parameters `analysis_type=\"questions\"` and `model_type=\"vqa\"`.  Set the list of questions as a list of strings `list_of_questions`,   and pass them to the function"
    ]
   },
   {
@@ -223,7 +223,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", list_of_questions = list_of_questions)"
+    "obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\",  model_type=\"vqa\")"
    ]
   },
   {

diff --git a/ammico/summary.py b/ammico/summary.py
@@ -9,6 +9,7 @@ class SummaryDetector(AnalysisMethod):
     allowed_model_types = [
         "base",
         "large",
+        "vqa",
     ]
     allowed_new_model_types = [
         "blip2_t5_pretrain_flant5xxl",
@@ -44,7 +45,7 @@ def __init__(
         Args:
             subdict (dict, optional): Dictionary containing the image to be analysed. Defaults to {}.
 
-            model_type (str, optional): Type of model to use. Can be "base" or "large" for blip_caption. Or can be one of the new models:
+            model_type (str, optional): Type of model to use. Can be "base" or "large" or "vqa" for blip_caption and VQA. Or can be one of the new models:
                 "blip2_t5_pretrain_flant5xxl",
                 "blip2_t5_pretrain_flant5xl",
                 "blip2_t5_caption_coco_flant5xl",