Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixing absence of vqa model #150

Merged
merged 2 commits into from
Sep 26, 2023
Merged

fixing absence of vqa model #150

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
11c4358
fixing absence of vqa model
piterand Sep 25, 2023
f5fae00
fixed notebook
piterand Sep 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ammico/notebooks/image_summary.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Here you can choose between two models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
"Here you can choose between models: `\"base\"` or `\"large\"`. This will generate the caption for each image and directly put the results in your dictionary `mydict`. Then you can transform it into the dataframe and this dataframe can be exported as a .csv file.\n",
"\n",
"The results are written in the columns: \n",
"- `const_image_summary` - the permanent summaries, which do not change from run to run (analyse_image).\n",
Expand Down Expand Up @@ -201,7 +201,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Set the list of questions as a list of strings `list_of_questions`, load the models to the memory and pass them to the function"
"Load the model to the memory through object creation with parameters `analysis_type=\"questions\"` and `model_type=\"vqa\"`. Set the list of questions as a list of strings `list_of_questions`, and pass them to the function"
]
},
{
Expand All @@ -223,7 +223,7 @@
"metadata": {},
"outputs": [],
"source": [
"obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", list_of_questions = list_of_questions)"
"obj = ammico.SummaryDetector(mydict, analysis_type=\"questions\", model_type=\"vqa\")"
]
},
{
Expand Down
3 changes: 2 additions & 1 deletion ammico/summary.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ class SummaryDetector(AnalysisMethod):
allowed_model_types = [
"base",
"large",
"vqa",
]
allowed_new_model_types = [
"blip2_t5_pretrain_flant5xxl",
Expand Down Expand Up @@ -44,7 +45,7 @@ def __init__(
Args:
subdict (dict, optional): Dictionary containing the image to be analysed. Defaults to {}.

model_type (str, optional): Type of model to use. Can be "base" or "large" for blip_caption. Or can be one of the new models:
model_type (str, optional): Type of model to use. Can be "base" or "large" or "vqa" for blip_caption and VQA. Or can be one of the new models:
"blip2_t5_pretrain_flant5xxl",
"blip2_t5_pretrain_flant5xl",
"blip2_t5_caption_coco_flant5xl",
Expand Down