Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The training loss and perplexity is not decreasing for the VQA task #78

Open
henryj18 opened this issue Feb 5, 2024 · 0 comments
Open

Comments

@henryj18
Copy link
Collaborator

henryj18 commented Feb 5, 2024

Refer to #30 about the PR to add caption and vqa task.

Refer to #77 about the PR to merge the code for caption task and vqa task (all of the work was completed on add_vqa branch) to master

Now the merge has completed, we submit this issue to keep track of multiple issues that need to be investigated/resolved

Issue 1: the training loss and perplexity is not decreasing for the VQA task

For the vqa task, the training loss and perplexity is not decreasing with the tests we have done up to about 14K images and their associated VQA. The test was done on Colab and we could not scale up for more images due to Colab I/O error to handle too many files in a folder.

We need to investigate this issue from at least two avenues:

  1. Scale up more training data on a hardware where I/O is not causing error for more images in a folder. This is for purpose to see whether the non-decreasing perplexity is due to training dataset is too small
  2. Inspect the algorithm for VQA and see whether it should be revised

Issue 2: need to implement a evaluate for caption task and vqa task in eval.py

Any other issue will be entered here as they show up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant