Code

Added VQA as a evaluator. #52

Code

Open

ruhanaazam wants to merge 16 commits intoEmergenceAI:devEmergenceAI/Agent-E:devfrom ruhanaazam:devruhanaazam/Agent-E:devCopy head branch name to clipboard

Commits

Commits on Jun 19, 2024

• Added VQA evaluator. Uses sequence of screenshots to evaluate a given task. VQA evaluator must be used with the flag --take_screenshots true

  ruhanaazam

  committed
  7cfde2a

  Copy full SHA for 7cfde2a

• Merge branch 'dev' of https://github.com/EmergenceAI/Agent-E into dev

  ruhanaazam

  committed
  e1e54a4

  Copy full SHA for e1e54a4

Commits on Jun 20, 2024

• Fixed VQA evaluator bugs -- wrong path to screenshots and missing import.

  ruhanaazam

  committed
  2ac8565

  Copy full SHA for 2ac8565

Commits on Jul 11, 2024

• Added the vqa model's reasoning to the evaluator's output.

  ruhanaazam

  committed
  ddd999b

  Copy full SHA for ddd999b

• Added inline documentation to functions list_items_in_folder() and compress_png()

  ruhanaazam

  committed
  bcbfe66

  Copy full SHA for bcbfe66

• Used path from ae.config file to get screenshots.

  ruhanaazam

  committed
  1d710c9

  Copy full SHA for 1d710c9

• Added the validation_agent package. This contains required files for doing a vqa evaluation.

  ruhanaazam

  committed
  bd53e53

  Copy full SHA for bd53e53

• Merge branch 'dev' of https://github.com/EmergenceAI/Agent-E into dev

  ruhanaazam

  committed
  6a1045d

  Copy full SHA for 6a1045d

Commits on Nov 6, 2024

• Merge remote-tracking branch 'upstream/dev' into dev

  ruhanaazam

  committed
  e624030

  Copy full SHA for e624030

• Resolve ruff linter complains

  ruhanaazam

  committed
  2088836

  Copy full SHA for 2088836

• More ruff linter formatting

  ruhanaazam

  committed
  de9dd05

  Copy full SHA for de9dd05

• Added warning for when screenshot folder is empty. This evaluation is treated like a .

  ruhanaazam

  committed
  fe5519c

  Copy full SHA for fe5519c

• Added comment to evaluators.py

  ruhanaazam

  committed
  3136f57

  Copy full SHA for 3136f57

• Updated readme to include how to do automatic/custom evaluation

  ruhanaazam

  committed
  589290c

  Copy full SHA for 589290c

• Removes old snapshots when tests are rerun with the same name.

  ruhanaazam

  committed
  93296ce

  Copy full SHA for 93296ce

• Better exception handling went vqa responses cannot be parsed. Evaluations here are and error is provided as the reasoning.

  ruhanaazam

  committed
  d6a13e9

  Copy full SHA for d6a13e9