Consolidate Retrieval/Reranking/Instruction Variants (#1359)

* update * merged retrieval; working * update tasks; working multilingual * everything working except instructions * working instructions; just need cleanup * add metadata for all but MindSmall * faster evaluation; mindsmall can compute in reasonable time * fix bad merge of docs * lint * fix test * qa * updated mindsmall * lint * fix debug * Update mteb/abstasks/dataloaders.py Co-authored-by: Roman Solomatin <[email protected]> * lint --------- Co-authored-by: Roman Solomatin <[email protected]>
embeddings-benchmark · Nov 13, 2024 · 9c58518 · 9c58518
1 parent e2520df
commit 9c58518
Show file tree

Hide file tree

Showing 54 changed files with 2,500 additions and 2,655 deletions.
diff --git a/README.md b/README.md
@@ -81,6 +81,7 @@ In prompts the key can be:
    8. `STS`
    9. `Summarization`
    10. `InstructionRetrieval`
+   11. `InstructionReranking`
 3. Pair of task type and prompt type like `Retrival-query` - these prompts will be used in all classification tasks
 4. Task name - these prompts will be used in the specific task
 5. Pair of task name and prompt type like `NFCorpus-query` - these prompts will be used in the specific task