Skip to content

Commit

Permalink
Update tasks table
Browse files Browse the repository at this point in the history
  • Loading branch information
github-actions[bot] committed Jun 12, 2024
1 parent d771437 commit 30433a9
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions docs/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,7 @@ The following tables give you an overview of the tasks in MTEB.
| [VieStudentFeedbackClassification](https://ieeexplore.ieee.org/document/8573337) (Nguyen et al., 2018) | ['vie'] | Classification | s2s | [Reviews] | {'test': 2048} | {'test': 14.22} |
| [WRIMEClassification](https://aclanthology.org/2021.naacl-main.169/) | ['jpn'] | Classification | s2s | [Social] | {'test': 2048} | {'test': 47.78} |
| [Waimai](https://aclanthology.org/2023.nodalida-1.20/) | ['cmn'] | Classification | s2s | | | |
| [WebLINXCandidatesReranking](https://mcgill-nlp.github.io/weblinx) (Xing Han Lù, 2024) | ['eng'] | Reranking | p2p | [Academic, Web] | {'validation': 1301, 'test_iid': 1438, 'test_cat': 3560, 'test_web': 3144, 'test_vis': 5298, 'test_geo': 4916} | {'validation': 1647.52, 'test_iid': 1722.63, 'test_cat': 2149.66, 'test_web': 1831.46, 'test_vis': 1737.26, 'test_geo': 1742.66} |
| [WikiCitiesClustering](https://huggingface.co/datasets/wikipedia) | ['eng'] | Clustering | p2p | | | |
| [WikiClusteringP2P.v2](https://github.com/Rysias/wiki-clustering) | ['bos', 'cat', 'ces', 'dan', 'eus', 'glv', 'ilo', 'kur', 'lav', 'min', 'mlt', 'sco', 'sqi', 'wln'] | Clustering | p2p | [Encyclopaedic] | {'test': 2048} | {'test': 625.3} |
| [WikipediaRerankingMultilingual](https://huggingface.co/datasets/ellamind/wikipedia-2023-11-reranking-multilingual) | ['ben', 'bul', 'ces', 'dan', 'deu', 'eng', 'fas', 'fin', 'hin', 'ita', 'nld', 'nor', 'por', 'ron', 'srp', 'swe'] | Reranking | s2p | [Encyclopaedic] | {'en': 1500, 'de': 1500, 'it': 1500, 'pt': 1500, 'nl': 1500, 'cs': 1500, 'ro': 1500, 'bg': 1500, 'sr': 1500, 'fi': 1500, 'da': 1500, 'fa': 1500, 'hi': 1500, 'bn': 1500, 'no': 1500, 'sv': 1500} | {'test': 452.0} |
Expand Down Expand Up @@ -814,7 +815,7 @@ The following tables give you an overview of the tasks in MTEB.
| ell | 3 | 7 | 0 | 0 | 1 | 2 | 0 | 3 | 0 | 0 |
| emi | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| emp | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| eng | 12 | 144 | 15 | 3 | 1 | 8 | 6 | 54 | 12 | 1 |
| eng | 12 | 144 | 15 | 3 | 1 | 8 | 7 | 54 | 12 | 1 |
| enq | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| epo | 3 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| eri | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Expand Down Expand Up @@ -1583,7 +1584,7 @@ The following tables give you an overview of the tasks in MTEB.
| zty | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| zul | 2 | 4 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
| zyp | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Total | 1361 | 970 | 107 | 3 | 28 | 67 | 45 | 335 | 73 | 2 |
| Total | 1361 | 970 | 107 | 3 | 28 | 67 | 46 | 335 | 73 | 2 |
<!-- TASK LANG TABLE END -->

</details>

0 comments on commit 30433a9

Please sign in to comment.