Code

Yaml crowspairs tasks#2488

Code

Open

NAM00 wants to merge 24 commits intoEleutherAI:mainEleutherAI/lm-evaluation-harness:mainfrom OpenGPTX:yaml_crowspairs_tasksOpenGPTX/lm-evaluation-harness:yaml_crowspairs_tasksCopy head branch name to clipboard

+19,768-26

Commits

Commits on Mar 13, 2024

• Tokenizer handling

  KlaudiaTH

  committed
  3393c84

• Added ogx gsm8k implementation

  KlaudiaTH

  committed
  47c3c4a

• Added multilingual arc, hellaswag, mmlu and truthfulqa from EleutherAI upstream

  KlaudiaTH

  committed
  cf9e190

• Added ogx hellaswagx task

  KlaudiaTH

  committed
  8fa981d

• Added ogx arcx tasks

  KlaudiaTH

  committed
  25f0b25

• Task fixes

  KlaudiaTH

  committed
  93f549c

• added truthfulqax yaml configs

  jjbuschhoff

  committed
  f96e585

• Task fixes

  KlaudiaTH

  committed
  2fcfe02

Commits on Mar 15, 2024

• added mmlux task configs

  jjbuschhoff

  committed
  bcb4104

• Merge branch 'yaml_multilingual_tasks' of https://github.com/OpenGPTX/lm-evaluation-harness into yaml_multilingual_tasks

  jjbuschhoff

  committed
  e8d9f38

Commits on Mar 18, 2024

• Task corrections: MMLU and GSM8k

  KlaudiaTH

  committed
  521504e

• added generation templates

  jjbuschhoff

  committed
  3a0035c

• Merge branch 'yaml_multilingual_tasks' of https://github.com/OpenGPTX/lm-evaluation-harness into yaml_multilingual_tasks

  jjbuschhoff

  committed
  de88d6b

Commits on Mar 22, 2024

• Corrected task names ogx_thruthfulqa_* to ogx_truthfulqa_*

  KlaudiaTH

  committed
  4900eff

• Tasks corrections: MMLU and GSM8k

  KlaudiaTH

  committed
  4fe393c

Commits on Apr 15, 2024

• Added --bootstrap_iters command line argument

  KlaudiaTH

  committed
  f6663f2

• Preliminary implementation of flores and belebele

  KlaudiaTH

  committed
  28bf93f

• Added flores nll fix

  KlaudiaTH

  committed
  f8e00f7

Commits on Apr 17, 2024

• Added missing nll metric implementation

  KlaudiaTH

  committed
  7447360

Commits on Jun 28, 2024

• added latvian flores200 configs

  jjbuschhoff

  committed
  485acdd

• added lvs config for belebele, also renamed config files

  jjbuschhoff

  committed
  6092ac5

Commits on Jul 6, 2024

• FLORES-200: Added missing Latvian and removed unnecessary Irish, Maltese and Croatian tasks

  KlaudiaTH

  committed
  6b544a9

Commits on Jul 10, 2024

• HF model: Added 'nccl_timeout' model arg

  KlaudiaTH

  committed
  6cc3360

Commits on Oct 30, 2024

• Crowspairs task yaml

  NAM00

  committed
  1bd5e6a