Keeping track of issues filed at LLM+RL projects

[mdingemanse]

This issue is just a place to keep track of issues filed at other github projects or HuggingFace hubs to ask for documentation

[mdingemanse]

Mistral: mistralai/mistral-inference#9

[mdingemanse]

Xwin-LM: Xwin-LM/Xwin-LM#14

[mdingemanse]

RedPajama: https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Instruct/discussions/9#64c7c30a527d763655434618

Now has an answer:

The instruction-tuned model was trained on instructions from the P3 and Natural Instructions datasets, which were decontaminated against HELM (you can find more details about the decontamination strategy in this blog bost).

You can find the resulting decontaminated dataset used to train the instruction tuned model here: https://huggingface.co/datasets/togethercomputer/RedPajama-Data-Instruct -- the metadata here contains a source field pointing to the task / dataset where the instance comes from.

[mdingemanse]

Solar 70B: https://huggingface.co/upstage/SOLAR-0-70b-16bit/discussions/15

[mdingemanse]

Boomer 1B: https://huggingface.co/budecosystem/boomer-1b/discussions/1#652be6c22aa5b27c77eab9c7

[mdingemanse]

GenZ-70B: https://huggingface.co/budecosystem/genz-70b/discussions/1#6533ffd624c306369652e7d1

[mdingemanse]

Erasmian Language Model (ELM): Joaoffg/ELM#1 (and see #65)

[mdingemanse]

Yi 34B Chat: https://huggingface.co/01-ai/Yi-34B-Chat/discussions/7

[mdingemanse]

Gemma: https://huggingface.co/google/gemma-7b-it/discussions/64

[mdingemanse]

Command R+: https://huggingface.co/CohereForAI/c4ai-command-r-v01/discussions/43#661fb1d3a3ce1d628c28aaa6

[mdingemanse]

Nanbeige Chat: https://huggingface.co/Nanbeige/Nanbeige2-8B-Chat/discussions/2#6621e15a4d17641cf788cbd5

Update: request for training data was closed without comment. https://huggingface.co/Nanbeige/Nanbeige2-8B-Chat/discussions/1

Update: request for post-training data was answered and closed