Resources

Publicly available training logbooks

The listing is in no particular order other than the year.

2021

BigScience pre-BLOOM 108B training experiments (2021): chronicles | the full spec and discussions

2022

BigScience BLOOM-176B (2022): chronicles-prequel | chronicles | the full spec and discussions
Meta OPT-175B (2022): logbook | Video
THUDM GLM-130B (2022): en logbook | Mandarin version

2023

HuggingFace IDEFICS-80B multimodal (Flamingo repro) (2023): Learning log | Training Chronicles
BloombergGPT 50B LLM - section C in BloombergGPT: A Large Language Model for Finance