Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model] Add OLMo November 2024 model #10503

Merged
merged 14 commits into from
Nov 25, 2024
+611 −2
Merged

[Model] Add OLMo November 2024 model #10503

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
67c6011
Add OLMo November release implementation
2015aroras Nov 20, 2024
e073590
Add OLMo model to registry
2015aroras Nov 20, 2024
75bc973
Add weight loading
2015aroras Nov 20, 2024
ab57eb0
Run formatter
2015aroras Nov 20, 2024
ffe121d
Update tests
2015aroras Nov 20, 2024
1d3c611
Update docs
2015aroras Nov 20, 2024
b8a47a8
Update comments
2015aroras Nov 20, 2024
65d62c9
Change module prefixes from Olmo to Olmo1124
2015aroras Nov 20, 2024
b941a24
Run formatter
2015aroras Nov 20, 2024
19e26b7
Pass Olmo1124Attention prefix to Attention
2015aroras Nov 24, 2024
b717887
Rename Olmo1124 to Olmo2
2015aroras Nov 25, 2024
17893b4
Update OLMo2 documentation
2015aroras Nov 25, 2024
b50a3bf
Add Olmo2Config implementation
2015aroras Nov 25, 2024
a398635
Run formatter
2015aroras Nov 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/source/models/supported_models.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,11 @@ Text Generation
- :code:`allenai/OLMo-1B-hf`, :code:`allenai/OLMo-7B-hf`, etc.
-
- ✅︎
* - :code:`OLMo2ForCausalLM`
- OLMo2
- :code:`allenai/OLMo2-7B-1124`, etc.
-
- ✅︎
* - :code:`OLMoEForCausalLM`
- OLMoE
- :code:`allenai/OLMoE-1B-7B-0924`, :code:`allenai/OLMoE-1B-7B-0924-Instruct`, etc.
Expand Down
1 change: 1 addition & 0 deletions tests/distributed/test_pipeline_parallel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ def iter_params(self, model_name: str):
"mosaicml/mpt-7b": PPTestSettings.fast(),
"nvidia/Minitron-8B-Base": PPTestSettings.fast(),
"allenai/OLMo-1B-hf": PPTestSettings.fast(),
"shanearora/OLMo-7B-1124-hf": PPTestSettings.fast(),
"allenai/OLMoE-1B-7B-0924-Instruct": PPTestSettings.fast(),
"facebook/opt-iml-max-1.3b": PPTestSettings.fast(),
"OrionStarAI/Orion-14B-Chat": PPTestSettings.fast(trust_remote_code=True),
Expand Down
1 change: 1 addition & 0 deletions tests/models/registry.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ class _HfExamplesInfo:
"MPTForCausalLM": _HfExamplesInfo("mosaicml/mpt-7b"),
"NemotronForCausalLM": _HfExamplesInfo("nvidia/Minitron-8B-Base"),
"OlmoForCausalLM": _HfExamplesInfo("allenai/OLMo-1B-hf"),
"Olmo2ForCausalLM": _HfExamplesInfo("shanearora/OLMo-7B-1124-hf"),
"OlmoeForCausalLM": _HfExamplesInfo("allenai/OLMoE-1B-7B-0924-Instruct"),
"OPTForCausalLM": _HfExamplesInfo("facebook/opt-iml-max-1.3b"),
"OrionForCausalLM": _HfExamplesInfo("OrionStarAI/Orion-14B-Chat",
Expand Down
Loading