Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

efficiency-benchmark via efficiency pentathlon and compression for OLMO #238

Merged
merged 50 commits into from
Sep 18, 2023
Merged

efficiency-benchmark via efficiency pentathlon and compression for OLMO #238

merged 50 commits into from
Sep 18, 2023

Conversation

pranjali97
Copy link

@pranjali97 pranjali97 commented Aug 2, 2023 •
edited
Loading

Efficiency-pantathlon

  • Added functionality to use OLMO 1B tokenizer to generate token level throughput metrics in efficiency_benchmark/steps.py (function: tabulate_efficiency_metrics, process)

AutoGPTQ

  • Added and integrated the model class for OLMO family of models

@pranjali97 pranjali97 marked this pull request as ready for review August 14, 2023 20:27
@pranjali97 pranjali97 changed the title PR to track and add changes for efficiency-benchmark via efficiency pentathlon efficiency-benchmark via efficiency pentathlon and compression for OLMO Aug 29, 2023
AkshitaB
AkshitaB reviewed Sep 7, 2023
View reviewed changes
Copy link
Contributor

@AkshitaB AkshitaB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a few suggestions

inference/efficiency/run_efficiency_benchmark.sh Outdated Show resolved Hide resolved
inference/efficiency/run_efficiency_benchmark.sh Outdated Show resolved Hide resolved
inference/efficiency/general_purpose_efficiency_benchmark.py Show resolved Hide resolved
inference/efficiency/general_purpose_efficiency_benchmark.py Show resolved Hide resolved
Pranjali Basmatkar and others added 24 commits September 8, 2023 13:51
refactoring dependencies folder
d25af73
updated requirements.txt
d49c9f2
ef-setup
2b4d720
setup2
5e81743
setup3
ea29806
setup4
16bd074
@AkshitaB
Merge branch 'main' into inference-efficiency-pentathlon
19cae7b
compression files
202da77
efficiency files
d94025b
@pranjali97
Update README.md
168b4f4
@pranjali97
Update steps.py
524690c
@AkshitaB
Merge branch 'main' into inference-efficiency-pentathlon
52026e1
@AkshitaB
Update .flake8
72e12ae
@AkshitaB
Update .flake8
0c2d21c
style and lib sort changes
4c38a2e
Merge branch 'inference-efficiency-pentathlon' of github.com:/allenai…
9da4cce 
…/LLM into inference-efficiency-pentathlon

merging style and readme changes
fixing lint errors
620fa86
lint check
f9c6d7e
@AkshitaB
make mypy happy
3758d8b
@AkshitaB
Merge branch 'main' into inference-efficiency-pentathlon
2d9e70a
fixed style check
6a64c9c
lint please
2b61dd4
Updated readme
35c730f
@AkshitaB
Merge branch 'main' into inference-efficiency-pentathlon
dcdc950
@AkshitaB AkshitaB merged commit 54dbd48 into main Sep 18, 2023
10 checks passed
@AkshitaB AkshitaB deleted the inference-efficiency-pentathlon branch September 18, 2023 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AkshitaB AkshitaB AkshitaB left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pranjali97 @AkshitaB