Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

llm server for lm-eval. #82

Merged
merged 17 commits into from
May 30, 2024
Merged

llm server for lm-eval. #82

merged 17 commits into from
May 30, 2024

Conversation

lkk12014402
Copy link
Collaborator

Description

init model with lm-eval https://github.com/opea-project/GenAIEval, which can support multi devices, and start a server with fastapi.

@lkk12014402
Copy link
Collaborator Author

lkk12014402 commented May 22, 2024 •
edited
Loading

usage

export MODEL="hf"
export MODEL_ARGS="pretrained=Intel/neural-chat-7b-v3-3"
export DEVICE="cpu"
python self_hosted_hf.py

corresponds to this pr

@lkk12014402 lkk12014402 mentioned this pull request May 22, 2024
Merged
@chensuyue chensuyue added this to the v0.6 milestone May 29, 2024
@lkk12014402 lkk12014402 force-pushed the enable_lm_eval_with_microservice branch from e32eebc to a3511b0 Compare May 30, 2024 04:30
lkk12014402 and others added 14 commits May 30, 2024 13:56
@lkk12014402
add requirements.
61ec124
@lkk12014402
add dockerfile and readme.
f847357
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
7bc8ac0 
for more information, see https://pre-commit.ci
@lkk12014402
update requirements.
428c355
@lkk12014402
update requirements.
efeece1
@lkk12014402
update requirements.
9da1f38
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
8257ae6 
for more information, see https://pre-commit.ci
@lkk12014402
update requirements.
bc08575
@lkk12014402
Merge branch 'enable_lm_eval_with_microservice' of https://github.com…
5f44c50 
…/lkk12014402/GenAIComps into enable_lm_eval_with_microservice
@lkk12014402
update requirements.
ad18ed7
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
34f7219 
for more information, see https://pre-commit.ci
@lkk12014402
update requirements.
023ef04
@lkk12014402
Merge branch 'enable_lm_eval_with_microservice' of https://github.com…
302d0d5 
…/lkk12014402/GenAIComps into enable_lm_eval_with_microservice
@lkk12014402
update requirements.
ec88862
@chensuyue chensuyue merged commit ed311f5 into opea-project:main May 30, 2024
6 checks passed
ganesanintel pushed a commit to ganesanintel/GenAIComps that referenced this pull request Jun 3, 2024
@lkk12014402 @ganesanintel
llm server for lm-eval (opea-project#82)
0fb1b51 
Signed-off-by: V, Ganesan <[email protected]>
lkk12014402 pushed a commit that referenced this pull request Aug 8, 2024
@Spycsh @chensuyue
Add AudioQnA example (#82)
cce8283 
Co-authored-by: chensuyue <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lvliang-intel lvliang-intel lvliang-intel approved these changes

@chensuyue chensuyue chensuyue approved these changes

@hshen14 hshen14 Awaiting requested review from hshen14

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v0.6
Development

Successfully merging this pull request may close these issues.
3 participants
@lkk12014402 @chensuyue @lvliang-intel