Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch from hub #1160

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

Fetch from hub #1160

wants to merge 25 commits into from

Conversation

mikeshi80
Copy link
Contributor

Add the function that user can register custom the model from model hub (HuggingFace, ModelScope) by selecting model type Hub, and input the model Id, and user can import basic info like context length, model size, quantization by click "Import Model" button.

The model will be download from model hub at first running.

TODO: the cached model will not show (cached) tag in model card in "Launch Model" page.

在自定义模型时加入了从Model Hub下载的功能,只要选择Model Source是Hub,并从HuggingFace或ModelScope中选择一个,然后指定Model Id就能注册新模型,并且可以通过点击"Import Model"从Hub中导入诸如context length, model size和量化信息。

注册后的模型会在第一次启动时从Model Hub下载。

TODO: 在"Launch Model"页面的模型卡片中,自定义模型在模型下载后不会显示cached标签。

@XprobeBot XprobeBot added this to the v0.9.4 milestone Mar 19, 2024
@ChengjieLi28
Copy link
Contributor

Could you please paste some screenshots to show your PR effect?

@mikeshi80
Copy link
Contributor Author

Add the "Model Source" option in "Register Model" Page.

"Self Hosted" is same as the original design, "Hub" is the new function for registering new model by model hub and model id directly.

image

If the model format is "GGML" or "GGUF", the extra fields "Model File Name Template" and "Model File Name Split Template" will show out. If "Model File Name Split Template" is not empty, the extra "Quantization Parts" field shows.

image

When the correct model id and model hub are provided, click "IMPORT MODEL" button, it will fetch the model info and fill the corresponding fields, like "Context Length", "Model Size in Billions", "Model File Name Template", "Model File Name Split Template", "Quantization" and "Quantization Parts", you can modify them if there is anything wrong.

image

When the other options are selected correctly, the model can be registered like a built-in model, you do not download it until the first running.

@XprobeBot XprobeBot modified the milestones: v0.9.4, v0.9.5 Mar 21, 2024
@XprobeBot XprobeBot modified the milestones: v0.10.0, v0.10.1 Mar 29, 2024
@XprobeBot XprobeBot modified the milestones: v0.10.2, v0.10.3, v0.11.0 Apr 19, 2024
@XprobeBot XprobeBot modified the milestones: v0.11.0, v0.11.1, v0.11.2 May 11, 2024
@XprobeBot XprobeBot modified the milestones: v0.11.2, v0.11.3 May 24, 2024
@XprobeBot XprobeBot modified the milestones: v0.11.3, v0.11.4, v0.12.0, v0.12.1 May 31, 2024
@XprobeBot XprobeBot modified the milestones: v0.12.1, v0.12.2 Jun 14, 2024
@XprobeBot XprobeBot modified the milestones: v0.12.2, v0.12.4, v0.13.0, v0.13.1 Jun 28, 2024
@XprobeBot XprobeBot modified the milestones: v0.13.1, v0.13.2 Jul 12, 2024
@XprobeBot XprobeBot modified the milestones: v0.13.2, v0.13.4 Jul 26, 2024
@XprobeBot XprobeBot modified the milestones: v0.14, v0.15 Sep 3, 2024
@XprobeBot XprobeBot modified the milestones: v0.15, v0.16 Oct 30, 2024
@XprobeBot XprobeBot modified the milestones: v0.16, v1.x Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants