Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support customized models path #140

Open
3 tasks done
kerol123 opened this issue Aug 23, 2024 · 2 comments
Open
3 tasks done

[Feature] Support customized models path #140

kerol123 opened this issue Aug 23, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@kerol123
Copy link

确认清单

  • 我已经仔细阅读项目文档,确认现有功能无法解决我的需求
  • 我已经检索过现有issue,确认与现有issue的内容并不重复
  • 确认并非问题讨论而是 Feature request

功能描述

I used common models for different projects, but I don't want to copy these models to each projects cuz some model weight files consume large disk space. We can config model path in environments.

可能的解决方案

When we load model from path, we can firstly read from relative environment variables, if not found, we use default ./models/ path.
For ChatTTS example in modules/core/models/zoo/ChatTTS.py

# before in line 32
custom_path="./models/ChatTTS",
# after
custom_path=os.getenv("CHATTTS_MODEL_PATH", "./models/ChatTTS"),

we can define environment variables in .env.global just like .env.api or .env.webui

@kerol123 kerol123 added the enhancement New feature or request label Aug 23, 2024
@kerol123
Copy link
Author

I can help contribute to this feature, also I'd like to support other models like SenseVoice in the story.

@zhzLuke96
Copy link
Member

thx!

Indeed, a "custom model dir" feature is necessary and worth developing.

Regarding the "custom model dir" functionality, I'd like to add some context about the current code logic:

  1. Currently, almost all environment variables correspond one-to-one with runtime parameters (argv). The specific implementation and logic can be found in modules/utils/env.py.
  2. The implementation of runtime parameters is split into setup_xx and process_xx. This separation facilitates different levels of execution scripts. For example, uvicorn parameters won't be parsed in the webui.
  3. After parsing, both environment variables and runtime parameters are mounted to config.runtime_env_vars[key] (This might be a bit confusing as the runtime_env_vars object doesn't have any typed attributes).

In short, this feature shouldn't be too difficult to implement. Feel free to ask if you have any questions.

Regarding the SenseVoice model adaptation, I suggest opening a new issue to track and discuss it, as it might not be straightforward. For ASR models, we currently support the Whisper model, but we may not have considered issues like multiple models, model switching, or API parameters. Therefore, there might be matters unrelated to SenseVoice that need to be considered alongside it.

thx again~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants