Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unload model does not close cuda process #69

Open
spaghiajoeojo opened this issue Oct 2, 2024 · 1 comment
Open

Unload model does not close cuda process #69

spaghiajoeojo opened this issue Oct 2, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@spaghiajoeojo
Copy link

Using --unload-timer 300 and --preload-model xtts_v2.0.2 I can see the model correctly loaded in memory through nvidia-smi.

After 5 minutes (300 seconds) GPU memory drops almost by 95% (from 1938MiB to 104MiB) but the python process remains active not allowing the GPU to enter idle state drawing much more power from PSU.

@matatonic
Copy link
Owner

I'm aware of this issue, the fix requires an architecture change to use a worker process to load/use the model, I'm working on the architecture change for this already.

@matatonic matatonic self-assigned this Oct 5, 2024
@matatonic matatonic added the enhancement New feature or request label Oct 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants