Cuda support for piper

[thiswillbeyourgithub]

Hi, i saw that in this commit you removed on purpose the cuda support for piper because apparently it didn't work for your setup?

I did a few test and it seemed to worked fine for me and was wondering if you could consider re adding support for it.

For me it was as simple as pip install onnxruntime-gpu then add --piper to the subprocess call. It could stay opt in and need an argument in the voice yaml for example.

What do you think? Am I missing something as to why it should be removed? Is it the size of the image?

Thanks!

[matatonic]

Interesting, yeah it was removed because it didn't work in the docker image. Maybe it's fixed! I'll check into adding onnxruntime-gpu back into the image.

[thiswillbeyourgithub]

No update on that front by any chance?

[matatonic]

not yet.

[thiswillbeyourgithub]

Fyi after taking a look at the cuda issues in the piper repo I think it would be a much better bet for low end users to try to setup the fish model with quantization ( #58 ).

Btw just found out about your other opened ai projects. Really awesome stuff thank you very much! Do you accept donations by any chance? I wanted to offer you a few beers coffees in advance while showing what feature would help me the most (in that case fisher support). Kinda like a bounty but with no strings attached of course. Have a nice day!

[matatonic]

I probably wont setup any tip thing, but thanks for the kind words. If you're ever Toronto, I'd happily accept the drinks in person :-)

I know I'm lagging a lot updating speech, but I expect to get around to it again soon!

[thetznecker]

@thiswillbeyourgithub did you see any performance gains while using gpu? CPU generation takes me around 0.5-0.7 seconds on a i9-14900kf, while gpu takes a couple seconds on a 3090.

[thiswillbeyourgithub]

I probably wont setup any tip thing, but thanks for the kind words. If you're ever Toronto, I'd happily accept the drinks in person :-)

Nice! Unfortunately I'm in thr capital of Baguette land so if you come by your drink is on me :)

@thiswillbeyourgithub did you see any performance gains while using gpu? CPU generation takes me around 0.5-0.7 seconds on a i9-14900kf, while gpu takes a couple seconds on a 3090.

I have not succedded in getting cuda to work on my piper setup in less than 20 minutes and it was about the time I had on my hand so can't help sorry.