an beginner for TGI, for to trigger the op which is written by triton, use which model-id, thanks #2759
Unanswered
alanguo1234
asked this question in
Q&A
an beginner for TGI, for to trigger the op which is written by triton, use which model-id, thanks
#2759
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi
I have installed the TGI based on Nivida RTX4090, if hope to use text-generation-launcher to trigger the op which is written by triton(such as server/text_generation_server/layers/attention/flash_attn_triton.py), pls tell me the whole command, thanks .
Beta Was this translation helpful? Give feedback.
All reactions