-
Notifications
You must be signed in to change notification settings - Fork 175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix hugging face prompt pipeline driver + docs #796
Conversation
8da45a1
to
8dba0d4
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅ 📢 Thoughts on this report? Let us know! |
@@ -123,6 +125,7 @@ all = [ | |||
"markdownify", | |||
"voyageai", | |||
"elevenlabs", | |||
"torch", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you explain a bit why we need this now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without this, running the example provided in the docs, I get:
/Users/dylan/Documents/boom/.venv/lib/python3.12/site-packages/stringcase.py:247: SyntaxWarning: invalid escape sequence '\W'
return re.sub("\W+", "", string)
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
[05/21/24 19:26:43] INFO PromptTask ed3273d35309405e87eaacc7bb3360ee
Input: Hello Girafatron, what is your favorite animal?
WARNING:root:<RetryCallState 6048187696: attempt #1; slept for 0.0; last result: failed (RuntimeError At least one of TensorFlow 2.0 or PyTorch should be installed. To install TensorFlow 2.0, read the instructions at https://www.tensorflow.org/install/ To install PyTorch, read the instructions at https://pytorch.org/.)>
WARNING:root:<RetryCallState 6048187696: attempt #2; slept for 2.0; last result: failed (RuntimeError At least one of TensorFlow 2.0 or PyTorch should be installed. To install TensorFlow 2.0, read the instructions at https://www.tensorflow.org/install/ To install PyTorch, read the instructions at https://pytorch.org/.)>
WARNING:root:<RetryCallState 6048187696: attempt #3; slept for 4.0; last result: failed (RuntimeError At least one of TensorFlow 2.0 or PyTorch should be installed. To install TensorFlow 2.0, read the instructions at https://www.tensorflow.org/install/ To install PyTorch, read the instructions at https://pytorch.org/.)>
I was surprised to see this as well, and I can't explain why now. Though it does makes sense to need an ML framework for local ML inferencing.
Changes:
Hugging Face Pipeline
prompt driver docs to useHuggingFacePipelinePromptDriver
instead ofHuggingFacePromptDriver
and removeapi_token
since it is not needed byHuggingFacePipelinePromptDriver
.max_tokens
field with a default value toHuggingFacePipelinePromptDriver
in order to get it to work with the example. This is the same default provided inHuggingFacePromptDriver
.drivers-prompt-huggingface-pipeline
, which includes pytorch (torch
) as a dependency to enable running models locally. Note that hugging face allows using pytorch or tensorflow, and I picked pytorch without any reasons in particular.Hugging Face Pipeline
example's model choice toTinyLlama/TinyLlama-1.1B-Chat-v0.6
so that is small enough to run in the github action test runner as a part of the docs integration tests. (Plus it's nicer for users who want to test out the functionality faster locally)HuggingFacePromptDriver
.Manual Testing:
Hugging Face Pipeline
prompt driver example locally.📚 Documentation preview 📚: https://griptape--796.org.readthedocs.build//796/