Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix hugging face prompt pipeline driver + docs #796

Merged
merged 2 commits into from
May 22, 2024

Conversation

dylanholmes
Copy link
Contributor

@dylanholmes dylanholmes commented May 21, 2024

Changes:

  • Edit example in Hugging Face Pipeline prompt driver docs to use HuggingFacePipelinePromptDriver instead of HuggingFacePromptDriver and remove api_token since it is not needed by HuggingFacePipelinePromptDriver.
  • Add max_tokens field with a default value to HuggingFacePipelinePromptDriver in order to get it to work with the example. This is the same default provided in HuggingFacePromptDriver.
  • Add a new extra drivers-prompt-huggingface-pipeline, which includes pytorch (torch) as a dependency to enable running models locally. Note that hugging face allows using pytorch or tensorflow, and I picked pytorch without any reasons in particular.
  • Change Hugging Face Pipeline example's model choice to TinyLlama/TinyLlama-1.1B-Chat-v0.6 so that is small enough to run in the github action test runner as a part of the docs integration tests. (Plus it's nicer for users who want to test out the functionality faster locally)
  • Add a couple of disclaimers about what models work with the HuggingFacePromptDriver.

Manual Testing:

  • Successfully ran the updated Hugging Face Pipeline prompt driver example locally.

📚 Documentation preview 📚: https://griptape--796.org.readthedocs.build//796/

@dylanholmes dylanholmes changed the title Edit hugging face prompt pipeline driver + docs Fix hugging face prompt pipeline driver + docs May 21, 2024
@dylanholmes dylanholmes force-pushed the fix/hugging-face-pipeline-prompt-driver branch from 8da45a1 to 8dba0d4 Compare May 21, 2024 12:13
Copy link

codecov bot commented May 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

@dylanholmes dylanholmes marked this pull request as ready for review May 21, 2024 17:00
@@ -123,6 +125,7 @@ all = [
"markdownify",
"voyageai",
"elevenlabs",
"torch",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain a bit why we need this now?

Copy link
Contributor Author

@dylanholmes dylanholmes May 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this, running the example provided in the docs, I get:

/Users/dylan/Documents/boom/.venv/lib/python3.12/site-packages/stringcase.py:247: SyntaxWarning: invalid escape sequence '\W'
  return re.sub("\W+", "", string)
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
[05/21/24 19:26:43] INFO     PromptTask ed3273d35309405e87eaacc7bb3360ee
                             Input: Hello Girafatron, what is your favorite animal?
WARNING:root:<RetryCallState 6048187696: attempt #1; slept for 0.0; last result: failed (RuntimeError At least one of TensorFlow 2.0 or PyTorch should be installed. To install TensorFlow 2.0, read the instructions at https://www.tensorflow.org/install/ To install PyTorch, read the instructions at https://pytorch.org/.)>
WARNING:root:<RetryCallState 6048187696: attempt #2; slept for 2.0; last result: failed (RuntimeError At least one of TensorFlow 2.0 or PyTorch should be installed. To install TensorFlow 2.0, read the instructions at https://www.tensorflow.org/install/ To install PyTorch, read the instructions at https://pytorch.org/.)>
WARNING:root:<RetryCallState 6048187696: attempt #3; slept for 4.0; last result: failed (RuntimeError At least one of TensorFlow 2.0 or PyTorch should be installed. To install TensorFlow 2.0, read the instructions at https://www.tensorflow.org/install/ To install PyTorch, read the instructions at https://pytorch.org/.)>

I was surprised to see this as well, and I can't explain why now. Though it does makes sense to need an ML framework for local ML inferencing.

@dylanholmes dylanholmes requested a review from collindutter May 21, 2024 17:51
collindutter
collindutter previously approved these changes May 21, 2024
@dylanholmes dylanholmes merged commit b0606c7 into dev May 22, 2024
12 checks passed
@dylanholmes dylanholmes deleted the fix/hugging-face-pipeline-prompt-driver branch May 22, 2024 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants