-
Notifications
You must be signed in to change notification settings - Fork 828
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: generic partition
brick with filetype detection
#132
Conversation
Test failure is due to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also need to add support in the Dockerfile? I believe the package is file-devel
Per offline convo, I'm removing the dockerfile since we're not using that for anything |
@@ -20,7 +20,7 @@ install-base: install-base-pip-packages install-nltk-models | |||
install: install-base-pip-packages install-dev install-nltk-models install-test install-huggingface install-unstructured-inference | |||
|
|||
.PHONY: install-ci | |||
install-ci: install-base-pip-packages install-test install-nltk-models install-huggingface install-unstructured-inference | |||
install-ci: install-base-pip-packages install-test install-nltk-models install-huggingface install-unstructured-inference local-inference |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
install-ci: install-base-pip-packages install-test install-nltk-models install-huggingface install-unstructured-inference local-inference | |
install-ci: install-base-pip-packages install-test install-nltk-models install-huggingface install-unstructured-inference install-local-inference |
Summary
Adds a generic
partition
brick that detects the file type and then invokes the appropriate partitioning brick. In support of this functionality, this PR also adds the following:magic
in the dependencies and instructions for installinglibmagic
"rb"
mode topartition_html
andpartition_eml
Testing