- UnstructuredDirectoryLoader uses ๐ฆ๏ธ๐ LangChain <langchain_community.document_loaders.unstructured> UnstructuredFileLoader to load files like '.txt', '.csv', '.pdf', '.msg' into a List[Document] using ๐ฆ๏ธ๐ LangChain <langchain_core.documents> Document
The Python version used when this was developed was 3.9 ๐ฎ
pip install UnstructuredDirectoryLoader
-
Just pass you local directory path in UnstructuredDirectoryLoader() and then use this directory loader.
directory_loader = UnstructuredDirectoryLoader(directory_path='*<Your_Local_Folder_Path>*')
listDocs = directory_loader.load()
-
Done! Go ahead and use a text splitter on this docs
text_splitter = SemanticChunker(HuggingFaceEmbeddings())