Doc file splitting into multiple image files

[llmwesee]

When uploading .doc or .docx files, the following warnings are displayed:
No acceptable contours found
Contour is not a quadrilateral
lib/python3.10/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: Since Chroma 0.4.x the manual persistence method is no longer supported as docs are automatically persisted. warn_deprecated(

After uploading, a single .doc file splits into around 59 documents in .png or .jpeg formats. These are shown in the Doc Counts sidebar and also form the metadata as illustrated in the attached screenshot.

The main issue is the inability of the .doc file to be parsed as a single document, unlike .pdf files. Instead, it splits into multiple .png or .jpeg files, leading to hallucination during querying.

Please address the parsing issue and provide a solution to handle .doc or .docx files correctly without splitting into multiple image files.

[pseudotensor]

We extract images so that they can be processed separately for image question-answer, but I understand if there are many images it might get messy.

I added an ENV so you can control with H2OGPT_DOCX_EXTRACT_IMAGES. set it to "0" to avoid this step.

Docker with this feature will be in new build in few hours.