Few different types of issues using cuda, including CUDA_ERROR and Invalid input type: "PdfDocument"

[JPZ4-5]

Platform: CentOS 7
Python: 3.10.14
CUDA: 11.8

Tried two types of environments:

1. With latest environment using pyproject.toml

After #58 is done, I have created my environment, where the key package versions include torch 2.3.1 surya-ocr 0.4.15 marker-pdf 0.2.16 flash-attn 2.6.1 (default), and the program is able to start normally under such circumstances.

python server.py --host 0.0.0.0 --port 20202 --media --documents

By using ssh -L 127.0.0.1:20202:server:20202, I can remotely access the management page at 127.0.0.1:20202.

However, I have encountered #24, unfortunately. So I had to try http://127.0.0.1:20202/docs instead.

In the first attempt, due to my mistakenly using the wrong CUDA version; in fact, I used CUDA-10.0, which led to an error like "driver too old". This caused the program to actually run on the CPU.
As #24 mentioned, I only tried to parse pdf. The process is slow but steady, which gives the expected result.

After changed to CUDA-11.8, a series of questions have arisen. After the server.py is started normally, access http://127.0.0.1:20202/docs, upload a PDF file, execute, and the result of the first run is as follows:

Once tried a few more times, the error changed to:

If continue trying, the error will remain the same and won't change.

Sometimes, after rerunning server.py and attempting the above, there is no error in the first figure but instead directly giving an error as shown in Figure 2.

2. With torch 2.2.2

After noticing your comment in pyproject.toml which suggesting torch 2.2.2 is more stable, I tend to tried another time.

I have found a combination of marker-pdf and surya-ocr that is compatible with torch 2.2.2. The key package versions in this environment are: torch 2.2.2, surya-ocr 0.4.5, marker-pdf 0.2.6, flash-attn 2.6.1 (wheel with abiFalse, since the one with abiTrue comes with more error).

Similar to the method described earlier for starting services, try parsing a PDF by accessing http://127.0.0.1:20202/docs since the problem in #24 remains.
The error is:

Hope information above would be useful!
I wonder if there is any way to solve and run the project correctly. I have tried singularity to use docker without sudo, but it did not work without sudo and usernamespace support.

I am more than happy to provide any further information needed