PDFMiner

PDFMiner is a text extraction tool for PDF documents.

Warning: As of 2020, PDFMiner is not actively maintained. The code still works, but this project is largely dormant. For the active project, check out its fork pdfminer.six.

Features:

• Pure Python (3.6 or above).
• Supports PDF-1.7. (well, almost)
• Obtains the exact location of text as well as other layout information (fonts, etc.).
• Performs automatic layout analysis.
• Can convert PDF into other formats (HTML/XML).
• Can extract an outline (TOC).
• Can extract tagged contents.
• Supports basic encryption (RC4 and AES).
• Supports various font types (Type1, TrueType, Type3, and CID).
• Supports CJK languages and vertical writing scripts.
• Has an extensible PDF parser that can be used for other purposes.

How to Use:

1. pip install -r requirements.txt
2. python3 -m tools.pdf2txt samples/simple1.pdf
3. python3 -m tools.pdf2txt [-t html, xml, text] samples/equations.pdf
4. python3 -m tools.pdffontsinfo samples/simple1.pdf

How to use the new feature

1. python3 -m tools.pdf2txt -ch [Chapter definition (ex. chapter)] samples/Crime_and_Punishment_T_short.pdf

How to test

Run python3 -m unittest in the root folder

How to check test coverage

Install coverage.py: pip install coverage

To check statement coverage run coverage run -m unittest on the root folder. Then, to see the report run coverage report --include=pdfminer/*,tools/*. In case of a more detailed report add the attribute -m.

To check the branch coverage run coverage run --branch -m unittest on the root folder. Then, to see the report run coverage report --include=pdfminer/*,tools/*

To see the report in a nicer way, run coverage html --include=pdfminer/*,tools/* and an html file wil be created htmlcov/index.html.

How to run the profilers

For the CPU profiler: python -m cProfile -o output.prof -s tottime -m module args

where output.prof is the file in which the profiler output is stored where module is the pdfminer module or .py file is going to be profiled and args are the arguments for that code.

After running the profiler, the output.prof generated can be visualized with SnakeViz. So, first install it ( pip install snakeviz ) and then run it like this: snakeviz output.prof

For the memory profiler:

Install the profiler: pip install memory_profiler

To the profiled piece of code add the @profile annotation before the method. (Notice from memory_profiler import profile is needed).

After adding the neccesary annotations, run the profiler: python -m memory_profiler module args

where module is the pdfminer module or .py file is going to be profiled and args are the arguments for that code.

Command Line Syntax:

pdf2txt.py

pdf2txt.py extracts all the texts that are rendered programmatically. It also extracts the corresponding locations, font names, font sizes, writing direction (horizontal or vertical) for each text segment. It does not recognize text in images. A password needs to be provided for restricted PDF documents.

> pdf2txt.py [-P password] [-o output] [-t text|html|xml|tag]
             [-O output_dir] [-c encoding] [-s scale] [-R rotation]
             [-Y normal|loose|exact] [-p pagenos] [-m maxpages]
             [-S] [-C] [-n] [-A] [-V]
             [-M char_margin] [-L line_margin] [-W word_margin]
             [-F boxes_flow] [-d]
             input.pdf ...

• -P password : PDF password.
• -o output : Output file name.
• -t text|html|xml|tag : Output type. (default: automatically inferred from the output file name.)
• -O output_dir : Output directory for extracted images.
• -c encoding : Output encoding. (default: utf-8)
• -s scale : Output scale.
• -R rotation : Rotates the page in degree.
• -Y normal|loose|exact : Specifies the layout mode. (only for HTML output.)
• -p pagenos : Processes certain pages only.
• -m maxpages : Limits the number of maximum pages to process.
• -S : Strips control characters.
• -C : Disables resource caching.
• -n : Disables layout analysis.
• -A : Applies layout analysis for all texts including figures.
• -V : Automatically detects vertical writing.
• -M char_margin : Speficies the char margin.
• -W word_margin : Speficies the word margin.
• -L line_margin : Speficies the line margin.
• -F boxes_flow : Speficies the box flow ratio.
• -d : Turns on Debug output.

dumppdf.py

dumppdf.py is used for debugging PDFs. It dumps all the internal contents in pseudo-XML format.

> dumppdf.py [-P password] [-a] [-p pageid] [-i objid]
             [-o output] [-r|-b|-t] [-T] [-O directory] [-d]
             input.pdf ...

• -P password : PDF password.
• -a : Extracts all objects.
• -p pageid : Extracts a Page object.
• -i objid : Extracts a certain object.
• -o output : Output file name.
• -r : Raw mode. Dumps the raw compressed/encoded streams.
• -b : Binary mode. Dumps the uncompressed/decoded streams.
• -t : Text mode. Dumps the streams in text format.
• -T : Tagged mode. Dumps the tagged contents.
• -O output_dir : Output directory for extracted streams.

TODO

• Replace STRICT variable with something better.
• Improve the debugging functions.
• Use logging module instead of sys.stderr.
• Proper test cases.
• PEP-8 and PEP-257 conformance.
• Better documentation.
• Crypto stream filter support.

Related Projects

• pyPdf
• xpdf
• pdfbox
• mupdf