Extracting and Counting Individual Pictures using PDF Plumber

[nigelkiernan]

Dear Jeremy & Samkit,

I wonder if I might be able to get your help with an issue extracting and counting photos in PDF Plumber.
When I extract an individual page, which contains 1 image made up of 4 photos, PDF Plumber allows me to extract the info
into a DataFrame which shows the 4 individual photos that make up the 1 collective image.
However, when I extract a whole document into a DataFrame, PDF Plumber extracts all of the images but classifies the extractions as images only. So, following the previous one page example, the four separate photos would only be classified as 1 single image. Also PDF Plumber counts non photos, such as signatures & graphics, as images.

As such, when extracting a whole document:

1. Is there a way to classify the extractions by the number of individual photos per page, rather than the collective images per page, such that I can count individual photos that make up images, as per extracting the single page example as before?
2. Is there a way to extract only photo images, but ignore images such as signatures, graphics etc?

Please see me code below just for your FYI. Thank you again for this program which has been super helpful.
Kind regards
Nigel

From a single page: extracting photos within 1 image. This page contains 4 photos within 1 single image:
page_5 = pdf.pages[5] '
images_in_page = page_5.images
print(images_in_page)
images_in_page_df = pd.DataFrame(images_in_page) # creating a DataFrame.

When this DataFrame is created, it contains 4 separate photos, each allocated to a separate row in the DataFrame

Extracting From Whole Document
pdf = pdfp.open('XXXXX.pdf')
for page in pdf.pages:
print(page.images)
images_df = pd.DataFrame({"Image": [p.images for p in pdf.pages]}, columns=["Image"])
images_df.head(10)

[samkit-jain]

Hi @nigelkiernan Appreciate your interest in the library. Request you to, if possible, attach the PDF (redacting any sensitive information) in question as it will help us debug the issue in a better way. Based on the information provided,

1. In the first code, when creating the dataframe, you are passing a list of dicts and seeing 4 rows. In the second code, you are passing a list of list of dicts and hence, you are seeing only 1 entry which is a list. My guess would be that the list is containing 4 dicts in which case the result is expected and you might be confusing that single row entry with the list as a single image. If the list indeed contains a single dict then it could be a bug and would need the PDF to investigate further.
2. I am not sure if it is possible to differentiate between the images. You would need to apply some post-processing logic to filter out the images that don't match the criteria. It could be based on the size or the colors or maybe some other property. Because, technically, if I embed a photo of a signature and a photo of a scenery, both are valid images. And, if I want to ignore the signature photo, then, would need to add some post-processing to first identify that an image is of a signature or not. Of course, your use case might be more simplified and having a filtering logic on the size or any of the other properties might be enough.

[nigelkiernan]

Hi Samkit,

Thanks very much for your reply which makes sense. In reply to each part in turn:

1. This makes sense; thank you for the explanation. Is it possible to extract a whole document and create a DataFrame which illustrates the extracted images as a list of dicts, rather than a list of list of dicts? If so, could you kindly share the code to do so please?
2. Sure, if it is not possible to differentiate between the images, I completely understand. The reason I asked is that, when a DataFrame is created that is made up of a list of dicts, like the example below, there is a range of information here; I was curious to know if graphics, for example, might have specific values for the ['stream'] column category that might distinguish them from pictures, such that certain rows could be counted whilst others are dropped. This is illustrated again in the image below.

If point 2. above is not technically possible, then no problem, however, if point 1. above is technically possible & you could share the required code then your help would be very appreciated.
BTW, the document I am experimenting with is the 2018 Wirecard Annual Report, which is in the public domain.
Wirecard_Annual-Report-2018.pdf

As always, thank you very much for all of your support - I very much appreciate the dialog and have found this tool to be very helpful.
Kind regards
Nigel

[samkit-jain]

For 1, the following might help

import pandas as pd
import pdfplumber

pdf = pdfplumber.open("file.pdf")
all_images = []

for page in pdf.pages:
    for image in page.images:
       all_images.append({**image, **{"page_number": page.page_number}}

df = pd.DataFrame(all_images)

For 2, can you tell me the page from where you want to discard the images? I checked page 9 where there is a signature but .images returns an empty list over there.

[nigelkiernan]

Thanks very much Samkit, this is super helpful. I'll check again on point 2) after running the above. Thank you!

Kind regards
Nigel

[rloibman]

Hey, really interesting!
How can I remount an image from the data stored in the DataFrame?
I already extracted the data using pdfplumber.
What I want is to save the images separately in a folder.

[jsvine]

Hi @rloibman, support for saving images is currently limited. For more context, see this discussion: #677