pythonweb-scrapingpdfminerpdf-scraping

Extract metadata info from online pdf using pdfminer in python


I am interested to find out some metadata of an online pdf using pdfminer. I am interested in extracting info such as Title, author, no of lines etc from the pdf

I am trying to use a related solution discussed in- https://stackoverflow.com/a/60151816/15143974

Which uses the following code-

from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.layout import LAParams
from pdfminer.converter import TextConverter
from pdfminer.pdfpage import PDFPage
import io
import urllib.request
import requests


def pdf_to_text(pdf_file):
    text_memory_file = io.StringIO()

    rsrcmgr = PDFResourceManager()
    device = TextConverter(rsrcmgr, text_memory_file, laparams=LAParams())
    interpreter = PDFPageInterpreter(rsrcmgr, device)
# get first 3 pages of the pdf file
    for page in PDFPage.get_pages(pdf_file, pagenos=(0, 1, 2)):
        interpreter.process_page(page)
    text = text_memory_file.getvalue()
    text_memory_file.close()
    return text

# # online pdf to text by urllib
# online_pdf_file=urllib.request.urlopen('http://www.dabeaz.com/python/UnderstandingGIL.pdf')
# pdf_memory_file=io.BytesIO()
# pdf_memory_file.write(online_pdf_file.read())
# print(pdf_to_text(pdf_memory_file))


# online pdf to text by requests
response = requests.get('http://www.dabeaz.com/python/UnderstandingGIL.pdf')
pdf_memory_file = io.BytesIO()
pdf_memory_file.write(response.content)
print(pdf_to_text(pdf_memory_file))

However, I am not able to find where to make the required changes to this code


Solution

  • You may find pdfplumber of interest - it's built on top of pdfminer.six and simplfies a lot of tasks.

    import io
    import pdfplumber
    import requests
    
    url = "http://www.dabeaz.com/python/UnderstandingGIL.pdf"
    content = io.BytesIO(requests.get(url).content)
    
    pdf = pdfplumber.open(content)
    
    >>> pdf.metadata
    {'Title': 'UnderstandingGIL',
     'Author': 'David Beazley',
     'Subject': '',
     'Producer': 'Mac OS X 10.6.2 Quartz PDFContext',
     'Creator': 'Keynote',
     'CreationDate': "D:20100220124003Z00'00'",
     'ModDate': "D:20100220124003Z00'00'",
     'Keywords': '',
     'AAPL:Keywords': ['']}