pythonregextext-miningpdftotextpdf-extraction

PDF Text extraction and storing them as key-value pair


I want to extract text from a PDF. The output I am getting from text extraction is not that much organised.

PDF Link (Only 1st Page): https://microprecision.com/wp-content/uploads/2020/08/Sample-Cert_rev-7-1.pdf

I want to extract parameters like MPC Control No #, Serial No., Model Number, etc. and may store them as key-value pair in dictionary.

I am trying this with the below code but not getting the desired output.

import io
from pdfminer.layout import LAParams, LTTextBox
from pdfminer3.pdfpage import PDFPage
from pdfminer3.pdfinterp import PDFResourceManager,PDFPageInterpreter
from pdfminer3.converter import PDFPageAggregator,TextConverter

def pdftotext(path):
   resource_manager = PDFResourceManager()
   file_handle = io.StringIO()
laprams = LAParams(word_margin=1.0,boxes_flow=0.5,char_margin=2.0,line_overlap=0.5,line_margin=0.5)
   converter = TextConverter(resource_manager,file_handle, laparams=laprams)
   page_interpreter = PDFPageInterpreter(resource_manager,converter)
   i = 1
   with open(path,'rb') as fh:
      for page in PDFPage.get_pages(fh,caching=False,check_extractable=True):
        page_interpreter.process_page(page)
        
      text = file_handle.getvalue()

   converter.close()
   file_handle.close()

   return text

 raw = pdftotext('Sample-Certificate.pdf')
 print(raw)

Solution

  • When working with PDF files I prefer to work with PyMuPDF library https://pypi.org/project/PyMuPDF/

    import fitz
    
    txt = []
    doc = fitz.open("Sample-Cert_rev-7-1.pdf")            # some existing PDF
    page = doc[0]
    text = page.getText("text")
    txt = list(text)
    print(text)
    text = text.split('\n')
    txt = list(text)
    print(txt)
    ix = text.index('MPC Control #:')
    print(ix)
    print(text[ix+18])
    

    Pay attention on how to install the library correctly
    Here is the output:

    "C:\Program Files\Python38\python.exe" C:/Python/stackoverflow extract_pdf_text1.py
    MICRO PRECISION CALIBRATION, INC.
    22835 INDUSTRIAL PLACE
    GRASS VALLEY CA 95949
    530-268-1860
    Cert No.
    551220083746791
    Date: Aug 3, 2020
    Certificate of Calibration
    AC-1969.00
    N/A
    July 01, 2021
    N/A
    Customer:
    MPC Control #:
    Asset ID:
    Gage Type:
    Manufacturer:
    Model Number:
    Size:
    Temp/RH:
    Serial Number:
    Department:
    Performed By:
    Received Condition:
    Returned Condition:
    Cal. Date:
    Cal. Interval:
    Cal. Due Date:
    Work Order #:
    DIGITAL MULTIMETER
    DANNY BOY B. BUTIAL
    0258964
    0258964
    NONE
    AGILENT
    34401A
    10MHZ
    SAMPLE
    N/A
    IN TOLERANCE
    IN TOLERANCE
     July 01, 2020
    N/A
    12 MONTHS
    Calibration Notes:
    SAMPLE COMPANY
    23.0°C / 40.0%
    Location:
    Calibration performed at MPC facility
    Standards Used to Calibrate Equipment
    I.D.
    Description.
    Model
    Serial
    Manufacturer
    Cal. Due Date
    Traceability #
    PH1405
    MULTI-PRODUCT CALIBRATOR
    5520A
    7575006
    FLUKE
    Sep 10, 2020
    551220083204793
    AL4394
    DIGITAL MULTIMETER
    3458A
    2823A09832
    AGILENT
    Aug 1, 2020
    551220083719099
    Procedures Used in this Event
    Procedure Name
    Description
    MPC Automated Procedure
    MPCCAL Rev. 00
    STATEMENTS OF PASS OR FAIL CONFORMANCE: The uncertainty of measurement has been taken into account when determining compliance with specification. All measurements and test results guard banded to ensure the
    probability of false-accept does not exceed 2% in compliance with ANSI/NCSL Z540.3-2006 and in case without guard banded the probability of false-accept depending on test uncertainty ratio.
    THE CALIBRATION REPORT STATUS:
    PASS- Term used when compliance statement is given, and the measurement result is PASS.
    PASSz- Term used when compliance statement is given, and the measurement result is conditional passed or PASSz.
    FAIL- Term used when compliance statement is given, and the measurement result is FAIL.
    FAILz- Term used when compliance statement is given, and the measurement result is conditional failed or FAILz.
    REPORT OF VALUE - Term used when reported measurement is not requiring compliance statement in report.
    ADJUSTED- When adjustments are made to an instrument which changes the value of measurement from what was measured as found to new value as left.
    LIMITED - When an instrument fails calibration but is still functional in a limited manner.
    The expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k=2, which for a normal distribution corresponds to a coverage probability of approximately 95%, unless otherwise stated. This
    calibration report complies with ISO/IEC 17025:2017 and ANSI/NCSL Z540.3. Calibration cycles and resulting due dates were submitted/approved by the customer. Any number of factors may cause an instrument to drift out of tolerance before the next
    scheduled calibration. Recalibration cycles should be based on frequency of use, environmental conditions and customer's established systematic accuracy. All standards are traceable to SI through the National Institute of Standards and Technology (NIST)
    and/or recognized national or international standards laboratories. Services rendered include proper manufacturer’s service instruction and are warranted for no less than thirty (30) days. The information on this report pertains only to the instrument identified,
    this may not be reproduced in part or in a whole without the prior written approval of the issuing MP Calibration Laboratory.
    Rick Hernandez
    Calibrating Technician:
    QC Approval:
    DANNY BOY B. BUTIAL
    (CERT, Rev 7)
    Page 1 of 1
    
    ['MICRO PRECISION CALIBRATION, INC.', '22835 INDUSTRIAL PLACE', 'GRASS VALLEY CA 95949', '530-268-1860', 'Cert No.', '551220083746791', 'Date: Aug 3, 2020', 'Certificate of Calibration', 'AC-1969.00', 'N/A', 'July 01, 2021', 'N/A', 'Customer:', 'MPC Control #:', 'Asset ID:', 'Gage Type:', 'Manufacturer:', 'Model Number:', 'Size:', 'Temp/RH:', 'Serial Number:', 'Department:', 'Performed By:', 'Received Condition:', 'Returned Condition:', 'Cal. Date:', 'Cal. Interval:', 'Cal. Due Date:', 'Work Order #:', 'DIGITAL MULTIMETER', 'DANNY BOY B. BUTIAL', '0258964', '0258964', 'NONE', 'AGILENT', '34401A', '10MHZ', 'SAMPLE', 'N/A', 'IN TOLERANCE', 'IN TOLERANCE', ' July 01, 2020', 'N/A', '12 MONTHS', 'Calibration Notes:', 'SAMPLE COMPANY', '23.0°C / 40.0%', 'Location:', 'Calibration performed at MPC facility', 'Standards Used to Calibrate Equipment', 'I.D.', 'Description.', 'Model', 'Serial', 'Manufacturer', 'Cal. Due Date', 'Traceability #', 'PH1405', 'MULTI-PRODUCT CALIBRATOR', '5520A', '7575006', 'FLUKE', 'Sep 10, 2020', '551220083204793', 'AL4394', 'DIGITAL MULTIMETER', '3458A', '2823A09832', 'AGILENT', 'Aug 1, 2020', '551220083719099', 'Procedures Used in this Event', 'Procedure Name', 'Description', 'MPC Automated Procedure', 'MPCCAL Rev. 00', 'STATEMENTS OF PASS OR FAIL CONFORMANCE: The uncertainty of measurement has been taken into account when determining compliance with specification. All measurements and test results guard banded to ensure the', 'probability of false-accept does not exceed 2% in compliance with ANSI/NCSL Z540.3-2006 and in case without guard banded the probability of false-accept depending on test uncertainty ratio.', 'THE CALIBRATION REPORT STATUS:', 'PASS- Term used when compliance statement is given, and the measurement result is PASS.', 'PASSz- Term used when compliance statement is given, and the measurement result is conditional passed or PASSz.', 'FAIL- Term used when compliance statement is given, and the measurement result is FAIL.', 'FAILz- Term used when compliance statement is given, and the measurement result is conditional failed or FAILz.', 'REPORT OF VALUE - Term used when reported measurement is not requiring compliance statement in report.', 'ADJUSTED- When adjustments are made to an instrument which changes the value of measurement from what was measured as found to new value as left.', 'LIMITED - When an instrument fails calibration but is still functional in a limited manner.', 'The expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k=2, which for a normal distribution corresponds to a coverage probability of approximately 95%, unless otherwise stated. This', 'calibration report complies with ISO/IEC 17025:2017 and ANSI/NCSL Z540.3. Calibration cycles and resulting due dates were submitted/approved by the customer. Any number of factors may cause an instrument to drift out of tolerance before the next', "scheduled calibration. Recalibration cycles should be based on frequency of use, environmental conditions and customer's established systematic accuracy. All standards are traceable to SI through the National Institute of Standards and Technology (NIST)", 'and/or recognized national or international standards laboratories. Services rendered include proper manufacturer’s service instruction and are warranted for no less than thirty (30) days. The information on this report pertains only to the instrument identified,', 'this may not be reproduced in part or in a whole without the prior written approval of the issuing MP Calibration Laboratory.', 'Rick Hernandez', 'Calibrating Technician:', 'QC Approval:', 'DANNY BOY B. BUTIAL', '(CERT, Rev 7)', 'Page 1 of 1', '']
    13
    0258964
    
    Process finished with exit code 0