pythonodbcpypyodbc

How to export MS Access table into a csv file in Python using e.g. pypyodbc


I have been trying to export a table from MS Access database into a csv file using pypydobc - using fetchone function is taking forever e.g. 200,000 rows are taking about 5 minutes to print. If fetchone was quicker I could have just printed the results into a csv file but it's taking too long. This is what I tried so far:

import pypyodbc
pypyodbc.lowercase = False
conn = pypyodbc.connect(
    r"Driver={Microsoft Access Driver (*.mdb, *.accdb)};" +
    r"Dbq=C:\temp\Temp_DB.accdb;")
cur = conn.cursor()
cur.execute("SELECT Column1, Column2, FROM Table1");
Col1 = []
Col2 = []
row = cur.fetchone()
while  row is not None:
    print(row)
    row = cur.fetchone()
    Col1.append(row.get("Column1"))
    Col2.append(row.get("Column2"))
cur.close()
conn.close()

Also, is there a documentation on all functions in pypyodbc which I have failed to find so far?


Solution

  • Consider using cur.fetchall() and the csv module to directly output query results without needing to append to individual lists (and consider refraining from print to save process time):

    import pypyodbc
    import csv
    
    # MS ACCESS DB CONNECTION
    pypyodbc.lowercase = False
    conn = pypyodbc.connect(
        r"Driver={Microsoft Access Driver (*.mdb, *.accdb)};" +
        r"Dbq=C:\temp\Temp_DB.accdb;")
    
    # OPEN CURSOR AND EXECUTE SQL
    cur = conn.cursor()
    cur.execute("SELECT Column1, Column2, FROM Table1");
    
    # OPEN CSV AND ITERATE THROUGH RESULTS
    with open('Output.csv', 'w', newline='') as f:
        writer = csv.writer(f)    
        for row in cur.fetchall() :
            writer.writerow(row)
    
    cur.close()
    conn.close()
    

    Aside - above you are connecting to the Jet/ACE SQL Engine (an object of the MSAccess.exe and not restricted to the program but available to all Office/Windows programs -the misnomer to think MS Access is a database but is actually a GUI console to one. Alternatively for your csv export, you can interact with the GUI application and run Access' TransferText() method to export tables/queries to text delimited files. And Python can open the database and call the export method with win32com module:

    import win32com.client
    
    # OPEN ACCESS APP AND DATABASE
    oApp = win32com.client.Dispatch("Access.Application")
    oApp.OpenCurrentDatabase('C:\temp\Temp_DB.accdb')
    
    # EXPORT TABLE TO CSV
    acExportDelim = 2
    oApp.DoCmd.TransferText(acExportDelim, None, "Table1", 'Output.csv'), True)
    
    oApp.DoCmd.CloseDatabase
    oApp.Quit
    oApp = None