pythoncsvsftpparamikotemporary-files

How to put data into a tempfile and post as CSV on SFTP

Goal is

Create a temporary SCP file filled with data and upload it to an sftp. The data to fill is TheList and is from class list.

What I am able to achieve

What happens with the code below

There is a file created/put to the SFTP, but the file is empty and has 0 byte.

Question

How can I achieve that I have a file with type SCP on SFTP with the content of TheList?

import paramiko
import tempfile
import csv

# code part to make and open sftp connection

TheList = [['name', 'address'], [ 'peter', 'london']]

csvfile = tempfile.NamedTemporaryFile(suffix='.csv', mode='w', delete=False)
filewriter = csv.writer(csvfile)
filewriter.writerows(TheList)
sftp.put(csvfile.name, SftpPath + "anewfile.csv")

# code part to close sftp connection
Solution

  • You do not need to create a temporary file. You can use csv.writer to write the rows directly to the SFTP with use of file-like object opened using SFTPClient.open:

    with sftp.open(SftpPath + "anewfile.csv", mode='w', bufsize=32768) as csvfile:
    writer = csv.writer(csvfile, delimiter=',')
    filewriter.writerows(TheList)

    See also pysftp putfo creates an empty file on SFTP server but not streaming the content from StringIO

    To answer your literal question: I believe you need to flush the temporary file before trying to upload it:

    filewriter.flush()

    See How to use tempfile.NamedTemporaryFile() in Python

    Though better option would be to use Paramiko SFTPClient.putfo to upload the NamedTemporaryFile object, rather then trying to refer to the temporary file via the filename (what allegedly would not work at least on Windows anyway):

    csvfile.seek(0)
sftp.putfo(csvfile, SftpPath + "anewfile.csv")