[SOLVED] Convert paramiko.sftp_file.SFTPFile object to unicode

Convert paramiko.sftp_file.SFTPFile object to unicode

What I need, is to copy a .csv file from an SFTP server to a Google Cloud Storage bucket without downloading the file locally.

Here is my code so far:

from google.cloud import storage 
import io 
import paramiko

# connection parameters
host = 'host'
port = 22
usr = 'me'
pwd = '0123'

# Connect to the SFTP server
sftp_client = paramiko.SSHClient()
sftp_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
sftp_client.connect(host, port, usr, pwd)
sftp = sftp_client.open_sftp()

# Get the file
sftp_file =  sftp.open('./file.csv', 'rb')

# Call the Google Cloud Storage API
storage_client = storage.Client()
bucket_name = 'the_bucket'
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(sftp_file)

# Upload to bucket
blob.upload_from_filename(sftp_file)

And here is the error returned:

ValueError: <paramiko.sftp_file.SFTPFile object at 0x0000022665279A00> could not be converted to unicode

Can someone unstuck me?

Solution

Use Blob.upload_from_file to upload the file-like object that you have opened using SFTPClient.open:

with sftp.open('./file.csv', 'rb') as sftp_file:
    sftp_file.prefetch()
    blob.upload_from_file(sftp_file)

For the purpose of SFTPFile.prefetch, see Reading file opened with Python Paramiko SFTPClient.open method is slow.

And just to be very clear, this code still downloads the file from SFTP server to the local machine – it just does not store it to a local file.