google-app-enginegoogle-cloud-platformgoogle-cloud-storagegoogle-cloud-dataflowgcsfuse

FTP to Google Storage


Some files get uploaded on a daily basis to an FTP server and I need those files under Google Cloud Storage. I don't want to bug the users that upload the files to install any additional software and just let them keep using their FTP client. Is there a way to use GCS as an FTP server? If not, how can I create a job that periodically picks up the files from an FTP location and puts them in GCS? In other words: what's the best and simplest way to do it?


Solution

  • I have successfully set up an FTP proxy to GCS using gcsfs in a VM in Google Compute (mentioned by jkff in the comment to my question), with these instructions: [url unlinked due to dangerous site] ilyapimenov.com/blog/2015/01/19/ftp-proxy-to-gcs.html

    Some changes are needed though:

    Some possible problems:

    Also, your ftp client needs to use the transfer mode set to "passive".