amazon-web-servicesamazon-s3nas

How to move files from on-prem NAS to AWS S3 when file arrives on NAS


I have a requirement where we need to move files from on-prem NAS storage to AWS S3. Files keep coming on NAS storage when it arrives we have notification set up in AWS and then we need to pull files from AWS to S3.

  1. Can I access NAS storage and pull files from AWS to S3?
  2. Does it require any additional configuration or simple EC2 or Lambda function can work based on size of the file?
  3. How about NAS --> SFTP --> S3 using AWS Transfer family solution.
  4. Is there any better way to move files from NAS to S3?

We want to avoid writing code as much as we can.


Solution

  • You should take a look at AWS Datasync. It is a data transfer service of AWS that allow to copy data to and from AWS storage services over the Internet or over AWS Direct Connect (protocols NFS, SMB).

    You don't need EC2 or AWS lambda. You have to install an agent that will read from a source location, and sync your data to S3. The agent is deployed on-premise. Please find the supported Hypervisor here: https://docs.aws.amazon.com/datasync/latest/userguide/agent-requirements.html and the deployment guide here: https://docs.aws.amazon.com/datasync/latest/userguide/deploy-agents.html