amazon-web-servicesmultithreadingdownloads3cmd

How to use s3cmd multithreaded?


Our goal is to download about 5 millions of tiny files from AWS to CentOS server. We found s3cmd utility and it was very good for almost all things. Except downloading, because it is only supports one thread :( which follow to 60 days of downloading, which is just crazy!

Is there any new version of s3cmd or another way to download all files in multithread mode?


Solution

  • I have open s4cmd for myself, awesome tool. And yes, it supports multithreading.