amazon-web-servicesamazon-s3parallel-processingcomputer-sciencedisk

Are the reads from S3 truly parallel?


Consider a large file uploaded on S3 (without multi part upload). In many articles like this, I see that programs/applications can parallelly read from a single file on S3

Are the reads from S3 truly parallel? I ask this because given there is only one head in a hard disk, how can the head parallelly seek different chunks parallelly and transfer them?

Extending this question on a local machine, can a file stored on a single hard disk be read parallelly in chunks?


Solution

  • Objects in Amazon S3 are stored on multiple devices in each data center and are replicated across multiple data centers. An object is certainly not stored on a single hard disk.

    It does not matter whether multiple users are accessing the same file or different files. Each disk block can be served from multiple S3 servers from multiple storage devices.

    This is different to your own computer where you only have 'one copy' of a file. While a NAS device stores multiple copies of data, there is only a single NAS and a single network connection between your computer and a NAS, so it isn't truly parallel.