javauploadopenstack-swiftjclouds

Upload Very Large File to Swift Container


I'm using jclouds java SDK to upload to Swift Container i saw the code in which i upload as a multipart,i already upload as a multipart as input stream using the code

    try (ByteArrayInputStream inputStream = new ByteArrayInputStream(bytes)) {
                    ///input here is InputStream not ByteSource
                    Payload payload = Payloads.newInputStreamPayload(inputStream);
                    Blob blob = blobStore.blobBuilder(path).payload(payload).build();
                    ///sednig the request
                    blobStore.putBlob(ContainerName, blob, multipart());
                    System.out.println(String.format("End upload all Parts, parts number=%s, part size=%s", strPartsCount, strPartSize));  
}

but i have a case where i have a very large file, the input to it is a bytes matrix, the solution is very easy which is put the same code in a for loop and each loop the input is an index of the matrix (which is a byte array)

but the question is, would the OpenStack consider each loop (multipart upload) is linked to the previous part? how to do that? you can say a multipart inside a multipart

Thanks


Solution

  • jclouds offers two options:

    1) Call the individual multipart methods, e.g., initiateMultipartUpload, uploadMultipartPart, and completeMultipartUpload.

    2) Create an InputStream or ByteSource payloads which concatenates individual parts, e.g. SequenceInputStream or ByteSource.concat.

    Note that using ByteSource.wrap(bytes) instead of ByteArrayInputStream(bytes) makes the payload repeatable and jclouds will retry on network errors.