[SOLVED] Fineuploader: large files to Amazon S3

Fineuploader: large files to Amazon S3

For small files (smaller than about 5MB) Fineuploader sends a complete policy objet for signing, and upload succeeds including the posprocessing which is sending a temporary link to the client, and some server-side administrative stuff. I use the official FineUploader S3 demo PHP code for server. https://github.com/Widen/fine-uploader-server/blob/master/php/s3/s3demo-thumbnails.php

Example of signing request (formatted, anonimized):

{
    "expiration": "2013-12-28T12:57:42.354Z",
    "conditions": [
        {"acl": "private"},
        {"bucket": "TOP-SECRET-BUCKET-NAME"},
        {"Content-Type": "application/pdf"},
        {"success_action_status": "200"},
        {"key": "bfccb67e-5343-4e01-97ff-2dcffe681da0.pdf"},
        {"x-amz-meta-qqfilename": "plakat_a3.pdf"},
        ["content-length-range","0","1000111000111"]
    ]
}

Response contains policy and signature (formatted):

{
    "policy":"eyJleHBpcmF0aW9uIjoiMjAxMy0xMi0yOFQxMzoxODoyNy4yODhaIiwiY29uZGl0aW9ucyI6W3siYWNsIjoicHJpdmF0ZSJ9LHsiYnVja2V0IjoiZGVtb2Nza2EifSx7IkNvbnRlbnQtVHlwZSI6ImFwcGxpY2F0aW9uXC9wZGYifSx7InN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyI6IjIwMCJ9LHsia2V5IjoiMWZiMjZmMjAtNjg2Ni00YjU1LTg3YTctZWZlMjNiOWMwZmY1LnBkZiJ9LHsieC1hbXotbWV0YS1xcWZpbGVuYW1lIjoicGxha2F0X2EzLnBkZiJ9LFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLCIwIiwiMTAwMDExMTAwMDExMSJdXX0=",
    "signature":"Wlw1QJjwmsASyQemUWrYuktiQwE="
}

... but for larger files Fineupload sends a different signing request, which only contains headers (formatted, anonimized):

{
    "headers": "POST\n\nvideo/quicktime\n\nx-amz-acl:private\nx-amz-date:Sat, 28 Dec 2013 12:53:13 GMT\nx-amz-meta-qqfilename:20MB_stopmot-minta.mov\n/TOP_SECRET-BUCKET-NAME/d2033a4c-1e55-49a0-8589-9b1725dcd013.mov?uploads"
}

Response contains only signature (formatted):

{
    "signature":"jYtFC91wIPkZj31W\/vwuK9ClawU="
}

In that case the OPTIONS request to amazon S3 fails:

Request Headers 14:09:09.000
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0
Pragma: no-cache
Origin: http://top-secret.example.com
Host: TOP-SECRET-BUCKET-NAME.s3.amazonaws.com
Connection: keep-alive
Cache-Control: no-cache
Access-Control-Request-Method: POST
Access-Control-Request-Headers: authorization,content-type,x-amz-acl,x-amz-date,x-amz-meta-qqfilename
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Response:

Status: 403
Body: empty

Response headers:

x-amz-request-id: 625913F399C626A2
x-amz-id-2: +enheIuZT0RJ+11THF1TuNsA6bCqE4a2ppsklK84k4KXdNX4UsuGhxPf00Eb137G
Transfer-Encoding: chunked
Server: AmazonS3
Date: Sat, 28 Dec 2013 13:18:01 GMT
Content-Type: application/xml

Test cases:

Tested with smaller .mov, .mp3, .pdf, .log files: course #1 (success)
Tested with larger files (same types): course #2 (fail)

What can cause the difference? How can / should I fix it?

Solution

I needed to add <ExposeHeader>ETag</ExposeHeader> to my CORS configuration at Amazon S3 bucket permissions. So finally it looks similar to:

<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>POST</AllowedMethod>
        <AllowedMethod>PUT</AllowedMethod>
        <AllowedMethod>DELETE</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <ExposeHeader>ETag</ExposeHeader>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

Reference:

http://docs.fineuploader.com/endpoint_handlers/amazon-s3.html