ajaxangularjsfile-uploadamazon-s3singlepage

Direct (and simple!) AJAX upload to AWS S3 from (AngularJS) Single Page App


I know there's been a lot of coverage on upload to AWS S3. However, I've been struggling with this for about 24 hours now and I have not found any answer that fits my situation.

What I'm trying to do

Upload a file to AWS S3 directly from my client to my S3 bucket. The situation is:

  1. It's a Single Page App, so upload request must be in AJAX
  2. My server and my client are not on the same domain
  3. The S3 bucket is of the newest sort (Frankfurt), for which some signature-generating libraries don't work (see below)
  4. Client is in AngularJS
  5. Server is in ExpressJS

What I've tried

It seems I'm facing 2 difficulties: having a file input that works in my Single Page App, and getting AWS's workflow right.

The kind of solution I'm looking for

If possible, I'd like to avoid 'all included' solutions that manage the whole process while hiding of all of the complexity, making it hard to adapt to special cases. I'd much rather have a simple explanation breaking down the flow of data between the various components involved, even if it requires some more plumbing from me.


Solution

  • I finally managed. The key points were:

    1. Let go of Angular's $http, and use native XMLHttpRequest instead.
    2. Use the getSignedUrl feature of AWS's SDK, instead on implementing my own signature-generating workflow like many libraries do.
    3. Set the AWS configuration to use the proper signature version (v4 at the time of writing) and region ('eu-central-1' in the case of Frankfurt).

    Here below is a step-by-step guide of what I did; it uses AngularJS and NodeJS on the server, but should be rather easy to adapt to other stacks, especially because it deals with the most pathological cases (SPA on a different domain that the server, with a bucket in a recent - at the time of writing - region).


    Workflow summary

    1. The user selects a file in the browser; your JavaScript keeps a reference to it.
    2. the client sends a request to your server to obtain a signed upload URL.
    3. Your server chooses a name for the object to put in the bucket (make sure to avoid name collisions!).
    4. The server obtains a signed URL for your object using the AWS SDK, and sends it back to the client. This involves the object's name and the AWS credentials.
    5. Given the file and the signed URL, the client sends a PUT request directly to your S3 Bucket.

    Before you start

    Make sure that:

    Step 1: setup a SPA-friendly file upload form / widget.

    All that matters is to have a workflow that eventually gives you programmatic access to a File object - without uploading it.

    In my case, I used the ng-file-select and ng-file-drop directives of the excellent angular-file-upload library. But there are other ways of doing it (see this post for example.).

    Note that you can access useful information in your file object such as file.name, file.type etc.

    Step 2: Get a signed URL for the file on your server

    On your server, you can use the AWS SDK to obtain a secure, temporary URL to PUT your file from someplace else (like your frontend).

    In NodeJS, I did it this way:

    // ---------------------------------
    // some initial configuration
    var aws = require('aws-sdk');
    
    aws.config.update({
      accessKeyId: process.env.AWS_ACCESS_KEY,
      secretAccessKey: process.env.AWS_SECRET_KEY,
      signatureVersion: 'v4',
      region: 'eu-central-1'
    });
    
    // ---------------------------------
    // now say you want fetch a URL for an object named `objectName`
    var s3 = new aws.S3();
    var s3_params = {
      Bucket: MY_BUCKET_NAME,
      Key: objectName,
      Expires: 60,
      ACL: 'public-read'
    };
    s3.getSignedUrl('putObject', s3_params, function (err, signedUrl) {
      // send signedUrl back to client
      // [...]
    });
    

    You'll probably want to know the URL to GET your object to (typically if it's an image). To do this, I simply removed the query string from the URL:

    var url = require('url');
    // ...
    var parsedUrl = url.parse(signedUrl);
    parsedUrl.search = null;
    var objectUrl = url.format(parsedUrl);
    

    Step 3: send the PUT request from the client

    Now that your client has your File object and the signed URL, it can send the PUT request to S3. My advice in Angular's case is to just use XMLHttpRequest instead of the $http service:

    var signedUrl, file;
    // ...
    var d_completed = $q.defer(); // since I'm working with Angular, I use $q for asynchronous control flow, but it's not mandatory
    var xhr = new XMLHttpRequest();
    xhr.file = file; // not necessary if you create scopes like this
    
    xhr.onreadystatechange = function(e) {
      if ( 4 == this.readyState ) {
        // done uploading! HURRAY!
        d_completed.resolve(true);
      }
    };
    xhr.open('PUT', signedUrl, true);
    xhr.setRequestHeader("Content-Type","application/octet-stream");
    xhr.send(file);
    

    Acknowledgements

    I would like to thank emil10001 and Will Webberley, whose publications were very valuable to me for this issue.