How to read JSON error response from $http if responseType is arraybuffer

I load some binary data using

$http.post(url, data, { responseType: "arraybuffer" }).success(
            function (data) { /*  */ });

In case of an error, the server responds with an error JSON object like

{ "message" : "something went wrong!" }

Is there any way to get the error response in a different type than a success response?

$http.post(url, data, { responseType: "arraybuffer" })
  .success(function (data) { /*  */ })
  .error(function (data) { /* how to access data.message ??? */ })

Edit: As @Paul LeBeau points out, my answer assumes that the response is ASCII encoded.

Basically you just need to decode the ArrayBuffer into a string and use JSON.parse().

var decodedString = String.fromCharCode.apply(null, new Uint8Array(data));
var obj = JSON.parse(decodedString);
var message = obj['message'];

I ran tests in IE11 & Chrome and this works just fine.