javascriptnode.jsdiscogs-api

How to change/remove/ignore a http header just before parsing the response in Node?


I'm currently working with Node to get data from an API (Discogs API) and I've got this error when requesting :

{ [Error: Parse Error] bytesParsed: 0, code: 'HPE_INVALID_CONSTANT' }

Using this link : http://api.discogs.com/releases/249504 (but I have the same error for every other requests when content-length != actual content length)

And this code :

var http  = require('http');

var options = {
  hostname: 'api.discogs.com',
  port: 80,
  path: '/releases/249504',
  method: 'GET'
};

var req = http.request(options, function(res) {
  console.log("statusCode: ", res.statusCode);
  console.log("headers: ", res.headers);

  res.on('data', function(d) {
    console.log(d);
  });
});
req.end();

req.on('error', function(e) {
  console.error(e);
});

I found that the Content-length value is always smaller than the real response byte length.

Content-length : 2142

Actual Byte Length : 7,734 Bytes (according to https://mothereff.in/byte-counter)

I've read that Node's Parser is really strict and that's why it fails parsing the response.

Finally, I'm asking you if there is a way to either ignore/modify/delete a Header just before Node parses the response, so the parser can do his work by ignoring the Content-Length ?


Solution

  • This isn't related to a (perceived, see below) invalid content-length, because it also doesn't work using cURL:

    $ curl  http://api.discogs.com/releases/249504
    curl: (52) Empty reply from server
    

    Apparently the API server requires that a user agent header is set:

    var options = {
      hostname : 'api.discogs.com',
      port     : 80,
      path     : '/releases/249504',
      method   : 'GET',
      headers  : { 'user-agent' : 'foo/1.0' }
    };
    

    As for the difference in content-length: 2142 is the size of a gzip-compressed response (content-encoding: gzip), 7734 is the size of an uncompressed response. Apparently, your byte-counter test requests uncompressed responses only but the client that you checked the header with is requesting a compressed response.