UnhandledPromiseRejectionWarning: FetchError: invalid json response body at

Error im getting:

UnhandledPromiseRejectionWarning: FetchError: invalid json response body at {url}
reason: Unexpected token < in JSON at position 0

my code:

const fetch = require('node-fetch');

const url = 'Reeeealy long url here';

fetch(url)
  .then(res => res.json())
  .then(console.log);

thing is if url longer than ~8k+ characters api returns

400 Bad Request
Request Header Or Cookie Too Large
nginx

obviously i don't control that api.

what can I do to prevent that?

url structure:

1) domain

2) api version

3) endpoint

4) request stuff (longest part)

5) id at the end

look like this: https://example.com/v1/endpoint/query?query=long_part_here&ids=2145132,532532,535

Solution

It sounds like a poorly designed api if it's expected that the 'long_part' is very long. Instead of a a GET request it should use POST so that the long set of data can be sent in a body object. Can you see if the API allows a POST version of the endpoint that allows that?

If no POST is available, and you don't control the API, you don't have a lot of options. The only thing I can think of is that you break your request into multiple separate endpoint calls (maybe one per id) if that is feasible and would result in shorter url sizes per request.

Multiple calls

If you're able to do multiple smaller requests, the code might look like this:

const urls = ["firstUrl","secondUrl","nthUrl"];
let combined = {};
for (const url of urls) {
  fetch(url)
    .then(res => res.json())
    .then(json => combined = {...combined, ...json};
}
console.log(combined);

This assumes that it's reasonable to merge the results all into one object. If they should be kept distinct, you could change the last then like this:

.then(json => combined = {...combined, {`url${count}`: json}};

where count is an integer that you increment each time and combined would look like

{url1: {/*json from url1*/}, url2: {/*json from url2*/}, ...}

Error Handling

To handle the error more gracefully you should check the result before assuming it's returned JSON. You got a JSON parse error because the data returned was not JSON. It was HTML so failed when it started with <. You could do something like this:

fetch(url)
  .then(res => {
    if (res.resultCode == "200") return res.json();
    return Promise.reject(`Bad call: ${res.resultCode}`);
  })
  .then(console.log);