javascriptnode.jsrequest-promisenode-fetch

How to make sure headers that im passing to node-fetch request are correct?


Im trying to rewrite one function which does the get request with request-promise library to be based on node-fetch:

Here is method in request-promise:

import { default as request } from 'request-promise';

async function sendGet(cookies, url) {
  const options = {
    method: 'GET',
    uri: url,
    headers: {
      Accept: 'text/html,application/xhtml+xml,application/xml',
      'Upgrade-Insecure-Requests': '1',
      'Accept-Encoding': 'gzip, deflate, sdch, br',
      'Accept-Language': 'en-US,en;q=0.8',
      Host: 'some.url.com',
      Connection: 'keep-alive',
      cookie: cookies,
    },
    json: true,
    resolveWithFullResponse: true,
    simple: false,
    followRedirect: false,
  };
  try {
    const response = await request(getNextOptions);
    return {
      cookies: response.headers['set-cookie'],
      location: response.headers['location'],
    };
  } catch (e) {
    throw e;
  }
}

When changed to node-fetch it gives following error in response:

"The requested URL /session/auth/failure was not found on this server. Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request."

import fetch from 'node-fetch';

async function sendGet(cookies, url) {
  const headers = {
    'Upgrade-Insecure-Requests': '1',
    'Accept-Language': 'en-US,en;q=0.8',
    Host: 'some.url.com',
    Connection: 'keep-alive',
    cookie: cookies,
  }
  try {
    const result = await fetch(url, {
      headers,
      compress: true
    });
    const body = await result.json();

    return {
      cookies: body.headers.raw()['set-cookie'],
      location: body.headers.get('location'),
    };
  } catch (e) {
    throw e;
  }

I think there might be something wrong with headers that Im passing but after couple of days of research I was unable to get it to work. Some explanation in difference of headers:

  1. Im not using Accept header because node-fetch by default adds Accept: */*
  2. Instead of 'Accept-Encoding': 'gzip, deflate, sdch, br' im using compression: true

Thanks in advance for help!


Solution

  • You can use something like https://requestbin.com/ to test your request.

    Basically, you create new Request BIN, and then use the BIN url instead of your real url, and see what you are requesting there