javascriptnode.jshttphttps

POST call using https.request returns 408


An overview on what the code is supposed to do:

I created a simple http request reverse proxy using express to to direct http/https requests to on-premise system for testing the application locally (connects to on-premise using VPN in local environment).

HTTP/s.request function implementation:

/**
 * Does HTTP/HTTPS request and returns a promise.
 * @param {{}} config Config object for the request.
 * @param {Buffer|String|{}} data Data to be sent in the request.
 * @param {Number} retries Number of retries to be done if request fails defualts to request_retries variable, @see{@link {request_retries}}.
 * @returns {Promise.<{status: Number|undefined,status_message:String|undefined,headers:http.IncomingHttpHeaders,body:any}>} resolve with  response or rejects with error.
 */
const request = (config, data = null, retries = request_retries) => {
  // eslint-disable-next-line no-unused-vars
  return new Promise((resolve, reject) => {
    try {
      // get the protocol agent
      const _agent = config.protocol === "https" ? https : http;

      delete config.protocol;

      let _data = [];
      for (let i = 0; i < retries; i++) {
        const req = _agent.request(config, (res) => {
          // collect the data
          res.on("data", (chunk) => {
            _data.push(chunk);
          });

          // create the response
          res.on("end", async () => {
            let _body;

            try {
              if (res.headers["content-type"]?.includes("application/json")) {
                // remove any JSON SYNTAXERROR wrt illegal whitespace.
                // _data = _data.map((chunk) => chunk.toString().replace(/[\n\r\s\t]+/g, " "));
                _body = JSON.parse(Buffer.concat(_data).toString("utf-8"));
              }
              //parse html for content type text/html
              else if (res.headers["content-type"]?.includes("text/html")) {
                _body = Buffer.concat(_data).toString("utf-8");
              } else {
                // check if header has encoding and use that to decode the buffer.
                _body = Buffer.concat(_data).toString(res.headers["content-encoding"] ?? "utf-8");
              }
            } catch (err) {
              _body = Buffer.concat(_data).toString();
            }

            const response = {
              status: res.statusCode,
              status_message: res.statusMessage,
              headers: res.headers,
              body: _body,
            };

            // check the condition for resolving the promise.
            if (res.statusCode >= 200 && res.statusCode < 300) {
              resolve(response);
            } else if (do_retry) {
              console.warn(`[${new Date().toISOString()}][PROXY] ${config.method} request to ${config.hostname ?? config.host + config.path} failed with status ${res.statusCode}.\nretrying...`);
              //call backoff and retry the request.
              await backoff(i);
            } else if (i === retries - 1) {
              resolve(response);
            } else {
              resolve(response);
            }
          });

          // timeout handler
          res.on("timeout", () => {
            reject(new Error(`Request to ${config.hostname ?? config.host + config.path} timed out.`));
          });

          res.on("error", (err) => {
            reject(err);
          });
        });

        // send the data depending on the type of data.
        if (data && data instanceof Buffer) {
          req.write(data);
        } else if (data && typeof data === "string") {
          req.write(data, "utf-8");
        } else if (data && typeof data === "object") {
          req.write(JSON.stringify(data), "utf-8");
        }
        // close the request
        req.end();
      }
    } catch (err) {
      reject(err);
    }
  });
};

Problem:

Able to successful do GET and HEAD calls to the on premise system, But with POST calls it fails with 408.

But when I try to do the same POST call using postman "directly", I do get a 201 status for the request.

So I tried to use postman to do the POST call "via" the proxy, to test if headers is missing and causing an 408 but on postman its a successful request with 201.

So I am thinking the request function is not implemented properly.

whole code here


Solution

  • Worth checking that the content-length header is consistent with the actual size of your payload (the one you send to the upstream server).

    If the numbers don't match and the context-length header is greater than the actual content of the payload, the server might be stuck waiting for bytes that will never get delivered. Hence the timeout response.