I created a simple http request reverse proxy using express to to direct http/https requests to on-premise system for testing the application locally (connects to on-premise using VPN in local environment).
utf-8
, So I don't need to use any package to decode the incoming body (eg. gzip).backoff
function is a simple exponential function if do_retry
flag is enabled, which is not for testing scenario./**
* Does HTTP/HTTPS request and returns a promise.
* @param {{}} config Config object for the request.
* @param {Buffer|String|{}} data Data to be sent in the request.
* @param {Number} retries Number of retries to be done if request fails defualts to request_retries variable, @see{@link {request_retries}}.
* @returns {Promise.<{status: Number|undefined,status_message:String|undefined,headers:http.IncomingHttpHeaders,body:any}>} resolve with response or rejects with error.
*/
const request = (config, data = null, retries = request_retries) => {
// eslint-disable-next-line no-unused-vars
return new Promise((resolve, reject) => {
try {
// get the protocol agent
const _agent = config.protocol === "https" ? https : http;
delete config.protocol;
let _data = [];
for (let i = 0; i < retries; i++) {
const req = _agent.request(config, (res) => {
// collect the data
res.on("data", (chunk) => {
_data.push(chunk);
});
// create the response
res.on("end", async () => {
let _body;
try {
if (res.headers["content-type"]?.includes("application/json")) {
// remove any JSON SYNTAXERROR wrt illegal whitespace.
// _data = _data.map((chunk) => chunk.toString().replace(/[\n\r\s\t]+/g, " "));
_body = JSON.parse(Buffer.concat(_data).toString("utf-8"));
}
//parse html for content type text/html
else if (res.headers["content-type"]?.includes("text/html")) {
_body = Buffer.concat(_data).toString("utf-8");
} else {
// check if header has encoding and use that to decode the buffer.
_body = Buffer.concat(_data).toString(res.headers["content-encoding"] ?? "utf-8");
}
} catch (err) {
_body = Buffer.concat(_data).toString();
}
const response = {
status: res.statusCode,
status_message: res.statusMessage,
headers: res.headers,
body: _body,
};
// check the condition for resolving the promise.
if (res.statusCode >= 200 && res.statusCode < 300) {
resolve(response);
} else if (do_retry) {
console.warn(`[${new Date().toISOString()}][PROXY] ${config.method} request to ${config.hostname ?? config.host + config.path} failed with status ${res.statusCode}.\nretrying...`);
//call backoff and retry the request.
await backoff(i);
} else if (i === retries - 1) {
resolve(response);
} else {
resolve(response);
}
});
// timeout handler
res.on("timeout", () => {
reject(new Error(`Request to ${config.hostname ?? config.host + config.path} timed out.`));
});
res.on("error", (err) => {
reject(err);
});
});
// send the data depending on the type of data.
if (data && data instanceof Buffer) {
req.write(data);
} else if (data && typeof data === "string") {
req.write(data, "utf-8");
} else if (data && typeof data === "object") {
req.write(JSON.stringify(data), "utf-8");
}
// close the request
req.end();
}
} catch (err) {
reject(err);
}
});
};
Able to successful do GET
and HEAD
calls to the on premise system, But with POST
calls it fails with 408
.
But when I try to do the same
POST
call using postman "directly", I do get a201
status for the request.
So I tried to use postman to do the POST
call "via" the proxy, to test if headers is missing and causing an 408
but on postman its a successful request with 201.
So I am thinking the request function is not implemented properly.
Worth checking that the content-length header is consistent with the actual size of your payload (the one you send to the upstream server).
If the numbers don't match and the context-length header is greater than the actual content of the payload, the server might be stuck waiting for bytes that will never get delivered. Hence the timeout response.