I am trying to limit the bandwidth (download / upload speed) that a HTTP request uses. I'm using the NPM package stream-throttle. I created a custom HTTP agent to pipe the socket through an instance of Throttle and have timed the speed at which a 5MB file downloads.
const http = require("http");
const net = require("net");
const {Throttle, ThrottleGroup} = require("stream-throttle");
const maxBandwidth = 100;
// an example 5MB file of random data
const str = "http://212.183.159.230/5MB.zip";
// this pipes an instance of Throttle
class SlowAgent extends http.Agent {
createConnection(options, callback){
const socket = new net.Socket(options);
socket.pipe(new Throttle({rate: maxBandwidth}));
socket.connect(options);
return socket;
}
}
const options = {
// this should slow down the request
agent: new SlowAgent()
};
const time = Date.now();
const req = http.request(str, options, (res) => {
res.on("data", () => {
});
res.on('end', () => {
console.log("Done! Elapsed time: " + (Date.now() - time) + "ms");
});
});
req.on('error', (e) => {
console.error(`problem with request: ${e.message}`);
});
req.on("end", () => {
console.log("done");
});
console.log("Request started");
req.end();
Regardless of the value of maxBandwidth
or whether SlowAgent
is used at all (I have tried commenting out agent: new SlowAgent()
), I notice no difference in the elapsed time (about 4000 milliseconds). How can I fix my SlowAgent class? Do I not understand socket.pipe
? Or is there something else I need to do?
freakish pointed out to change SlowAgent to be this:
// this pipes an instance of Throttle
class SlowAgent extends http.Agent {
createConnection(options, callback){
const socket = new net.Socket(options);
socket.connect(options);
return socket.pipe(new Throttle({rate: 10}));
}
}
but that causes this problem:
problem with request: Parse Error: Expected HTTP/ Error: Parse Error: Expected HTTP/
at Throttle.socketOnData (_http_client.js:456:22)
at Throttle.emit (events.js:209:13)
at addChunk (_stream_readable.js:305:12)
at readableAddChunk (_stream_readable.js:286:11)
at Throttle.Readable.push (_stream_readable.js:220:10)
at Throttle.Transform.push (_stream_transform.js:150:32)
at /home/max/Documents/Personal/node-projects/proxy/node_modules/stream-throttle/src/throttle.js:37:14
at processTicksAndRejections (internal/process/task_queues.js:75:11) {
bytesParsed: 0,
code: 'HPE_INVALID_CONSTANT',
reason: 'Expected HTTP/',
rawPacket: <Buffer 47>
}
I manage to get it worked by doing away with a custom agent
and use createConnection
inside http.request
options:
const options = {
createConnection(options) {
const socket = new net.Socket();
return socket.connect({host: options.host, port: options.port});
},
hostname: "212.183.159.230",
path: "/5MB.zip"
};
const time = Date.now();
const req = http.request(options, (res) => {
res.pipe(new Throttle({rate: 200 * 1024}))
.on("data", (chunk) => {
console.log(chunk.length);
})
res.on("end", () => {
console.log("Done! Elapsed time: " + (Date.now() - time) + "ms");
});
});