I am learning Nodejs, and try to do web scraping with node.js
I am using node module request
and cheerio
but when I request the url it returns some symbol instead of the html
body
var request = require('request');
var cheerio = require('cheerio');
request({
url:"http://mangafox.me/manga/shingeki_no_kyojin/v00/c000/1.html"
},(err, res, body) => {
if(err) throw err;
else {
var $ = cheerio.load(body);
console.log(body);
}
});
output in command prompt
Can anyone please tell me What is the problem here?
Thank You
The problem is that the server is sending a compressed response, even though you aren't requesting a compressed response.
The easy fix is to just add gzip: true
to your request()
options, which will not only automatically decompress responses but will also send the appropriate Accept-Encoding
header to the server.