I've got a simple Google apps script that is fetching a series of RSS feeds:
for (let i = 0; i < rssUrls.length; i++) {
const rssUrl = rssUrls[i];
const response = UrlFetchApp.fetch(rssUrl);
console.log(response.getContentText());
}
The URL that's failing is: http://tenfootpole.org/ironspike/?feed=rss2
The response I get is:
<html><head><title>Error 406 - Not Acceptable</title><head><body><h1>Error 406 - Not Acceptable</h1><p>Generally a 406 error is caused because a request has been blocked by Mod Security. If you believe that your request has been blocked by mistake please contact the web site owner.</p></body></html>
It works fine through the browser or POSTman - & even using the =IMPORTFEED() function in google sheets. Not sure what makes GAS unique?
This is the fix I made after reading Google App Script external API return error 406 & Google Apps Script Blocked by Mod Security Error 406 which seem related.
const options = {
headers: {
Accept: '*/*'
},
'muteHttpExceptions': true
};
const response = UrlFetchApp.fetch(rssUrl, options);
console.log(response.getContentText());
But I still get the same problem. I've tried various combinations of content type, encoding types and languages but I always the same error. I suspect I'm seeing a default response for something else? Any idea what?
I could correctly replicate your situation. Unfortunately, even when Accept, Accept-Encoding, and Accept-Language are used in the request header, the status code of 406 couldn't be removed. From this situation, I thought that your situation might be related to this thread. In this case, how about the following workaround?
In this workaround, when the response code is 406, the URL is requested using IMPORTFEED or IMPORTDATA on the Spreadsheet. Namely, a Spreadsheet is used as a temporal sheet for retrieving the data from the URL. By this, the values can be obtained.
This script used Spreadsheet. So, please copy and paste the following script to the script editor of Spreadsheet. And, please set the sheet name. This sheet is used as a temporal sheet.
function sample() {
const sheetName = "Sheet1"; // Please set your sheet name.
const rssUrls = ["http://tenfootpole.org/ironspike/?feed=rss2", , ,]; // Please set your URLs.
rssUrls.forEach(url => {
const response = UrlFetchApp.fetch(url, { muteHttpExceptions: true });
let res = "";
const statusCode = response.getResponseCode();
if (statusCode == 406) {
const sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(sheetName);
sheet.clear();
const range = sheet.getRange("A1");
range.setFormula(`=IMPORTFEED("${url}")`);
SpreadsheetApp.flush();
res = sheet.getDataRange().getDisplayValues().join("\n");
} else if (statusCode == 200) {
res = response.getContentText();
} else {
console.log(`Error occurs at "${url}".`);
return;
}
console.log(res); // Here, you can confirm the retrieved value.
// do something. Please set your script using the response value.
});
}
When this script is run, when the response code is 406, the value is retrieved using IMPORTFEED with a temporal sheet.
If you want to use IMPORTDATA instead of IMPORTFEED, please modify it as follows.
From
range.setFormula(`=IMPORTFEED("${url}")`);
SpreadsheetApp.flush();
res = sheet.getDataRange().getDisplayValues().join("\n");
To
range.setFormula(`=IMPORTDATA("${url}")`);
SpreadsheetApp.flush();
res = sheet.getDataRange().getDisplayValues().map(r => r.filter(String).join(",")).join("\n");