I'm trying to send an HTTP request to a website using the requests module, expecting to get a 200 status code. However, when I run the script, I always receive an error, which I will paste below. I've added the headers, but I still haven't received any response.
import requests
link = 'https://apps.hcr.ny.gov/BuildingSearch/'
headers = {
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
'accept-language': 'en-US,en;q=0.9',
'host': 'apps.hcr.ny.gov',
'origin': 'https://apps.hcr.ny.gov',
'referer': 'https://apps.hcr.ny.gov/BuildingSearch/',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36'
}
with requests.Session() as s:
s.headers.update(headers)
res = s.get(link,headers=headers)
print(res.status_code)
I receive the following error:
File "C:\Users\CL\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\adapters.py", line 682, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
The website you're accessing is not RFC 7230 compliant. The server appears to be Microsoft-IIS/10.0
HTTP headers should be treated as case-insensitive.
Here's the proof:
import requests
import json
URL = 'https://apps.hcr.ny.gov/BuildingSearch/'
UA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.15"
for key in "user-agent", "User-Agent":
headers = {
key: UA
}
print("\nTrying with:", json.dumps(headers, indent=2))
try:
with requests.get(URL, headers=headers) as response:
response.raise_for_status()
print("Success")
except Exception:
print("Failed")
Output:
Trying with: {
"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.15"
}
Failed
Trying with: {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.15"
}
Success