i am trying to download a .torrent file from a password protected site. I have managed to get to the site using cookies like so:
cookies = {'uid': '232323', 'pass': '31321231jh12j3hj213hj213hk',
'__cfduid': 'kj123kj21kj31k23jkl21j321j3kl213kl21j3'}
try:
# read site content
read = requests.get(s_string, cookies=cookies).content
except RequestException as e:
raise print('Could not connect to somesite: %s' % e)
soup = BeautifulSoup(read, 'html.parser')
With this above code i get access to the site and scrape the data i need. With the scraped data i build a link to a .torrent file, which i then want to download, but this is where i am stuck.
Here is what im trying right now: (cookie data not real obviously, like its not in above code either)
cookies = {'uid': '232323', 'pass': '31321231jh12j3hj213hj213hk',
'__cfduid': 'kj123kj21kj31k23jkl21j321j3kl213kl21j3'}
# construct download URL
torrent_url = ('https://www.somesite.com/' + torrent_url)
# for testing purposes DELETE!
print('torrent link:', torrent_url)
# download torrent file into a folder
filename = torrent_url.split('/')[-1]
save_as = 'torrents/' + filename + '.torrent'
try:
r = request.urlretrieve(torrent_url, save_as, data=cookies)
print("Download successful for: " + filename)
except request.URLError as e:
raise print("Error :%s" % e)
This code would work without the cookies on a normal site, but this .torrent file im trying to get is behind a passworded/captchaed site, so i need to use cookies to scrape it.
So question is, what am i doing wrong here? without data=cookies
i get http 404 error
and with the data=cookies
i get the following error:
File "/usr/lib/python3.6/http/client.py", line 1064, in _send_output
+ b'\r\n'
TypeError: can't concat str to bytes </error>
ps. before anyone asks, yes im 100% sure the torrent_url is correct, i have it printed and manually copy/pasting it into my own browser promps the download window for the .torrent file in question
EDIT:
try:
read = requests.session().get(torrent_url)
with open(save_as, 'wb') as w:
for chunk in read.iter_content(chunk_size=1024):
if chunk:
w.write(chunk)
w.close()
print("Download successful for: " + filename)
except request.URLError as e:
print("Error :%s" % e)
made this based on furas's suggestion, it works now, but when i try to open the .torrent, torrent client says "invalid coding, cannot open".
When i open the .torrent file, inside is this:
<h1>Not Found</h1>
<p>Sorry pal :(</p>
<script src="/cdn-cgi/apps/head/o1wasdM-xsd3-9gm7FQY.js"></script>
am i still doing something wrong or has this something to do with the site owner preventing programs from downloading .torrents from his site or something of that nature?
This works, but not ideal i think.
cookies = {'uid': '232323', 'pass': '31321231jh12j3hj213hj213hk',
'__cfduid': 'kj123kj21kj31k23jkl21j321j3kl213kl21j3'}
try:
read = requests.get(torrent_url, cookies=cookies)
with open(save_as, 'wb') as w:
for chunk in read.iter_content(chunk_size=512):
if chunk:
w.write(chunk)
print(filename + ' downloaded successfully!!!')
except request.URLError as e:
print("Error :%s" % e)