pythonauthenticationurllib2

Python - View bot?


I'm trying to write some code for viewbot.

Code:

import requests
from bs4 import BeautifulSoup
import html5lib
import urllib
import argparse, os, time
import urllib.parse, random

headers = {
    'user-agent':'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Mobile Safari/537.36'
    }
login_data = {
    'login': 'xxx',
    'pass': 'xxx',
    'back_url': '' 
    }

Login on site works correctly.

Lists people:

def getPeopleLinks(page):
    links = []
    for link in soups.find_all('a'):
        url = link.get('href')
        if url:
            if 'profile/' in url:
                links.append(url)
    return links

Working...

And other code:

with requests.Session() as session:
        url = "https://xxxxxx.com/Login/?form_login=1"

        post = session.post(url, data=login_data, headers=headers)
print (post.status_code)
print (post.cookies)
r = session.get("https://xxxxxxx.com/online/GIRL")
print (r.status_code)
print (r.cookies)
soups = BeautifulSoup(r.content, 'html5lib')
x = getPeopleLinks(soups)
print(x)
print("http://www.xxxxx.com"+ x[2])
for link in x: 
        urllib.request.urlopen("http://www.xxxxxxx.com"+link)
print(link)

Login: Works correctly.
List of online users: Works correctly; I get all list of profiles.

I think it's a problem here:

for link in x: 
        urllib.request.urlopen("http://www.xxxxxxx.com"+link)
print(link)

I was logged in on another account by mobile, my profile was on the list, but the bot on the PC did not view my profile.


Solution

  • It is because of the problem in the syntax of the url. Or maybe because of there might be an issue in this particular code, try this:

    import urllib.request
    for path in paths:
        url = 'http://example.com/view-online-profiles/' + path
        page = urllib.request.urlopen(url)
        print(page.read())
    

    Or you can also follow another way:

    import requests
    for path in paths:
        url = 'http://example.com/view-online-profiles/' + path
        page = requests.get(url)
        print(page) # Would return response object, can obtain status_code or body