I'm trying to scrape the number of followers of an array of username. I'm using BeautifulSoup.
The code I'm using is the following
import requests
from bs4 import BeautifulSoup
def instagram_followers(username):
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'
}
response = requests.get(f'https://www.instagram.com/{username}/')
soup = BeautifulSoup(response.text, 'html.parser')
info = soup.find('meta', property='og:description')
if info:
followers = info['content'].split(" ")[0]
return followers
else:
return -1
The function always returns -1
Code works fine in focus of your question, so issue is not reproducable, without any additional information.
Check following:
response.status_code
as first indicator, may you scrape to aggressivly and the server will handle this by blocking your ip.headers
, they are not used in your codeimport requests
from bs4 import BeautifulSoup
def instagram_followers(username):
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'
}
response = requests.get(f'https://www.instagram.com/{username}/', headers=headers)
if response.status_code == 200:
soup = BeautifulSoup(response.text, 'html.parser')
info = soup.find('meta', property='og:description')
if info:
followers = info['content'].split(" ")[0]
return followers
else:
return -1
else:
print('Something went wrong with your request: ' + response.status_code)
instagram_followers('thestackoverflow')
Output:
52K