I'm just trying to do a telegram bot which sends me updates about the floor price of a NFT project. I tried with BeautifulSoup to scrape the floor price with the following code:
from urllib.request import urlopen
from bs4 import BeautifulSoup
import requests
import urllib.request
url = "https://magiceden.io/marketplace/bulldog_billionaires"
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
result = soup.find("div", {"p-3 bg-color-third d-flex flex-column border-radius-8px h-100 position-relative attributes-main"})
print(result.title)
But it always gives me the following error: "AttributeError: 'NoneType' object has no attribute 'title'"
Can anybode help me to solves this problem? I have to keep a eye on this NFT project so I need this bot!
I thank everyone!
This webpage is dinamically loaded so your get request won't return many of the data on the page. So when you try to find the specified data with beautifulsoup it won't find it and that's why it raises an Attribute Error.
To get around this you can use a webdriver like selenium:
from selenium import webdriver
from bs4 import BeautifulSoup
url = "https://magiceden.io/marketplace/bulldog_billionaires"
driver = webdriver.Firefox()
driver.get(url)
page = driver.page_source
soup = BeautifulSoup(page, "html.parser")
result = soup.find("div", {"p-3 bg-color-third d-flex flex-column border-radius-8px h-100 position-relative attributes-main"})
This is the documentation: https://selenium-python.readthedocs.io