pythonweb-scrapinghome-assistant

Python web scraping script does not find element by css selector

I'm trying to get this web scraper to get current electricity price from this website, it's in finnish but it's right under "Hinta nyt". https://sahko.tk/

Here's my code:

import requests
from bs4 import BeautifulSoup

url = "https://sahko.tk/"

element_selector = ""
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
elements = soup.find_all(element_selector)

if len(elements) == 0:
    print("No element found with selector '%s'" % element_selector)
else:
    element_text = elements[0].text
    print(element_text)

I left the element_selector to empty because what ever I tried it just did not work. I'm not even sure if I'm on the right tracks.

Solution

  • The data you see is embedded inside <script> in that page. To parse the current price you can use next example:

    import re
import json
import requests

url = "https://sahko.tk/"

data = requests.get(url).text

data = re.search(r"function prices_today\(\)\{var t= (.*?});", data).group(1)
data = json.loads(data)

print("Hinta nyt", data["now"], "snt/kWh")

    Prints:

    Hinta nyt 33.27 snt/kWh