python-3.xweb-scrapingtextsteamworks-apisteambot

Scraping the data out of the text


I am working on a price checker app for steamcommunity market. I have used the following code to extract the source-code out of the website, which includes all of the sales that has been made untill today. Can you please help me to get the data, which is between "[[]]" signs?

import requests
sites = [
    "https://steamcommunity.com/market/listings/730/AK-47%20%7C%20Redline%20%28Field-Tested%29"
]
for url in sites:
    r = requests.get(url)
    page_source = r.text
    page_source = page_source.split('\n')
    print("\nURL:", url) 
    
    for row in page_source[:]:
        print(row)

Solution

  • I used regex to extract the data

    import requests
    import re
    import json
    
    sites = [
        "https://steamcommunity.com/market/listings/730/AK-47%20%7C%20Redline%20%28Field-Tested%29"
    ]
    for url in sites:
        r = requests.get(url)
        page_source = r.text
        # print(page_source)
        results = re.search(r'var line1=\[.*\]',page_source).group()
        print(results[10:])