pythonhtmlregexscreen-scrapinghtml-content-extraction

Python HTML scraping


It's not really scraping, I'm just trying to find the URLs in a web page where the class has a specific value. For example:

<a class="myClass" href="/url/7df028f508c4685ddf65987a0bd6f22e">

I want to get the href value. Any ideas on how to do this? Maybe regex? Could you post some example code? I'm guessing html scraping libs, such as BeautifulSoup, are a bit of overkill just for this...

Huge thanks!


Solution

  • Regex is usally a bad idea, try using BeautifulSoup

    Quick example:

    html = #get html
    soup = BeautifulSoup(html)
    links = soup.findAll('a', attrs={'class': 'myclass'})
    for link in links:
        #process link