pythonpython-3.xbeautifulsoup

Find all "a" tags in multiple divs with same class with BeautifulSoup


I want to find all "a" elemnts in multiple divs with same class.

from bs4 import BeautifulSoup
links = soup.find_all("div", class_="va-columns").find_all("a")

but this doesnt work and gives me an error. Can someone help me? Im trying to find all links on the main content of a website.


Solution

  • soup.find_all("div") will return a list. So you could simply loop through that list and for each div, you do div.find_all('a'). This way, you could have a list of all a tags in all div tags you wanted to search for.

    Here's the code.

    from bs4 import BeautifulSoup
    links = [div.find_all("a") for div in soup.find_all("div", class_="va-columns")]
    

    See this if you didn't get the for loop inside the list. (it's called a list comprehension in Python)