seleniumbeautifulsoupsplinter

Beautiful Soup - get text from all <li> elements in <ul>


With this code:

match_url = f'https://interativos.globoesporte.globo.com/cartola-fc/mais-escalados/mais-escalados-do-cartola-fc'

browser.visit(match_url)
browser.find_by_tag('li[class="historico-rodadas__rodada historico-rodadas__rodada--ativa"]').click()

soup = BeautifulSoup(browser.html, 'html.parser')
innerContent = soup.findAll('ul',class_="field__players")

print (innerContent)

I've managed to fetch the <ul>:

[<ul class="field__players"><li class="player"...] 

enter image description here

Now how can I access text for player__name and player__value for all players in the list?


Solution

  • This should help u:

    from selenium import webdriver
    from bs4 import BeautifulSoup
    
    driver = webdriver.Chrome()
    
    driver.get('https://interativos.globoesporte.globo.com/cartola-fc/mais-escalados/mais-escalados-do-cartola-fc')
    
    src = driver.page_source
    
    driver.close()
    
    soup = BeautifulSoup(src,'html5lib')
    
    innerContent = soup.find('ul',class_="field__players")
    
    li_items = innerContent.find_all('li')
    
    for li in li_items:
        p_tags = li.find_all('p')[:-1] #The [:-1] removes the last p tag from the list, which is player__label
    
        for p in p_tags:
            print(p.text)
    

    Output:

    Keno
    2.868.755
    Pedro
    2.483.069
    Bruno Henrique
    1.686.894
    Hugo Souza
    809.186
    Guilherme Arana
    1.314.769
    Filipe Luís
    776.147
    Thiago Galhardo
    2.696.853
    Vinícius
    1.405.012
    Nenê
    1.369.209
    Jorge Sampaoli
    1.255.731
    Réver
    1.505.522
    Víctor Cuesta
    1.220.451