pythonweb-scrapingpython-requestslxml.html

Getting empty list while using xpath with html.fromstring

I am trying to extract text from a webpage using below code. It is working fine for other websites but here i am getting empty list

import requests
from lxml import html

siteurl = 'https://clinicaltrials.gov/ct2/show/NCT03752268?cond=cancer&draw=2&rank=1'
rq = requests.get(siteurl)
get_soup = html.fromstring(rq.content)
name = get_soup.xpath('//*[@id="tab-body"]/div/div[7]/div[2]/div[3]/table/tbody/tr/td[1]//text()')
print(name)
Solution

  • try this xpath:

    //td[@headers="contactName"]//text()

    and for future, try to create a more readable "xpath" than picking the one from browser dev tools.