xpathhttprequestmake.com

XPath inside HTTP request


I want to scrape daily value changes from public page:

[1] http://www.example.com/page.html

I've got a full xpath:

[2] /html/body/div[5]/table[1]/tbody/tr[4]/td[2]/@data-val

or command that works to get that value thru Chrome console:

[3] $x("string(/html/body/div[5]/table[1]/tbody/tr[4]/td[2]/@data-val)")

But i'm stuck how to make/encode [1] + [2]/[3], that could retrive that data-val using just a http request? (i'm using integromat, to make http request, but failed to find any reasonable examples).


Solution

  • You will have to make a get request to load the document. Afterwards you can use a library to extract the value by xpath.

    Please Provide more Info on which Language/Framework you are on. Here is an example in python for a refrence:

    from scrapy.selector import Selector
    from scrapy.http import HtmlResponse
    response = HtmlResponse(url='http://example.com', body=body)
    Selector(response=response).xpath('//span/text()').get()