I am trying to learn scrapy for a project. I receive this error ValueError: XPath error: Invalid expression but I don't understand what is wrong in my script.
It's my script
def parse(self, response):
yield {
'user_agent': str(response.request.headers['User-Agent']),
'links' : response.xpath('//a[@class="sc-996f251d-0 leAMGT"]/@href').getall()
}
next = response.xpath('//a[@title="Page suivante"]/')
print(next)
Here is a piece of the scraped page:
<a class="indexstyles__PaginateItem-sc-8rtl5h-3 YJHOY" title="Page suivante" href="/locations_saisonnieres/offres/p-2">
It's the error:
ValueError: XPath error: Invalid expression in //a[@title="Page suivante"]/
It seems like there is an extra '/' in your Xpath that's why it's showing an error!
so instead of //a[@title="Page suivante"]/
use //a[@title="Page suivante"]
and That should work!
def parse(self, response):
yield {
'user_agent': str(response.request.headers['User-Agent']),
'links' : response.xpath('//a[@class="sc-996f251d-0 leAMGT"]/@href').getall()
}
next = response.xpath('//a[@title="Page suivante"]/')
print(next)