rubyweb-scrapinghttparty

DEPRECATION: HTTParty will no longer override `response#nil?`. What does this Deprecation warning mean?


This question is probably poorly structured so please bear with me, I'm new at this.

I'm trying to build a simple web scraper but every time i run my code i get this warning in terminal. I have tried to follow the link to the github issues hoping i would get a clearer explanation but i didn't understand it there either. Tried googling, but nothing there either.

[DEPRECATION] HTTParty will no longer override `response#nil?`.
This functionality will be removed in future versions.
Please, add explicit check `response.body.nil? || response.body.empty?`.
For more info refer to: https://github.com/jnunemaker/httparty/issues/568

I guess what i want to know is, in basic terms, what does this warning mean? and is there something i should do different from now on when using the HTTParty gem?


Solution

  • This was annoying me as well. You are probably passing the response from HTTParty.get to some other code that is testing for nil. I was passing the body to Nokogiri::HTML and that code was eventually calling HTTParty.response.nil?. To fix, I started sending the "body" to Nokogiri instead of the response object. Quite a few people will probably run into this as the Nokogiri example code with this pattern is all over the place.

    I now use this:

    doc = HTTParty.get('https://some/site')
    parsed ||= Nokogiri::HTML(doc.body)
    

    instead of before when getting the warning it was:

    doc = HTTParty.get('https://some/site')
    parsed ||= Nokogiri::HTML(doc)