rubyrsscurbfeedjira

Curb doesn't respond

I parse RSS stream with Feedjira. When I used a fetch_and_parse method it sometimes blocked and doesn't respond.

The same thing happens with manual curb downloading.

I write in a loop:

@my_logger.info "--- Before perform ---"
easy = Curl::Easy.new
easy.follow_location = true
easy.max_redirects = 3 
easy.connect_timeout = 120
easy.url = url
easy.useragent = "Ruby/Curb"
easy.perform
@my_logger.info "--- After perform ---"
doc = easy.body_str
easy.close

After some time (it may be a day or an hour), process stops on the easy.perform line and doesn't respond. E.g. process outputs --- Before perform --- and nothing else.

Solution

  • It can be related to a network issue happening randomly.

    If you use a timeout you can skip this kind of situations in long running tasks.

    require 'timeout'
begin
  Timeout.timeout(5) do      
    easy.perform
  end
rescue Timeout::Error
  puts 'timeout'
end