HTTP get Timeout

Posted 07/03/2006 by dnaffis 0 CommentsAdd Your Comment

One of my apps does quite a few http requests (using Net::HTTP.get) for pages from another site.

I kept running into timeout problems that I initially assumed were http timeouts since it took up to 5 minutes to run sometimes. After banging my head against it for a while I finally came up with a solution for handling slow responding or non-responsive websites. Basically it catches the timeout error and reattempts the request. In my case I don’t care about the exception raised except to make a new request, but you might want to keep track of it and raise it at the end to do something with it.

# retrieve a page
  def self.get_page(url)
    retrycount = 0
    resp = nil
    begin
      timeout(60) do
        resp = Net::HTTP.get(URI.parse(url))    
        resp.to_s    
      end
    rescue TimeoutError      
      if(retrycount < 5)
        retrycount+=1
        retry
      else
        logger.info("ERROR url: " + url)
        logger.info("ERROR Timeout error in get_page, attempt #" + retrycount.to_s)
        nil
      end
    end
    resp.to_s    
  rescue Exception => exception
    logger.info("ERROR Unknown error in get_page")
    logger.info(exception.class.to_s + " " + exception.message.to_s + " " + exception.backtrace.to_s) 
    nil
  end

This example will retry 5 times before giving up but you can change yours to attempt retries for specific cases.

Tagged Under:

Leave a Reply

Dave is the cofounder of Intridea and leads Intridea's product development efforts.

Before Intridea, Dave spent years at both AOL and IMAKE and received a Masters in Systems Engineering from the University of Virginia.