An unresponsive service is worse than a down one. It can tie up your entire system if not handled properly. All network requests should have a timeout.
Here’s how to add timeouts for popular Ruby gems. All have been tested. You should avoid Ruby’s Timeout
module. The default is no timeout, unless otherwise specified. Enjoy!
- connect (or open) - time to open the connection
- read (or receive) - time to receive data after connected
- write (or send) - time to send data after connected
- checkout - time to checkout a connection from the pool
- statement - time to execute a database statement
Data Stores
- activerecord
- bunny
- connection_pool
- dalli
- elasticsearch
- mongo
- mongoid
- mysql2
- pg
- redis
- rsolr
- searchkick
- sequel
HTTP Clients
- curb
- em-http-client
- excon
- faraday
- http
- httparty
- httpclient
- httpi
- net/http
- open-uri
- patron
- rest-client
- typhoeus
Web Servers
Rack Middleware
External Services
- actionmailer
- bitly
- firebase
- gibbon
- geocoder
- hipchat
- koala
- mechanize
- net-dns
- net/smtp
- net-ssh
- omniauth-oauth2
- slack-notifier
- stripe
- twilio-ruby
- zendesk_api
Bonus
-
ActiveRecord::Base.establish_connection(connect_timeout: 1, checkout_timeout: 1, ...)
or in
config/database.yml
production: connect_timeout: 1 checkout_timeout: 1
Raises
PG::ConnectionBad
on connect and read timeoutsActiveRecord::ConnectionTimeoutError
on checkout timeout
See also PostgreSQL statement timeouts
-
ActiveRecord::Base.establish_connection(connect_timeout: 1, read_timeout: 1, write_timeout: 1, checkout_timeout: 1, ...)
or in
config/database.yml
production: connect_timeout: 1 read_timeout: 1 write_timeout: 1 checkout_timeout: 1
Raises
Mysql2::Error
on connect and read timeoutsActiveRecord::ConnectionTimeoutError
on checkout timeout
See also MySQL statement timeouts
Bunny.new(connection_timeout: 1, read_timeout: 1, ...)
Raises
Bunny::TCPConnectionFailedForAllHosts
on connect timeoutBunny::NetworkFailure
on read timeout
ConnectionPool.new(timeout: 1) { ... }
Raises Timeout::Error
Dalli::Client.new(host, socket_timeout: 1, ...)
Default: 0.5s
Raises Dalli::RingError
Elasticsearch::Client.new(transport_options: {request: {timeout: 1}}, ...)
Raises
Faraday::ConnectionFailed
on connect timeoutFaraday::TimeoutError
on read timeout
Mongo::Client.new([host], connect_timeout: 1, socket_timeout: 1, server_selection_timeout: 1, ...)
Raises Mongo::Error::NoServerAvailable
production:
clients:
default:
options:
connect_timeout: 1
socket_timeout: 1
server_selection_timeout: 1
Raises Mongo::Error::NoServerAvailable
Mysql2::Client.new(connect_timeout: 1, read_timeout: 1, write_timeout: 1, ...)
Raises Mysql2::Error
PG.connect(connect_timeout: 1, ...)
Raises PG::ConnectionBad
Redis.new(connect_timeout: 1, timeout: 1, ...)
Default: 5s
Raises
Redis::CannotConnectError
on connect timeoutRedis::TimeoutError
on read timeout
RSolr.connect(open_timeout: 1, read_timeout: 1)
Raises
Net::OpenTimeout
on connect timeoutNet::ReadTimeout
on read timeout
Searchkick.timeout = 1
Searchkick.search_timeout = 1
Default: 10s
Raises same exceptions as elasticsearch
-
Sequel.connect(connect_timeout: 1, pool_timeout: 1, ...)
Sequel::DatabaseConnectionError
on connect and read timeoutsSequel::PoolTimeout
on checkout timeout
-
Sequel.connect(timeout: 1, read_timeout: 1, connect_timeout: 1, pool_timeout: 1, ...)
Raises
Sequel::DatabaseConnectionError
on connect and read timeoutsSequel::PoolTimeout
on checkout timeout
curl = Curl::Easy.new(url)
curl.connect_timeout = 1
curl.timeout = 1
curl.perform
Raises Curl::Err::TimeoutError
EventMachine.run do
http = EventMachine::HttpRequest.new(url, connect_timeout: 1, inactivity_timeout: 1).get
http.errback { http.error }
end
No exception is raised, but http.error
is set to Errno::ETIMEDOUT
in http.errback
.
Excon.get(url, connect_timeout: 1, read_timeout: 1, write_timeout: 1)
Raises Excon::Errors::Timeout
Faraday.get(url) do |req|
req.options.open_timeout = 1
req.options.timeout = 1
end
or
Faraday.new(url, request: {open_timeout: 1, timeout: 1}) do |faraday|
# ...
end
Raises
Faraday::ConnectionFailed
on connect timeoutFaraday::TimeoutError
on read timeout
HTTP.timeout(connect: 1, read: 1, write: 1).get(url)
Raises HTTP::TimeoutError
HTTParty.get(url, timeout: 1)
Raises
Net::OpenTimeout
on connect timeoutNet::ReadTimeout
on read timeout
client = HTTPClient.new
client.connect_timeout = 1
client.receive_timeout = 1
client.send_timeout = 1
client.get(url)
Raises
HTTPClient::ConnectTimeoutError
on connect timeoutHTTPClient::ReceiveTimeoutError
on read timeout
HTTPI::Request.new(url: url, open_timeout: 1)
Raises same errors as underlying client
Net::HTTP.start(host, port, open_timeout: 1, read_timeout: 1) do
# ...
end
Raises
Net::OpenTimeout
on connect timeoutNet::ReadTimeout
on read timeout
Note: Read timeouts are retried automatically
open(url, open_timeout: 1, read_timeout: 1)
Raises
Net::OpenTimeout
on connect timeoutNet::ReadTimeout
on read timeout
Note that open-uri didn't support (and so didn't pass to underlying net/http) the :open_timeout
argument until Ruby 2.2.
sess = Patron::Session.new
sess.connect_timeout = 1
sess.timeout = 1
Raises Patron::TimeoutError
RestClient::Request.execute(method: :get, url: url, open_timeout: 1, timeout: 1)
Raises RestClient::RequestTimeout
response = Typhoeus.get(url, connecttimeout: 1, timeout: 1)
No exception is raised. Check for a timeout with
response.timed_out?
# config/puma.rb
worker_timeout 15
Default: 30s
This kills and respawns the worker process. Note that this is for the worker and not threads. This isn’t a request timeout either. Use Rack middleware for request timeouts.
# config/puma.rb
worker_shutdown_timeout 8
Default: 60s
This causes Puma to send a SIGKILL signal to a worker if it hasn’t shutdown within the specified time period after having received a SIGTERM signal.
# config/unicorn.rb
timeout 15
Default: 60s
This kills and respawns the worker process.
It’s recommended to use this in addition to Rack middleware.
Rack::Timeout.timeout = 5
Rack::Timeout.wait_timeout = 5
Default: 15s service timeout, 30s wait timeout
Raises Rack::Timeout::RequestTimeoutError
or Rack::Timeout::RequestExpiryError
Slowpoke.timeout = 5
Default: 15s
Raises same exceptions as rack-timeout
Not configurable at the moment, and no timeout by default
Bitly.new(username, api_key, timeout)
Raises BitlyTimeout
firebase = Firebase::Client.new(url)
firebase.request.connect_timeout = 1
firebase.request.receive_timeout = 1
firebase.request.send_timeout = 1
Raises
HTTPClient::ConnectTimeoutError
on connect timeoutHTTPClient::ReceiveTimeoutError
on read timeout
Gibbon::Request.new(open_timeout: 1, timeout: 1, ...)
Raises Gibbon::MailChimpError
Geocoder.configure(timeout: 1, ...)
No exception is raised by default. To raise exceptions, use
Geocoder.configure(timeout: 1, always_raise: :all, ...)
Raises Timeout::Error
[HipChat::Client, HipChat::Room, HipChat::User].each { |c| c.default_timeout(1) }
Raises
Net::OpenTimeout
on connect timeoutNet::ReadTimeout
on read timeout
Koala.http_service.http_options = {request: {open_timeout: 1, timeout: 1}}
Raises
Faraday::ConnectionFailed
on connect timeoutFaraday::TimeoutError
on read timeout
Not configurable at the moment, and no timeout by default, but there is a pull request
agent = Mechanize.new
agent.open_timeout = 1
agent.read_timeout = 1
Raises
Net::OpenTimeout
on connect timeoutNet::HTTP::Persistent::Error
on read timeout
Net::DNS::Resolver.new(udp_timeout: 1)
Default: 5s
Raises Net::DNS::Resolver::NoResponseError
smtp = Net::SMTP.new(host, 25)
smtp.open_timeout = 1
smtp.read_timeout = 1
Raises
Net::OpenTimeout
on connect timeoutNet::ReadTimeout
on read timeout
Net::SSH.start(host, user, timeout: 1)
Raises Net::SSH::ConnectionTimeout
Not configurable at the moment, and no timeout by default
Slack::Notifier.new(webhook_url, http_options: {open_timeout: 1, read_timeout: 1})
Raises
Net::OpenTimeout
on connect timeoutNet::ReadTimeout
on read timeout
Stripe.open_timeout = 1
Stripe.read_timeout = 1
Default: 30s connect timeout, 80s read timeout
Raises Stripe::APIConnectionError
Twilio::REST::Client.new(account_sid, auth_token, timeout: 1)
Default: 30s
Raises
Net::OpenTimeout
on connect timeoutNet::ReadTimeout
on read timeout
Not configurable at the moment
Default: 10s connect timeout, 30s read timeout
Not configurable at the moment
Default: 10s connect timeout, no read timeout
Let us know. Even better, create a pull request for it.
Take advantage of inheritance. Instead of
rescue Net::OpenTimeout, Net::ReadTimeout
you can do
rescue Timeout::Error
Use
Timeout::Error
for bothNet::OpenTimeout
andNet::ReadTimeout
Faraday::ClientError
for bothFaraday::ConnectionFailed
andFaraday::TimeoutError
HTTPClient::TimeoutError
for bothHTTPClient::ConnectTimeoutError
andHTTPClient::ReceiveTimeoutError
Redis::BaseConnectionError
for bothRedis::CannotConnectError
andRedis::TimeoutError
Rack::Timeout::Error
for bothRack::Timeout::RequestTimeoutError
andRack::Timeout::RequestExpiryError
Adding timeouts to existing services can be a daunting task, but there’s a low risk way to do it.
- Select a timeout - say 5 seconds
- Log instances exceeding the proposed timeout
- Fix them
- Add the timeout
- Repeat this process with a lower timeout, until your target timeout is achieved
git clone https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts.git
cd the-ultimate-guide-to-ruby-timeouts
bundle install
node test/server.js # in a separate window
rake
Prevent single queries from taking up all of your database’s resources. Set a statement timeout in your config/database.yml
production:
variables:
statement_timeout: 250 # ms
or set it on your database role
ALTER ROLE myuser SET statement_timeout = 250;
Test statement timeouts with
SELECT pg_sleep(5);
For MySQL 5.7.8 and higher, set a statement timeout in your config/database.yml
production:
variables:
max_execution_time: 250 # ms
Test with
SELECT 1 FROM information_schema.tables WHERE sleep(5);
Because time is not going to go backwards, I think I better stop now. - Stephen Hawking
🕓