ankane / the-ultimate-guide-to-ruby-timeouts Goto Github PK

https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts/blob/0573b4f52cad438c1341e6d6254378ea7ea9370b/README.md#mysql

For migrations, you likely want to set a longer statement timeout. You can do this with

MySQL's max_execution_time only affects SELECT statements. And in migration, Rails looks like it only emits a small number of SELECT queries for schema_migrations or ar_internal_metadata, which are relatively smaller tables, so it seems not needed to set max_execution_time in migration for most cases.

Unlike MySQL, MariaDB's max_statement_time or Postgresql's statement_timeout seems affects any queries, not only SELECT. So it makes sense to change these timeout values in migration.

Hello :)

Test script:

require 'net/http'

uri = URI('http://localhost:4567/foo/bar')

Net::HTTP.start(uri.host, uri.port, read_timeout: 1) do |http|
  http.request(Net::HTTP::Get.new(uri))
end

Test server output:

$ ruby -r sinatra -e 'get("/foo/bar"){ sleep 3; "baz" }'
== Sinatra/1.4.5 has taken the stage on 4567 for development with backup from Thin
Thin web server (v1.6.2 codename Doc Brown)
Maximum connections set to 1024
Listening on localhost:4567, CTRL+C to stop
127.0.0.1 - - [21/Dec/2015:00:03:21 +0300] "GET /foo/bar HTTP/1.1" 200 3 3.0183
127.0.0.1 - - [21/Dec/2015:00:03:22 +0300] "GET /foo/bar HTTP/1.1" 200 3 3.0059

I think this retry statement is to blame:

https://github.com/ruby/ruby/blob/v2_2_4/lib/net/http.rb#L1436

I'm not sure what to do with this information. The behaviour is unexpected. Add a warning maybe?

example

conn = Faraday.new(...) do |f|
  f.adapter :net_http_persistent do |http| # yields Net::HTTP::Persistent
    http.idle_timeout = 100
    http.retry_change_requests = true
  end
end

Hello,

Thank you for this great resource 🙏 !

not sure if this type of issue is acceptable for the repo, but here it goes:

I have a potentially expensive query that could take more time than I would want. I can think of the following ways to limit it:

1. Wrap it in Timeout.timeout (I know this guide is clearly against it, please bear with me):

begin
  Timeout::timeout(TIMEOUT_SECONDS, Timeout::Error) do
    @count = calculate_expensive_count
  end
rescue Timeout::Error => ex
  # handle
end

What happens to the query in case it exceeds TIMEOUT_SECONDS? Is it killed, or will it go on until it finishes?

2. Temporarily set global timeout

begin
  ActiveRecord::Base.connection.execute("SET statement_timeout TO #{TIMEOUT_SECONDS*1000}")
  @count = calculate_expensive_count
rescue ActiveRecord::StatementInvalid
  # handle
ensure
  ActiveRecord::Base.connection.execute("SET statement_timeout TO #{ORIGINAL_PSQL_TIMEOUT_SECONDS*1000")
end

3. Start a transaction and set a local timeout

begin
  ActiveRecord::Base.transaction do
    ActiveRecord::Base.connection.execute("SET LOCAL statement_timeout TO #{TIMEOUT_SECONDS*1000}")
    @count = calculate_expensive_count
  end  
rescue ActiveRecord::StatementInvalid
  # handle
end

Between 2 and 3 I prefer 3 because it does not mess with the connection's statement timeout, which is cleaner. I know the repo clearly instructs against Timeout::Timeout, I have just added it because the example given in the linked article is about Redis, and not the DB. Stylewise, I would prefer 1 if it's not dangerous (for the particular use case), because I don't need multiple statements in Postgres.

Which is preferable and why?

It says in the README that

All [gems] have been tested

But in reality (at least) rack-timeout is not. Two solutions come to mind:

• Test the gem
• Change the language to something like "Some of these gems are tested here"

The latter looks simpler to me.

This might not look like a big deal but it might give certain people absolute confidence when using the gem. It's been two years since they've wanted to test the gem but they haven't.

Found your guide, very helpful!

If I am running Rails 6 with Postgres, Sidekiq, Redis on Heroku what timeout settings do you recommend? You stated Postgres was the most important, but what would be 2nd and 3rd? Also, do you have examples of how best to rescue a timeout? Especially Postgres timeouts?

It seems setting the database.yml will setup the database. But you also mentioned a way to setup ActiveRecord. Is one preferred? Does the ActiveRecord version prevent the migration timeout issue?

production:
variables:
statement_timeout: 5s # or ms, min, etc

not sure if this needs to be fixed but libpq recommends not using a connect_timeout under 2 seconds, your example is 1 second.

https://www.postgresql.org/docs/9.1/libpq-connect.html

We use the activerecord-sql-server-adapter gem at my work and experince timeout issues.

I'm experimenting with reaping_frequency in the database.yml currently and I am also considering changing checkout_timeout would appreciate more information from people that have solved practical problems related to timeouts with ActiveRecord and MSSQL.

I'm trying to run:

sql_string = "SET LOCAL statement_timeout = 1; <SOME LONG QUERY>"
results = ActiveRecord::Base.connection.execute(sql_string)

If I added BEGIN; SET LOCAL statement_timeout = 1; <SOME LONG QUERY> ; COMMIT; , I'm not getting any return value from ActiveRecord::Base.connection.execute(sql_string).

Expecting it to timeout , but it seem to finish the query.
Anything that I'm missing?

At the beginning of the guide, it says to avoid the Timeout library, but during the guide it suggests using timeouts that are created using Timeout, like net/http timeouts.

I would suggest recommending a different network library, like the HTTP gem, that does not use the Timeout stdlib.

Apologies in advance, if this obvious to others but it is not clear to me; does this guide just show how to set timeouts or the numbers used in examples are considered as best practices? For example, postgresql's STATEMENT_TIMEOUT is set to 90. Is there a specific reason for it or was it just a sensible number?

Hi, thanks everyone for contributing! This repo doesn't currently have a license, which isn't great, so I'd like to license it under the MIT license. To do that, I'd like to get approval from all contributors.

• @cben
• @rud
• @ghiculescu
• @dlackty
• @watsonian
• @sidonath
• @pointlessone
• @myersjustinc
• @kennethkalmer
• @jordan-brough
• @johndouthat
• @janko
• @abinoda
• @Nakilon

Do you agree to license your contribution(s) under the MIT license? If so, please comment "I agree" below.

(also, sorry for the noise - you may want to unsubscribe from the issue after reviewing)

This is awesome. That is all 👍🏽

Net::SSH.start has a timeout: argument, but Net::SSH.exec! does not. Is there a work around that doesn't use Ruby timeout for that?