nesquena / query_reviewer Goto Github PK

This project forked from weplay/query_reviewer

An excellent development aid. Will run "EXPLAIN" before each of your select queries in development, and provides a display which makes them easy to browse. In this fork, gemified and compatible with Rails 2 + 3.

Home Page: http://code.google.com/p/query-reviewer/

License: MIT License

Ruby 95.56% JavaScript 4.44%

query_reviewer's Introduction

QueryReviewer

Introduction

QueryReviewer is an advanced SQL query analyzer. It accomplishes the following goals:

View all EXPLAIN output for all SELECT queries to generate a page
Rate a page's SQL usage into one of three categories: OK, WARNING, CRITICAL
Attach meaningful warnings to individual queries, and collections of queries
Display interactive summary on page

Use This Fork

I use this utility for most of my MySQL-backed Rails projects. Still the best out there in my opinion for analyzing and understanding your generated SQL queries. I forked the original query_reviewer and applied a collection of patches that have been made since the plugin was created. A list of the biggest additions below:

Snazzed up the README into markdown for better readability
Full compatibility for Rails 3 (including Railtie)
Cleanup and move rake task to lib/tasks to fix deprecation warnings
Added gemspec for use with Bundler (as a gem)
Fixed missing tags and additional XHTML escaping
Fix SQL escaping for better XHTML compatibility
Fixes for deprecation warnings and for 1.9 compatiblity
Converts templates to more modern foo.html.erb naming

As of August 2011, This has become the "official" version to use. David has deprecated the original repo and recommends that people looking for this utility use this fork instead. This fork compiles a variety of patches that were made since that time along with additional work to support compatibility with 1.9 and Rails 3. Also: If anyone else creates generally useful enhancements to this utility please start by forking this and then issue me a pull request.

Note: This plugin should work for Rails 2.X and Rails 3. If you run into issues, please file a bug report or send a pull request. I consider this new fork to be a group effort so if you want to be added as a contributor to help maintain this, please let me know!

Database Limitations

As of right now, this plugin supports analyzing queries only for MySQL. I consider this the primary limitation of this gem. I am now using PostgreSQL for a number of my projects and the "EXPLAIN" output is totally different. Several people have started a PostgreSQL adapter for this plugin, but none that are mature enough to include here.

If anyone is interested at taking a stab at implementing PostgreSQL support for this utility, that would be hugely appreciated! I hope this plugin can come to support all the major Rails adapters over time.

Installation

All you have to do is install it into your Rails 2 or 3 project.

gem install query_reviewer

Right now if you use bundler, simply add this to your Gemfile:

# Gemfile
gem "query_reviewer"

or to use the latest from github:

# Gemfile
gem "query_reviewer", :github => "nesquena/query_reviewer"

If you are not using bundler, you might want to start using it. You can also install this as a plugin:

script/plugin install git://github.com/nesquena/query_reviewer.git

In Rails 2, the rake tasks are not loaded automatically (as a gem), you’ll need to add the following to your Rakefile:

# Rakefile
begin
  require 'query_reviewer/tasks'
rescue LoadError
  STDERR.puts "The query_reviewer gem could not be found!"
end

You can then run:

$ rake query_reviewer:setup

Which will create config/query_reviewer.yml in your application, see below for what these options mean. If you don't create a config file, the gem will use the default in vendor/plugins/query_reviewer.

Configuration

The configuration file allows you to set configuration parameters shared across all rails environment, as well as overriding those shared parameteres with environment-specific parameters (such as disabling analysis on production!)

enabled: whether any output or query analysis is performed. Set this false in production!
inject_view: controls whether the output automatically is injected before the </body> in HTML output.
profiling: when enabled, runs the MySQL SET PROFILING=1 for queries longer than the warn_duration_threshold / 2.0
production_data: whether the duration of a query should be taken into account
stack_trace_lines: number of lines of call stack to include in the "short" version of the stack trace
trace_includes_vendor: whether the "short" version of the stack trace should include files in /vendor
trace_includes_lib: whether the "short" version of the stack trace should include files in /lib
warn_severity: the severity of problem that merits "WARNING" status
critical_severity: the severity of problem that merits "CRITICAL" status
warn_query_count: the number of queries in a single request that merits "WARNING" status
critical_query_count: the number of queries in a single request that merits "CRITICAL" status
warn_duration_threshold: how long a query must take in seconds (float) before it's considered "WARNING"
critical_duration_threshold: how long a query must take in seconds (float) before it's considered "CRITICIAL"

Example

If you disable the inject_view option above, you'll need to manually put the analyzer's output into your view:

# view.html.haml
= query_review_output

and that will display the analyzer view!

Resources

Random collection of resources that might be interesting related to this utility:

Other related gems that prove useful for database optimization:

Alternatives

There have been other alternatives created since this was originally released. A few of the best are listed below. I for one still prefer this utility over the other options:

Know of a better alternative? Let me know!

Acknowledgements

Created by Kongregate & David Stevenson. Refactorings and maintenance done by Nathan Esquenazi. Also @ajvargo helped with several fixes.

query_reviewer's People

Contributors

Stargazers

Watchers

query_reviewer's Issues

box won't render unless tracing is enable?

I'd like the query review box to render if query reviewer is enabled in the configuration yml, even if the cookie isn't enabled and no tracing occurs. As it is right now, the query review view and all associated javascript does't get loaded so there's no clickable interface to enable tracing anywhere on the page, unless I manually include query_review_output in a view somewhere.

But if I do this, once I enable the cookie by clicking the link in the view, I get the query reviewer view injected into my html twice.

Am I missing some configuration to avoid this? If not, I think the solution is to always add_query_output_to_view in controller_extensions (that is, remove the return unless Thread.current["query_reviewer_enabled"]) and, instead, in perform_action_with_query_review, change the Thread.current[...] = to return unless.

I can submit a pull request if you'd like.

Problem using Devise 2.0.4 and Rails 3.2.2

Hi!

After upgrading to Rails 3.2.2 and Devise 2.04, every request I do requiring authorization exits with this error:

NoMethodError (protected method `_prefixes' called for #<SessionsController:0x00000107935578>)

The full stacktrace:

Started GET "/salir" for 127.0.0.1 at 2012-03-22 12:42:40 +0100
Processing by Devise::SessionsController#destroy as HTML
  Parameters: {"locale"=>"es"}
   (0.2ms)  SET PROFILING=1
  User Load (0.4ms)  SELECT SQL_NO_CACHE `users`.* FROM `users` WHERE `users`.`id` = 23 LIMIT 1
   (0.3ms)  BEGIN
   (0.5ms)  UPDATE `users` SET `last_request_at` = '2012-03-22 11:42:40', `updated_at` = '2012-03-22 11:42:40' WHERE `users`.`id` = 23
[paperclip] Saving attachments.
   (0.9ms)  COMMIT
   (0.2ms)  BEGIN
[paperclip] Saving attachments.
   (0.1ms)  COMMIT
Redirected to http://localhost:3000/
Completed 302 Found in 551ms (ActiveRecord: 7.4ms)

NoMethodError (protected method `_prefixes' called for #<Devise::SessionsController:0x0000012e4f3f60>):
  config/initializers/robots_middleware.rb:28:in `call'
  config/initializers/flash_session_cookie_middleware.rb:16:in `call'


  Rendered /Users/dgilperez/.rvm/gems/ruby-1.9.2-p290/gems/actionpack-3.2.2/lib/action_dispatch/middleware/templates/rescues/_trace.erb (2.6ms)
  Rendered /Users/dgilperez/.rvm/gems/ruby-1.9.2-p290/gems/actionpack-3.2.2/lib/action_dispatch/middleware/templates/rescues/_request_and_response.erb (1.1ms)
  Rendered /Users/dgilperez/.rvm/gems/ruby-1.9.2-p290/gems/actionpack-3.2.2/lib/action_dispatch/middleware/templates/rescues/diagnostics.erb within rescues/layout (13.8ms)

After I remove query_reviewer, everything goes back to normal.

Related issue on Devise: heartcombo/devise#1693

Exception when saving data on Rails 3.1

undefined method `gsub!' for #<Arel::InsertManager:0x007fd015e4c630>

Got this when running a create method on a model.

Full trace: https://gist.github.com/1307784

Template Missing (clobbers view path?)

This is what I get when I try to use this gem. It makes me sad. :(

Template is missing

Missing template tasks/edit.erb in view path /Users/sdhull/.rvm/gems/ree-1.8.7-2011.03@my_gemset/gems/query_reviewer-0.1.5/lib/query_reviewer/lib/query_reviewer/views:/Users/sdhull/.rvm/gems/ree-1.8.7-2011.03@my_gemset/gems/preferences-0.4.2/app/views

I've tried disabling the inject_view option, but it doesn't help. Googling shows a few similar errors/stacktraces in pasties and gists, but no answers. Help please!

the configuration file isn't being found and is always using query_reviewer_defaults.yml

the configuration file isn't being found and is always using query_reviewer_defaults.yml
Rails.root is returning '' when the app starts up and thats keeping the proper config file from loading

Here is my output from the debugger:

Pairing3:ip 0 03-01 18:57~/Dev/health-teams
! rails s --debugger
DEPRECATION WARNING: railtie_name is deprecated and has no effect. (called from <top (required)> at /Users/ip/Dev/health-teams/config/application.rb:7)
[13, 22] in /Users/ip/.rvm/gems/ruby-1.9.2-p0@health_teams/bundler/gems/query_reviewer-726684eccc25/lib/query_reviewer.rb
13 CONFIGURATION.merge!(default_config[Rails.env || "test"] || {})
14
15 app_config_file = File.join(Rails.root.to_s, "config", "query_reviewer.yml")
16 require 'ruby-debug'
17 debugger
=> 18 if File.exist?(app_config_file)
19 app_config = YAML.load(ERB.new(IO.read(app_config_file)).result)
20 CONFIGURATION.merge!(app_config["all"] || {})
21 CONFIGURATION.merge!(app_config[Rails.env || "test"] || {})
22 end
/Users/ip/.rvm/gems/ruby-1.9.2-p0@health_teams/bundler/gems/query_reviewer-726684eccc25/lib/query_reviewer.rb:18
if File.exist?(app_config_file)
(rdb:1) app_config_file
"/config/query_reviewer.yml"
(rdb:1) Rails.env
"development"
(rdb:1) Rails.root
nil
(rdb:1)

Problem using Devise 2.0.4 and Rails 3.2.2

Hi!

After upgrading to Rails 3.2.2 and Devise 2.04, every request I do requiring authorization exits with this error:

NoMethodError (protected method `_prefixes' called for #<SessionsController:0x00000107935578>)

The full stacktrace:

Started GET "/salir" for 127.0.0.1 at 2012-03-22 12:42:40 +0100
Processing by Devise::SessionsController#destroy as HTML
  Parameters: {"locale"=>"es"}
   (0.2ms)  SET PROFILING=1
  User Load (0.4ms)  SELECT SQL_NO_CACHE `users`.* FROM `users` WHERE `users`.`id` = 23 LIMIT 1
   (0.3ms)  BEGIN
   (0.5ms)  UPDATE `users` SET `last_request_at` = '2012-03-22 11:42:40', `updated_at` = '2012-03-22 11:42:40' WHERE `users`.`id` = 23
[paperclip] Saving attachments.
   (0.9ms)  COMMIT
   (0.2ms)  BEGIN
[paperclip] Saving attachments.
   (0.1ms)  COMMIT
Redirected to http://localhost:3000/
Completed 302 Found in 551ms (ActiveRecord: 7.4ms)

NoMethodError (protected method `_prefixes' called for #<Devise::SessionsController:0x0000012e4f3f60>):
  config/initializers/robots_middleware.rb:28:in `call'
  config/initializers/flash_session_cookie_middleware.rb:16:in `call'


  Rendered /Users/dgilperez/.rvm/gems/ruby-1.9.2-p290/gems/actionpack-3.2.2/lib/action_dispatch/middleware/templates/rescues/_trace.erb (2.6ms)
  Rendered /Users/dgilperez/.rvm/gems/ruby-1.9.2-p290/gems/actionpack-3.2.2/lib/action_dispatch/middleware/templates/rescues/_request_and_response.erb (1.1ms)
  Rendered /Users/dgilperez/.rvm/gems/ruby-1.9.2-p290/gems/actionpack-3.2.2/lib/action_dispatch/middleware/templates/rescues/diagnostics.erb within rescues/layout (13.8ms)

After I remove query_reviewer, everything goes back to normal.

Related issue on Devise: heartcombo/devise#1693

Nested models error: " undefined method `reverse' for nil:NilClass "

Hello! I use nested models(think reason near this) and hard relations between them. When I try to create some model instance, i get such error in create action.

NoMethodError in AdvertsController#create
undefined method `reverse' for nil:NilClass

When gem is disabled - error disappears.

Query Reviewer for Datamapper

I am using MySql and Datamapper on a Rails 3.2.11 app. Was trying to use query reviewer but getting this error on server start:

.rvm/gems/ruby-1.9.3-p194@bookings/gems/query_reviewer-0.1.8/lib/query_reviewer/rails.rb:6:in `inject_reviewer': uninitialized constant QueryReviewer::ActiveRecord (NameError)

I guess query reviewer does not support DataMapper yet ? Let me know if you would prefer a full stack trace.

query_reviewer breaks send_file in Rails 3.0

The problem is this code ( https://github.com/nesquena/query_reviewer/blob/master/lib/query_reviewer/controller_extensions.rb#L42 )

        if response.body.is_a?(String) && response.body.match(/<\/body>/i)

The problem is that as soon as you call response.body the response is ruined and a blank page is shown to the user. You might want to replace response.body.is_a?(String) with response.instance_eval{ @body.is_a?(String) }

Deprecation warnings with recent rubygems versions

Using query_reviewer with recent versions of rubygems prints warnings like this:

NOTE: Gem.searcher is deprecated, use Specification. It will be removed on or after 2011-11-01.
Gem.searcher called from XXXXXX/bundler/gems/query_reviewer-9de60e75d180/lib/query_reviewer.rb:25.
NOTE: Gem::GemPathSearcher#initialize is deprecated with no replacement. It will be removed on or after 2011-10-01.
Gem::GemPathSearcher#initialize called from XXXXXX/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems.rb:952.
NOTE: Gem::GemPathSearcher#find is deprecated with no replacement. It will be removed on or after 2011-10-01.
Gem::GemPathSearcher#find called from XXXXXX/bundler/gems/query_reviewer-9de60e75d180/lib/query_reviewer.rb:25.

Dynamically load/unload per request

It would be great if you could enable query_reviewer for a single request. Mostly so you can profile a particular page once, in production mode, but not have the overhead of leaving it on for all requests. It almost looks like you could do that by overriding how enabled? works, but it loads a bunch of modules, and I don't think there's an easy way to unload them after the request is completed.

If I can get good direction on this, it'd probably be worth my time to fork and pull.

error upgrading to Rails 3.2

Hi!

I'm upgrading my rails 3.1 app to 3.2. I have been experiencing some weirdness I believe related to the query_reviewer

   (0.2ms)  SET PROFILING=1
  Measurement Load (0.8ms)  SELECT SQL_NO_CACHE `houses`.* FROM `houses` INNER JOIN `lanes` ON `lanes`.`id` = `houses`.`variable_id` WHERE `lanes`.`name` = 'weight' AND `houses`.`user_id` = 23 ORDER BY date DESC LIMIT 1
  EXPLAIN (25.5ms)  EXPLAIN SET PROFILING=1
Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SET PROFILING=1' at line 1: EXPLAIN SET PROFILING=1
Completed 500 Internal Server Error in 803ms

It seems Rails is confusing the query that has to EXPLAIN. This happens with many slow queries and works good with Rails 3.1 or by removing query_reviewer in Rails 3.2.

I use query_reviewera lot and find it really helpful, so any ideas will be appreciated!

thanks for the effort put into this,

David

in rails 3.2.5 project no display on 1st request bc cookie not set, which means no ability to SET cookie

I've used query_reviewer in a 2.3.5 environment and have ported my project to rails 3.2.5.

When I restart 'thin' and load a page, I don't get any view injected by query_review. I debugged the problem and file
gems/query_reviewer-0.1.6/lib/query_reviewer/controller_extensions.rb
def
add_query_output_to_view(total_time)
is returning on the first line
return unless Thread.current["query_reviewer_enabled"]

which is set to false because in the same file, in method
perform_action_with_query_review(*args)

the first line is setting Thread.current["query_reviewer_enabled"] false because while
CONFIGURATION["enabled"] = true
unfortunately
cookies["query_review_enabled"] = false

How do I set the cookie true? By having the view injected and clicking the "on" button...but it's a catch-22 because the on button isn't present.

I've worked around this by adding a line to force the display, but what's the right thing?

My config looks like this:
all:
inject_view: true
stack_trace_lines: 3
trace_includes_vendor: false
trace_includes_lib: true
profiling: enabled
production_data: true
max_safe_key_length: 100
disable_sql_cache: true

warn_severity: 3
critical_severity: 7

warn_select_count: 20
critical_select_count: 50

warn_update_count: 5
critical_update_count: 10

warn_insert_count: 5
critical_insert_count: 10

warn_delete_count: 5
critical_delete_count: 10

warn_duration_threshold: 0.2
critical_duration_threshold: 1.0

warn_affected_rows: 10
critical_affected_rows: 100

development:
enabled: true

production:
enabled: false

test:
enabled: false

Failing test in v0.1.3 and v0.1.2

Started
.E
Finished in 0.000411 seconds.
1) Error:
test_analyze_select_type(SqlSubqueryTest):
ArgumentError: wrong number of arguments (1 for 0)
    /home/dwo/source/query_reviewer/test/sql_sub_query_test.rb:15:in `initialize'
    /home/dwo/source/query_reviewer/test/sql_sub_query_test.rb:15:in `new'
    /home/dwo/source/query_reviewer/test/sql_sub_query_test.rb:15:in `get_query'
    /home/dwo/source/query_reviewer/test/sql_sub_query_test.rb:5:in `test_analyze_select_type'

2 tests, 1 assertions, 0 failures, 1 errors

It looks like this has been a problem for at least the last two versions. When I run the tests for 0.1.1 I get another unrelated error.