simplecov¶ ↑

SimpleCov is a code coverage analysis tool for Ruby 1.9. It uses 1.9’s built-in Coverage library to gather code coverage data, but makes processing it’s results much easier by providing a clean API to filter, group, merge, format and display those results, thus giving you a complete code coverage suite with just a couple lines of code.

In most cases, you’ll want overall coverage results for your projects, including all types of tests, cucumber features etc. Simplecov automatically takes care of this by caching and then merging results when generating reports, so your coverage report gives you a more accurate report on the blank spots in your test coverage.

The official formatter of SimpleCov is packaged as a separate gem called simplecov-html and will be installed automatically when you do gem install simplecov. You can find it at github.com/colszowka/simplecov-html

Basic usage¶ ↑

Update your Gemfile with this and do a bundle install:

group :test do
  gem 'simplecov', '>= 0.3.5', :require => false # Will install simplecov-html as a dependency
end

Then, add the following to your Rails test/test_helper.rb (right at the top, line 00):

require 'simplecov'
SimpleCov.start 'rails'

Now, when running rake test you’ll get a coverage/ folder inside your app’s root where you can browse your code coverage.

Example output¶ ↑

For the Devise Ruby gem (some tests were removed, they just have too awesome test coverage…)

Usage with Cucumber and RSpec (name your framework)¶ ↑

Similarily to the usage with Test::Unit described above, the only thing you have to do is to add the simplecov config to the very top of your Cucumber/RSpec/whatever setup file.

Just add the setup snippet at the very(!) top of features/support/env.rb (for Cucumber) or spec/spec_helper.rb (for RSpec). Other test frameworks should work accordingly.

require 'simplecov'
SimpleCov.start 'rails'

Configuration¶ ↑

Configuration settings can be applied in three formats.

The ‘direct’ way:

SimpleCov.some_config_option 'foo'

Using a block:

SimpleCov.configure do
  some_config_option 'foo'
end

Using a block and automatically starting the coverage:

SimpleCov.start do
  some_config_option 'foo'
end

Most times, you’ll want to use the latter, so loading and setting up simplecov is in one place at the top of your test helper.

Filters¶ ↑

Filters can be used to remove selected files from your coverage data. By default, a filter is applied that removes all files OUTSIDE of your project’s root directory - otherwise you’d end up with a billion of coverage reports for source files in the gems you are using.

Of course you can define your own to remove things like configuration files, tests or whatever you don’t need in your coverage report.

Defining custom filters¶ ↑

You can currently define a filter using either a String (that will then be Regexp-matched against each source file’s path), a block or by passing in your own Filter class.

String filter¶ ↑

SimpleCov.start do
  add_filter "/test/"
end

This simple string filter will remove all files that match “/test/” in their path.

Block filter¶ ↑

SimpleCov.start do
  add_filter do |source_file|
    source_file.lines.count < 5
  end
end

Block filters receive a SimpleCov::SourceFile instance and expect your block to return either true (if the file is to be removed from the result) or false (if the result should be kept). Please check out the RDoc for SimpleCov::SourceFile to learn about the methods available to you. In the above example, the filter will remove all files that have less then 5 lines of code.

Custom filter class¶ ↑

class LineFilter < SimpleCov::Filter
  def passes?(source_file)
    source_file.lines.count < filter_argument
  end
end

SimpleCov.add_filter LineFilter.new(5)

Defining your own filters is pretty easy: Just inherit from SimpleCov::Filter and define a method ‘passes?(source_file)’. When running the filter, a true return value from this method will result in the removal of the given source_file. The filter_argument method is being set in the SimpleCov::Filter initialize method and thus is set to 5 in this example.

Groups¶ ↑

You can separate your source files into groups. For example, in a rails app, you’ll want to have separate listings for Models, Controllers, Helpers, Libs and Plugins. Group definition works similar to Filters (and indeed also accepts custom filter classes), but source files end up in a group when the filter passes (returns true), as opposed to filtering results, which exclude files from results when the filter results in a true value.

Add your groups with:

SimpleCov.start do
  add_group "Models", "app/models"
  add_group "Controllers", "app/controllers"
  add_group "Long files" do |src_file|
    src_file.lines.count > 100
  end
  add_group "Short files", LineFilter.new(5) # Using the LineFilter class defined in Filters section above
end

Merging results¶ ↑

Normally, you want to have your coverage analyzed across ALL of your test suites, right?

Simplecov automatically caches coverage results in your (coverage_path)/resultset.yml. Those results will then be automatically merged when generating the result, so when coverage is set up properly for cucumber and your unit / functional / integration tests, all of those test suites will be taken into account when building the coverage report.

There are two things to note here though:

Test suite names¶ ↑

Simplecov tries to guess the name of the currently running test suite based upon the shell command the tests are running on (from v0.3.2+). This should work fine for Unit Tests, RSpec and Cucumber. If it fails, it will use the shell command that invoked the test suite as a command name.

If you have some non-standard setup and still want nicely labeled test suites, you have to give Simplecov a cue what the name of the currently running test suite is. You can do so by specifying SimpleCov.command_name in one test file that is part of your specific suite.

So, to customize the suite names on a Rails app (yeah, sorry for being Rails biased, but everyone knows what the structure of those projects is. You can apply this accordingly to the RSpecs in your Outlook-WebDAV-Calendar-Sync gem), you could do something like this:

# test/unit/some_test.rb
SimpleCov.command_name 'test:units'

# test/functionals/some_controller_test.rb
SimpleCov.command_name "test:functionals"

# test/integration/some_integration_test.rb
SimpleCov.command_name "test:integration"

# features/steps/web_steps.rb
SimpleCov.command_name "features"

Note that this has only to be invoked ONCE PER TEST SUITE, so even if you have 200 unit test files, specifying it in some_test.rb is fair enough.

simplecov-html prints the used test suites in the footer of the generated coverage report.

Timeout for merge¶ ↑

Of course, your cached coverage data is likely to become invalid at some point. Thus, result sets that are older than SimpleCov.merge_timeout will not be used any more. By default, the timeout is 600 seconds (10 minutes), and you can raise (or lower) it by specifying SimpleCov.merge_timeout 3600 (1 hour), or, inside a configure/start block, with just “merge_timeout 3600”.

You can deactivate merging altogether with “SimpleCov.use_merging false”.

Adapters¶ ↑

By default, Simplecov’s only config assumption is that you only want coverage reports for files inside your project root. To save you from repetitive configuration, you can use predefined blocks of configuration, called ‘adapters’, or define your own.

You can then pass the name of the adapter to be used as the first argument to SimpleCov.start. For example, simplecov comes bundled with a ‘rails’ adapter. It looks somewhat like this:

SimpleCov.adapters.define 'rails' do
  add_filter '/test/'
  add_filter '/config/'

  add_group 'Controllers', 'app/controllers'
  add_group 'Models', 'app/models'
  add_group 'Helpers', 'app/helpers'
  add_group 'Libraries', 'lib'
  add_group 'Plugins', 'vendor/plugins'
end

As you can see, it’s just a glorified SimpleCov.configure block. In your test_helper.rb, launch simplecov with:

SimpleCov.start 'rails'

  OR

SimpleCov.start 'rails' do
  # additional config here
end

Custom adapters¶ ↑

You can load additional adapters with the SimpleCov.load_adapter(‘xyz’) method. This allows you to build upon an existing adapter and customize it so you can reuse it in unit tests and cucumber features, for example.

# lib/simplecov_custom_adapter.rb
require 'simplecov'
SimpleCov.adapters.define 'myadapter' do
  load_adapter 'rails'
  add_filter 'vendor' # Don't include vendored stuff
end

# features/support/env.rb
require 'simplecov_custom_adapter'
SimpleCov.start 'myadapter'

# test/test_helper.rb
require 'simplecov_custom_adapter'
SimpleCov.start 'myadapter'

Customizing exit behaviour¶ ↑

You can define what simplecov should do when your test suite finishes by customizing the at_exit hook:

SimpleCov.at_exit do
  SimpleCov.result.format!
end

Above is the default behaviour. Do whatever you like instead!

Using your own formatter¶ ↑

You can use your own formatter with:

SimpleCov.formatter = SimpleCov::Formatter::HTMLFormatter

When calling SimpleCov.result.format!, it will be invoked with SimpleCov::Formatter::YourFormatter.new.format(result), “result” being an instance of SimpleCov::Result. Do whatever your wish with that!

Configuration options¶ ↑

SimpleCov.root ‘/some/path/to/coverage’¶ ↑

The root directory for the project coverage is being generated for. This defaults to the current working directory and should be just fine. Note that by default, all files outside of root will be filtered and thus not included in your coverage report, so you’ll probably rarely want to adjust this setting other than specifying an absolute path here.

SimpleCov.coverage_dir ‘coverage’¶ ↑

The name of the subdirectory of root that coverage will be generated to. Defaults to ‘coverage’.

…TODO…¶ ↑

Kudos¶ ↑

Thanks to Aaron Patterson (engineering.attinteractive.com/2010/08/code-coverage-in-ruby-1-9/) for the original idea for this!

TODO¶ ↑

Improve on tests (integration tests)

Note on Patches/Pull Requests¶ ↑

Fork the project.
Make your feature addition or bug fix.
Add tests for it. This is important so I don’t break it in a future version unintentionally.
Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
Send me a pull request. Bonus points for topic branches.

reddyonrails / simplecov Goto Github PK

simplecov's Introduction