grodowski / undercover

undercover warns about methods, classes and blocks that were changed without tests, to help you easily find untested code and reduce the number of bugs. It does so by analysing data from git diffs, code structure and SimpleCov coverage reports

undercover warns about methods, classes and blocks that were changed without tests, to help you easily find untested code and reduce the number of bugs. It does so by analysing data from git diffs, code structure and SimpleCov coverage reports.

Works with any Ruby CI pipeline as well as locally as a CLI.

• Visit https://undercover-ci.com to set up code review checks with the GitHub App, or use one of the CI integrations
• Learn how to find untested code changes locally with the CLI

A sample output of undercover ran before a commit may look like this:

And like this, given that specs were added:

Installation

Add this line to your application's Gemfile:

gem 'undercover'

And then execute:

$ bundle

Or install it yourself as:

$ gem install undercover

Setting up required LCOV reporting

To make your specs or tests compatible with undercover by providing an LCOV report, please add simplecov and simplecov-lcov to your test setup.

# Gemfile
group :test do
  gem 'simplecov'
  gem 'simplecov-lcov'
end

# the very top of spec_helper.rb
require 'simplecov'
require 'simplecov-lcov'
SimpleCov::Formatter::LcovFormatter.config.report_with_single_file = true
SimpleCov.formatter = SimpleCov::Formatter::LcovFormatter
SimpleCov.start do
  add_filter(/^\/spec\//) # For RSpec
  add_filter(/^\/test\//) # For Minitest
  enable_coverage(:branch) # Report branch coverage to trigger branch-level undercover warnings
end

require 'undercover'

# ...

Then run your test suite once through to generate the initial coverage/lcov/*.lcov file before you can run the undercover command

Usage

Invoked with no arguments, Undercover will flag all untested methods and classes from the current diff:

undercover

Use the -c --compare ref flag to specify a git ref (commit hash, branch name, tag) to compare against. This is a recommended usage for CI/CD build environments, as undercover will exit 1 if there are any warnings.

undercover --compare origin/master

Check out docs/ for CI configuration examples:

• Travis CI
• CircleCI - simple
• CircleCI - advanced
• Semaphore
• Codeship

Merging coverage results (sample gist) is required for parallel tests before processing with undercover.

Code review integrations

A few options exist to provide automated comments from undercover in Pull Request reviews, which is the most streamlined way to add Undercover to your development workflow.

• UndercoverCI - undercover Pull Request feedback delivered natively with GitHub Checks
• pronto-undercover
• danger-undercover
• undercover-checkstyle

Configuration

CLI Options

Options can be passed when running the command from the command line:

undercover -h
Usage: undercover [options]
    -l, --lcov path                  LCOV report file path
    -p, --path path                  Project directory
    -g, --git-dir dir                Override `.git` with a custom directory
    -c, --compare ref                Generate coverage warnings for all changes after `ref`
    -h, --help                       Prints this help
        --version                    Show version

Configuration File

A configuration file named .undercover can be created at the top level of a project's directory containing the same set of options for the CLI. Example file:

-l path/to/different.lcov
-c origin/master

The options set in the file can be overriden by passing arguments when invoking the executable.

Options assume that the program is run from the top level of the project directory.

Ignoring/skipping coverage

Projects with low or nonexistent test coverage are likely to generate large numbers of warnings. While the default workflow would be to address them before the PR approval, your strategy might be different.

In order to acknowledge an untested change and remove the UndercoverCI warning with the intention to improve later (or never), you can wrap the code block with the :nocov: syntax, e.g.

# :nocov:
def skip_this_method
    never_reached
end
# :nocov:

Read more about the :nocov: syntax in SimpleCov's readme.

Why?

I wanted to create a tool to help others and myself ensure that tests are written for all the recent code changes. This should be useful for any ruby project, but especially those large or legacy codebases that lack testing (and we can't or don't want to invest in full test coverage).

The goal was to provide automated warnings, that are:

• relevant, so scoped to the actual code changes
• timely, so we don't end up writing tests long after the implementation
• actionable, so we can fix them before the code is committed or reaches production

For more background, please read the blog post.

Development

After checking out the repo, run bundle to install dependencies. Then, run rake to run the tests and RuboCop. You can also run pry -r 'undercover' for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/grodowski/undercover.

License

The gem is available as open source under the terms of the MIT License.

undercover's Issues

Create a dedicated hook for Overcommit

https://github.com/brigade/overcommit/blob/master/lib/overcommit/hook/pre_commit/pronto.rb

Characters in file names

I'm working on a project that has some files with scores in the names. The lcov parser is truncating the file names and the process fails when trying to process that file. If I replace the regex to accept any character it works fine. Why is the regex restriction necessary?

undercover/lib/undercover/lcov_parser.rb

Line 30 in 8f9cb32

when /^SF:([\.\/\w]+)/

Branch coverage fail with if...else conditions

I wrote this simple code :

class MyFile
  attr_accessor :body

  def my_method
    if body.present?
      puts 'ok'
    else
      puts 'yo'
    end
  end
end

And this spec :

require 'spec_helper'

describe MyFile do
  describe '#my_method' do
    it 'works' do
      my_file = MyFile.new
      my_file.body = ''
      my_file.my_method
      my_file.body = 'test'
      my_file.my_method
    end
  end
end

When I run this spec, this lcov file is generated :

SF:./lib/my_file.rb
DA:1,1
DA:2,1
DA:4,1
DA:5,2
DA:6,1
DA:8,1
BRDA:5,0,1,1
BRDA:7,0,2,1
BRF:2
BRH:2
end_of_record

Visualy, we can see every lines and branches are covered but when I run undercover -c origin/master -l coverage/lcov/lib-my_file.rb.lcov I have this output :

undercover: some methods have no test coverage! Please add specs for methods listed below

🚨 1) node `my_method` type: instance method,
      loc: lib/my_file.rb:4:10, coverage: 100.0%
 4:   def my_method hits: n/a
 5:     if body.present? hits: 2 branches: 1/1
 6:       puts 'ok' hits: 1
 7:     else hits: n/a
 8:       puts 'yo' hits: 1
 9:     end hits: n/a
10:   end hits: n/a

It seems everything is covered so I'm not supposed to have the "some methods have no test coverage" message.

Here is my spec_helper :

require 'simplecov'
require 'simplecov-lcov'

SimpleCov.formatter = SimpleCov::Formatter::LcovFormatter
SimpleCov.start do
  enable_coverage :branch
end

require 'undercover'

require File.expand_path("../../config/environment", __FILE__)
require 'rspec/rails'

Report generated with simplecov-html show 100% coverage and no error on this file.

I have the success message with this code :

class MyFile
  attr_accessor :body

  def my_method
    body.present? ? puts('ok') : puts('yo')
  end
end

undercover reports <100% method coverage even though it appears by eye to be 100%

Undercover does not show any red in the output, other than the initial "loc" line, and there are no "hits: 0/" or "branches: 0/" lines. Every line that is not "hits: n/a" is green.

Output from undercover:

undercover: 👮‍♂️ some methods have no test coverage! Please add specs for methods listed below
🚨 1) node `fun` type: instance method,
      loc: foo/applications/bar/services/baz.rb:92:168, coverage: 94.12%
 92:     def fun( hits: n/a
 93:       arg_a:, hits: n/a
 94:       arg_b:, hits: n/a
 95:       arg_c:, hits: n/a
 96:       arg_d:, hits: n/a
 97:       arg_e:, hits: n/a
 98:       arg_f:, hits: n/a
 99:       arg_g:, hits: n/a
100:       arg_h:, hits: n/a
101:       arg_i:, hits: n/a
102:       arg_j:, hits: n/a
103:       arg_k:, hits: n/a
104:       arg_l: hits: n/a
105:     ) hits: n/a
106:       var_a = fun_a(arg_b, arg_h, arg_l) hits: 181
107:       var_b = arg_e || arg_c hits: 181
108:       var_c = arg_g || arg_d hits: 181
109:       var_d = { hits: n/a
110:         key_a: arg_b.key_a, hits: 181
111:         key_b: var_b, hits: n/a
112:         key_c: var_c, hits: n/a
113:       }.merge(arg_i) hits: n/a
114: 
115:       if arg_k == :sym_a && (var_b.nil? || var_c.nil?) hits: 181 branches: 2/2
116:         return Result.new(var_a.fun_b, true, var_d) hits: 2
117:       end hits: n/a
118: 
119:       var_e = ModuleA::ClassA.new( hits: 179
120:         arg_f, hits: n/a
121:         var_b, hits: n/a
122:         var_c, hits: n/a
123:         nil, hits: n/a
124:         arg_k, hits: n/a
125:       ) hits: n/a
126: 
127:       var_f = var_a.find_unit( hits: 179
128:         var_e, hits: n/a
129:         key_d: arg_a, hits: n/a
130:         key_e: false, hits: n/a
131:         key_f: arg_j, hits: n/a
132:       ) hits: n/a
133: 
134:       if var_f hits: 179 branches: 1/1
135:         Result.new(var_f, false, var_d) hits: 125
136:       else hits: n/a
137:         var_f = var_a.find_unit( hits: 54
138:           var_e, hits: n/a
139:           key_d: arg_a, hits: n/a
140:           key_e: true, hits: n/a
141:           key_f: arg_j, hits: n/a
142:         ) hits: n/a
143:         if var_f hits: 54 branches: 1/1
144:           Result.new(var_f, true, var_d) hits: 32
145:         else hits: n/a
146:           raise( hits: 22
147:             RuntimeError.new( hits: n/a
148:               { hits: n/a
149:                 reason: 'Something sensible', hits: n/a
150: 
151:                 arg_k: arg_k, hits: n/a
152:                 arg_i: arg_i, hits: n/a
153:                 arg_d: arg_d, hits: n/a
154:                 arg_g: arg_g, hits: n/a
155:                 arg_f: arg_f, hits: n/a
156:                 key_a: arg_b&.key_a, hits: n/a
157:                 arg_a: arg_a, hits: n/a
158:                 arg_e: arg_e, hits: n/a
159:                 arg_c: arg_c, hits: n/a
160:                 arg_l: arg_l, hits: n/a
161:                 arg_h: arg_h, hits: n/a
162:                 arg_j: arg_j, hits: n/a
163:               }, hits: n/a
164:             ), hits: n/a
165:           ) hits: n/a
166:         end hits: n/a
167:       end hits: n/a
168:     end hits: n/a
Undercover finished in 0.953s

Relevant lines from lcov file:

…
SF:./foo/applications/bar/services/baz.rb
…
DA:92,1
DA:106,181
DA:107,181
DA:108,181
DA:110,181
DA:115,181
DA:116,2
DA:119,179
DA:127,179
DA:134,179
DA:135,125
DA:137,54
DA:143,54
DA:144,32
DA:146,22
…

Unknown parser errors

to be addressed with a version lock in imagen
known issue in: whitequark/parser#518

Loosen requirement on rugged ?

Is it possible to loosen the gem version requirement for rugged ? We are currently using the 1.2.0 version of rugged.

Maybe it's possible to use < 2.0, similar to https://rubygems.org/gems/licensee ?

Friendly error message when coverage is not set up

This would make things easier for users who don't RTFM

• show an instruction instead of an exception when lcov cannot be guessed or read from the cli option
• maybe improve the guessing mechanism?
• maybe support something else than lcov as well?

Does not find uncovered sinatra routes

Undercover does not find this uncovered sinatra route:

get '/dummy_path' do
  puts "This line is not covered"
end

git diff master:

+    get '/dummy_path' do
+      puts "This line is not covered"
+    end

undercover -c master:

undercover: ✅ No coverage is missing in latest changes
Undercover finished in 0.0505s

simplecov HTML formatter output:

Replace node classes with human types

Current output of undercover includes class names, like:

🚨 5) node `parse` type: Imagen::Node::IMethod,
      loc: lib/undercover/options.rb:34:58,coverage: 69.23%

Replace type with a friendly text like "instance method", "class method" or "class". This involves adding a method to subclasses of Imagen::Node and displaying it in Undercover::Formatter

Project path and lcov path options are failing

Reproduce

• run from a directory different than target dir: undercover -p path/to/project
• fails to guess the lcov path, because it starts looking in pwd

Or:

• run from a directory different than target dir: undercover -p path/to/project --lcov path/to/project/lcov.lcov
• fails to parse files, because parser receives relative paths (need to prepend project path or make them absolute)

Sample backtrace:

./undercover/bin/undercover -p undercover -l undercover/coverage/lcov/undercover.lcov -c HEAD~5
/Users/mrgrodo/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/parser-2.5.1.0/lib/parser/base.rb:65:in `read': No such file or directory @ rb_sysopen - lib/undercover.rb (Errno::ENOENT)
	from /Users/mrgrodo/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/parser-2.5.1.0/lib/parser/base.rb:65:in `parse_file'
	from /Users/mrgrodo/dev/undercover/lib/undercover.rb:80:in `block in each_result_arg'
	from /Users/mrgrodo/dev/undercover/lib/undercover.rb:78:in `each'
	from /Users/mrgrodo/dev/undercover/lib/undercover.rb:78:in `each_result_arg'
	from /Users/mrgrodo/dev/undercover/lib/undercover.rb:35:in `build'
	from /Users/mrgrodo/dev/undercover/lib/undercover/cli.rb:28:in `run'
	from ./undercover/bin/undercover:11:in `block in <main>'
	from /Users/mrgrodo/.rbenv/versions/2.4.4/lib/ruby/2.4.0/benchmark.rb:308:in `realtime'
	from ./undercover/bin/undercover:10:in `<main>'

EncodingError: invalid byte sequence in US-ASCII

Hi @grodowski

I got the following problem when tried to use undercover... Any thought where should I start to fix this? 👇

Report on files from changeset not present in the lcov

undercover should create warnings on entire files that weren't required by specs and hence don't show up in the lcov output.

This can be done by changing logic starting from Undercover::Report#load_and_parse_file

Improve analysis performance

Stale coverage warning after cloning the repo

Using undercover in CI may be problematic, because given such a workflow:

- job 1:
  - clone
  - run specs
  - store coverage report
- job 2:
  - clone
  - restore coverage
  - run undercover

we will get the stale coverage message, because files from second clone have newer mtime than the produced coverage report.

Fix: switch from using mtime to git metadata in Undercover::Changeset#last_modified?
Alternative: turn the stage coverage error to a warning, because it's only useful for local usage.

rainbow 3.X compatibility

Hi @grodowski !
rainbow has released 3.X ver in 2017 :)
we actually switched and using it now
and we can not start using undercover now, because dependency in latest version rainbow ~> 2.1:
the issue we have:

ostet@daisy:~/coding/overhaul-backend$bundle install
Warning: the running version of Bundler (1.17.2) is older than the version that created the lockfile (1.17.3). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.
Fetching gem metadata from https://rubygems.org/.......
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies........
Bundler could not find compatible versions for gem "rainbow":
  In snapshot (Gemfile.lock):
    rainbow (= 3.0.0)

  In Gemfile:
    pronto was resolved to 0.10.0, which depends on
      rainbow (>= 2.2, < 4.0)

    pronto-reek was resolved to 0.10.0, which depends on
      reek (>= 4.2, < 6.0) was resolved to 5.3.1, which depends on
        rainbow (>= 2.0, < 4.0)

    rubocop-rails_config was resolved to 0.4.3, which depends on
      rubocop (~> 0.58) was resolved to 0.63.1, which depends on
        rainbow (>= 2.2.2, < 4.0)

    undercover (~> 0.3.0) was resolved to 0.3.0, which depends on
      rainbow (~> 2.1)

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

would be nice to have undercover compatible with latest rainbow
thanks in advance for any help
have a nice day!

Undercover is slow comparing binary files

Normally:

$ bundle exec undercover -c "$(git merge-base origin/master HEAD)"
🚨 WARNING: Coverage data is older than your latest changes and results might be incomplete. Re-run tests to update
undercover: ✅ No coverage is missing in latest changes
Undercover finished in 20.747s

With a binary file in the staging area:

$ git s
On branch test-merge-base
Your branch is up to date with 'origin/test-merge-base'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	git-cheat.pdf

nothing added to commit but untracked files present (use "git add" to track)
$ git add git-cheat.pdf 
$ ls -lah git-cheat.pdf 
-rw-r--r--@ 1 tkuah  staff   7.0M 23 Jun 11:59 git-cheat.pdf
$ bundle exec undercover -c "$(git merge-base origin/master HEAD)"
🚨 WARNING: Coverage data is older than your latest changes and results might be incomplete. Re-run tests to update
undercover: ✅ No coverage is missing in latest changes
Undercover finished in 119.039s

Do you think it might be OK to add an option to exclude binary files ?

empty diff on changeset with two refs from master

reproduce with a changeset comparing HEAD and HEAD~1 using commit refs

Create readme

undercover with out LCOV

I wanted to have both simplecov reports and also undercover missed test cases.

But right now undercover rely on LCOV report of RSpec, without that it is failing.

(We wont be having any coverage report)Since our Rspec run time is high, we cannot run more than one time. Do we have a solution for this?

Integrate with circlemator

Lack of a formatters interface

Hello @grodowski 👋

First of all, I'd like to say thank you for this great project! I've been using it extensively for the past year on the CI of a few internal repositories and it's quite handy to me and my peers. Great work!

After some time with undercover however, I feel that it lacks a formatter interface. I see on this line a comment that even hints that a formatter interface was planned: https://github.com/grodowski/undercover/blob/master/lib/undercover/result.rb#L63

    # TODO: create a formatter interface instead and add some tests.

That said, this is something that concerns me so I'd like to help 😄 I've been actually working a bit on some changes (which you can check here) that aim de-couple the Result from Formatter - thus allowing other formatters to be used.
I would personally like to see this being possible (which I think is exactly how rubocop handle this):

undercover --require $(pwd)/my_custom_formatter.rb --formatter MyCustomFormatter

# or if the formatter is on a gem and the gem is on the gemfile along with undercover:
bundle exec undercover --formtter MyCustomFormatter

I'm a bit concerned though as this would be a slightly big change - so I'd like to ask your opinion before opening my first PR on this project. What do you think @grodowski? Can I go ahead and try to tackle this one? 😄

ability to parse results.json from simplecov

Warn on added files from changelog that have no specs

Currently a file will have warning only if it is present in the LCOV report, if we add it but it's not loaded anywhere in the spec, it won't trigger warnings.

Upgrade to pronto (-> 0.10.0)

Hey, I'm having some issues with installing pronto-undercover alongside other plugins e.g. (rubocop, etc) due undercover still uses pronto version 0.9.0, Do you think'd be easy to upgrade?!

Thanks

Add --no-fail option to override the default exit code

That will be useful for experimenting with the CLI in the build environment, so that it never fails but just prints out the warnings

Run mode select

I had a couple ideas for the diff matching algo, which can be represented as a "run mode" option. Could be overkill for now!

cc @kruszczynski

Run Mode	Behaviour
DIFF_FILE	Reports on all blocks with missing coverage in every added / changed file from Changeset. That's the nazi mode that forces you to improve coverage for every file you touch
DIFF_BLOCK	Reports on all code blocks with missing coverage that have changed lines in them. Fosters fixing coverage for methods/blocks we changed.
DIFF_LINE_STRICT	Reports on code blocks with missing coverage, where specifically the lines we changed are missing coverage (that's how it works now).

Travis CI example config runs git pull into branch

It looks like with the way Travis CI works, if you git pull origin master:master, it tries to pull in masters changes into the checked out branch causing it to fail. Example build:

https://travis-ci.com/github/tomekr/jamroulette/jobs/302690415

Switching it to git fetch origin master:master fixed the issue for me.

Integrate with GitHub's Checks API

https://blog.github.com/2018-05-07-introducing-checks-api/

Support Ruby < 2

Migrating old projects involves migrating to new versions of Ruby. But I cannot do this without touching project code. Undercover would be the perfect tool here, but I cannot use it, because of imagen's dependency. But since imagen is also your gem, I guess this is not a problem, right 🤓 ?

File with a flat function list not parsed correctly

Given a file like

def untested
  "foo"
  "bar"
end

def tested
  "foo"
  "bar"
end

only the untested function would be parsed, because of lib/undercover.rb:97

Add a way to ignore coverage checks

Needs specification, but it would be useful to mark files or parts of them as nocov. Could be good to make it an unobtrusive way (external yml file?)

NameError: uninitialized constant Imagen::AVAILABLE_RUBY_VERSIONS

Hi!

When using the gem imagen in version 0.1.2, it raises an error as the following:

$ bundle exec undercover --version
warning: parser/current is loading parser/ruby22, which recognizes
warning: 2.2.x-compliant syntax, but you are running 2.3.5.
warning: please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
bundler: failed to load command: undercover (/home/fabioperrella/.rbenv/versions/2.3.5/bin/undercover)
NameError: uninitialized constant Imagen::AVAILABLE_RUBY_VERSIONS
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/undercover-0.3.2/lib/undercover/options.rb:111:in `ruby_syntax_option'
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/undercover-0.3.2/lib/undercover/options.rb:53:in `block in parse'
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/2.3.0/optparse.rb:1062:in `initialize'
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/undercover-0.3.2/lib/undercover/options.rb:36:in `new'
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/undercover-0.3.2/lib/undercover/options.rb:36:in `parse'
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/undercover-0.3.2/lib/undercover/cli.rb:18:in `run'
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/undercover-0.3.2/bin/undercover:11:in `block in <top (required)>'
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
  /home/fabioperrella/.rbenv/versions/2.3.5/lib/ruby/gems/2.3.0/gems/undercover-0.3.2/bin/undercover:10:in `<top (required)>'
  /home/fabioperrella/.rbenv/versions/2.3.5/bin/undercover:23:in `load'
  /home/fabioperrella/.rbenv/versions/2.3.5/bin/undercover:23:in `<top (required)>'

I think it is necessary to change the gemspec file and require the correct minimum version for the gem imagen to avoid this problem.

Create a Code Climate engine

In order for undercover to work with Code Climate (similar to Rubocop) it could have its own "engine". More info about engines here: https://docs.codeclimate.com/docs/building-a-code-climate-engine .

Wrong exit status when no changes

Breaks the master build 😱

$ undercover --compare master
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_parser.rb:5306:1-5307:0: error: literal contains escape sequences incompatible with UTF-8
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_parser.rb:5306:          \xd0\xd2\xcf\xd7\xc5\xd2\xcb\xc1 = 42
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_parser.rb:5306: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_parser.rb:5307:          puts \xd0\xd2\xcf\xd7\xc5\xd2\xcb\xc1}.b,
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_parser.rb:5307:                                                   
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_parser.rb: literal contains escape sequences incompatible with UTF-8 literal contains escape sequences incompatible with UTF-8
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_source_buffer.rb:44:9: error: literal contains escape sequences incompatible with UTF-8
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_source_buffer.rb:44:         "# \xf9"
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_source_buffer.rb:44:         ^~~~~~~~
./vendor/bundle/ruby/2.5.0/gems/parser-2.5.1.0/test/test_source_buffer.rb: literal contains escape sequences incompatible with UTF-8 literal contains escape sequences incompatible with UTF-8
✅ No reportable changes
Undercover finished in 52.043931070000326ms
The command "undercover --compare master" exited with 1.

Unable to use on Heroku CI

I'm not sure if I'm doing something wrong, but it doesn't look like a Heroku CI build dyno contains a copy of the .git repo.
As such there is no way for this tool to check against that and run correctly, am I correct?

I posted a more generic question attempting to double-check Heroku CI dyno here https://stackoverflow.com/questions/62377009/how-can-i-access-git-files-on-a-heroku-ci-dyno

Add Changeset information to the printed report

for more visibility on why things are reported... TBD

Branch coverage

Do not parse the whole project dir with imagen (perf + get rid of utf-8 parser errors)

• bundle install --deployment installs to ./vendor/bundle
• undercover does not skip this directory while parsing the project

A reasonable solution might be to change the parsing strategy to explicitly parse individual files referenced in Changeset and LcovParser while building the cov report. This is also good for performance which is terrible ATM!

$ git fetch origin master && undercover --compare origin/master
From https://github.com/grodowski/undercover
 * branch            master     -> FETCH_HEAD
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_parser.rb:5306:1-5307:0: error: literal contains escape sequences incompatible with UTF-8
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_parser.rb:5306:          \xd0\xd2\xcf\xd7\xc5\xd2\xcb\xc1 = 42
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_parser.rb:5306: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_parser.rb:5307:          puts \xd0\xd2\xcf\xd7\xc5\xd2\xcb\xc1}.b,
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_parser.rb:5307:                                                   
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_parser.rb: literal contains escape sequences incompatible with UTF-8 literal contains escape sequences incompatible with UTF-8
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_source_buffer.rb:44:9: error: literal contains escape sequences incompatible with UTF-8
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_source_buffer.rb:44:         "# \xf9"
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_source_buffer.rb:44:         ^~~~~~~~
./vendor/bundle/ruby/2.4.0/gems/parser-2.5.1.0/test/test_source_buffer.rb: literal contains escape sequences incompatible with UTF-8 literal contains escape sequences incompatible with UTF-8
undercover: 👮‍♂️ some methods have no test coverage! Please add specs for methods listed below
🚨 1) node `untested` type: Imagen::Node::IMethod,
      loc: lib/undercover/formatter.rb:25:31, coverage: 0.0%
25:     def untested hits: n/a
26:       if 'lol'.nil? hits: 0
27:         puts 'wow' hits: 0
28:         exit 8 hits: 0
29:       end hits: n/a
30:       0 hits: 0
31:     end hits: n/a
🚨 2) node `success` type: Imagen::Node::IMethod,
      loc: lib/undercover/formatter.rb:33:40, coverage: 50.0%
33:     def success hits: n/a
34:       if 'lol'.nil? hits: 4
35:         puts 'wow' hits: 0
36:         puts 'blah' hits: 0
37:       end hits: n/a
38:       "#{Rainbow('undercover').bold.green}: ✅ No coverage" \ hits: 4
39:       ' is missing in latest changes' hits: n/a
40:     end hits: n/a
🚨 3) node `initialize` type: Imagen::Node::IMethod,
      loc: lib/undercover/lcov_parser.rb:47:51, coverage: 0.0%
47:     def initialize(test) hits: n/a
48:       @test = test hits: 0
49:       @baz = 'bar' hits: 0
50:       @foo = 'bacon' hits: 0
51:     end hits: n/a
Undercover finished in 36.06677220900019ms
The command "git fetch origin master && undercover --compare origin/master" exited with 1.

Undercover fails with shallow repository + grafted commits

On GitLab CI, the default depth for the cloned git repository is 50 (for projects like https://gitlab.com/gitlab-org/gitlab/, we set it even lower to 20, for speed purposes).

This leads to errors like below

$ cat scripts/undercoverage
#!/usr/bin/env bash

bundle exec undercover -c "${1:-$(git merge-base origin/master HEAD)}"

$ UNDERCOVERAGE_COMPARE="${CI_MERGE_REQUEST_TARGET_BRANCH_SHA:-$CI_MERGE_REQUEST_DIFF_BASE_SHA}"
$ echo $UNDERCOVERAGE_COMPARE
b0238c51c69da80150745d92be243dc244d947b9

$ scripts/undercoverage ${UNDERCOVERAGE_COMPARE}
bundler: failed to load command: undercover (/builds/gitlab-org/gitlab/vendor/ruby/2.7.0/bin/undercover)
/builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/lib/undercover/changeset.rb:77:in `merge_base': object not found - no match for id (a0036d11ff015325fe97e86aa0278b3e01e13bb3) (Rugged::OdbError)
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/lib/undercover/changeset.rb:77:in `compare_base_obj'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/lib/undercover/changeset.rb:70:in `full_diff'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/lib/undercover/changeset.rb:27:in `update'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/lib/undercover.rb:34:in `initialize'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/lib/undercover/cli.rb:26:in `new'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/lib/undercover/cli.rb:26:in `run_report'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/lib/undercover/cli.rb:21:in `run'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/bin/undercover:12:in `block in <top (required)>'
	from /usr/local/lib/ruby/2.7.0/benchmark.rb:308:in `realtime'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/gems/undercover-0.4.4/bin/undercover:11:in `<top (required)>'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/bin/undercover:25:in `load'
	from /builds/gitlab-org/gitlab/vendor/ruby/2.7.0/bin/undercover:25:in `<top (required)>'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/cli/exec.rb:58:in `load'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/cli/exec.rb:58:in `kernel_load'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/cli/exec.rb:23:in `run'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/cli.rb:483:in `exec'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/cli.rb:31:in `dispatch'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/cli.rb:25:in `start'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/exe/bundle:48:in `block in <top (required)>'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	from /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.3.15/exe/bundle:36:in `<top (required)>'
	from /usr/local/bin/bundle:23:in `load'
	from /usr/local/bin/bundle:23:in `<main>'

From https://gitlab.com/gitlab-org/gitlab/-/jobs/2627986054

Steps to reproduce

1. Setup merged result pipelines on a GitLab project (https://docs.gitlab.com/ee/ci/pipelines/merged_results_pipelines.html)
2. Create a merge request
3. In the CI, run undercover -c ${CI_MERGE_REQUEST_TARGET_BRANCH_SHA}
4. If the merge request's branch is old enough, it should fail with the above error.

libgit2/rugged#846 (comment) seems related.

Add RuboCop to Rakefile

report JSON marshalling

Print undercover result to a file

I was trying to use the Undercover gem with Danger. For the Danger to read data from undercover, the report was needed to be printed in a file.

Is it possible to add a -o $output_path option within undercover which can be super useful for a lot of people out there?

Trouble getting started

I followed the Installation and Setting up required LCOV reporting steps, but I cannot run undercover. Terminal gives back "command not found: undercover"

After running bundle exec rspec, I see that coverage/lcov/*.lcov was generated successfully.

Have you seen this issue before?

literal contains escape sequences incompatible with UTF-8

• fix and update imagen to handle this failure
• bump version and release

Relates to whitequark/parser#283

Undercover picking up changes from master and reporting false negative

I have a branch which implements undercover in our ci (buildkite) pipeline. I have added undercover as a step after all specs run. I see false negative results every now and then. The changes that undercover points out needs coverage are always things someone else on the team has merged to master recently.

The command that I run is undercover --lcov <file.lcov> --compare origin/master

Steps:

• Create a branch and add all logic for undercover to work.(logic - Generate all lcov files in each ci container, aggregate them into one file which is passed as an argument to undercover command. Similar to what is described here - https://github.com/grodowski/undercover/blob/master/docs/circleci_advanced.yml )

• Push the branch to build on ci - it works fine so far. Undercover does not identify any non-covered changes

• Pull master into my branch and push branch to origin

• The ci build that was triggered shows bunch of methods as not covered. These are changes already on master, merged by team member recently.

• The changes which undercover reports as uncovered are not part of what I am working on.

Set up undercover to check undercover specs in .travis.yml

Add [files] CLI argument to allow more control over what's reported

Support ERB/view test coverage (ruby 3.2+)

One-line update is not being recognized

undercover/lib/undercover/result.rb

Line 16 in a5a26c1

ln > first_line && ln < last_line

I came across an issue where, if I update just one line block in a file, the first_line and last_line of the Result are the same, and the @coverage becomes an empty array.

Is this intentional? If so, could you please explain the reasoning behind it? Thanks!

Add CLI option to filter files