ory / go-acc

Accurate Code Coverage reporting for Golang (Go)

go-acc

A tool for reporting accurate Code Coverage in Golang. It is a cross platform (osx, windows, linux) adaption of the following bash script:

touch ./coverage.tmp
echo 'mode: atomic' > coverage.txt
go list ./... | grep -v /cmd | grep -v /vendor | xargs -n1 -I{} sh -c 'go test -race -covermode=atomic -coverprofile=coverage.tmp -coverpkg $(go list ./... | grep -v /vendor | tr "\n" ",") {} && tail -n +2 coverage.tmp >> coverage.txt || exit 255' && rm coverage.tmp

Installation & Usage

$ go install github.com/ory/go-acc@latest
$ go-acc
A tool for reporting accurate Code Coverage in Golang.

Usage:
  go-acc <packages...> [flags]

Examples:
$ go-acc github.com/some/package
$ go-acc -o my-coverfile.txt github.com/some/package
$ go-acc ./...
$ go-acc $(glide novendor)

Flags:
      --covermode string   Which code coverage mode to use (default "atomic")
      --ignore strings     Will ignore packages that contains any of these strings
  -o, --output string      Location for the output file (default "coverage.txt")
  -t, --toggle             Help message for toggle
      --tags               Build tags for go build and go test commands

You can pass regular go flags in bash after --, for example:

go-acc ./... -- -v -failfast -timeout=20m -tags sqlite

go-acc's Issues

Only cover packages that the tested package depends on

I get a lot of "warning: no packages being tested depend on packagename" errors because each package is tested with every other package, even if it doesn't import it. This also skews coverage, because I have a few generated packages that I don't want counted.

It would be better to only cover the packages that the tested package actually depends on. Something like this: master...nkovacs:master

Doesn't work with go 1.16

I just updated our project to go 1.16 and getting "unable to run go list: exit status" error

Modifies `go test -json` output, making it invalid JSON

The output of go test -json contains coverage summaries, e.g.,

{"Time":"2021-06-04T15:08:38.3398169-07:00","Action":"output","Package":"example","Output":"coverage: 14.8% of statements\n"}

These lines are modified by

go-acc/cmd/root.go

Line 187 in 65df6ab

fmt.Println(strings.Split(line, "% of statements in")[0])

Namely, the last part of the split is removed:

{"Time":"2021-06-04T15:08:38.3398169-07:00","Action":"output","Package":"example","Output":"coverage: 14.8

Unfortunately, that is no longer parsable JSON.

Ability to override `go test`

The repo https://github.com/rakyll/gotest provides a gotest binary which wraps go test and adds colors for FAIL/PASS lines. It would be nice if we can override go test internally with an argument, or with some environment variable.

go get fails because of hashicorp dependency

Not sure if this is a temporary misconfiguration of the HCL repo, but go-acc may want to point at a specific tag for the hcl module.

Daves-MacBook-Pro:go dave$ go get github.com/ory/go-acc
package github.com/hashicorp/hcl/hcl/printer: cannot find package "github.com/hashicorp/hcl/hcl/printer" in any of:
	/Users/dave/.gvm/gos/go1.14.9/src/github.com/hashicorp/hcl/hcl/printer (from $GOROOT)
	/Users/dave/go/src/github.com/hashicorp/hcl/hcl/printer (from $GOPATH)

go get github.com/ory/go-acc fails - cannot find package

fails with message

package go.opentelemetry.io/otel/plugin/httptrace: cannot find package "go.opentelemetry.io/otel/plugin/httptrace"

Reproduced for GO 1.14.3 on Mac and Linux

Question

Looking at the go.sum this is pulling in tons of items, is this due to using items in other ory pkgs.

Failing to compute coverage correctly

Hello,
it might be me doing some stupid mistake - in this case sorry for the bold issue title - but taking https://github.com/arekkas/accurate-test-coverage as example code base and running go-acc ./... on it (as mentioned in your post https://www.ory.sh/golang-go-code-coverage-accurate) I get the following coverage.txt:

mode: atomic
github.com/arekkas/accurate-test-coverage/otherpkg/cc.go:5.31,7.2 1 1
github.com/arekkas/accurate-test-coverage/pkg/funcs.go:3.23,5.2 1 1
github.com/arekkas/accurate-test-coverage/pkg/funcs.go:7.27,9.2 1 1
github.com/arekkas/accurate-test-coverage/pkg/funcs.go:3.23,5.2 1 1
github.com/arekkas/accurate-test-coverage/pkg/funcs.go:7.27,9.2 1 0

According to your blog post I would expect the HTML output to show all code as being executed, but it does not. Probably due to the fact the the function in question has to differing entries in coverage.txt.

I am using go in version 1.12.7 on macOS and the latest version of go-acc.

Where's the race flag?

In the readme, the goal of go-acc is to replicate this shell command:

go list ./... | grep -v /cmd | grep -v /vendor | xargs -n1 -I{} sh -c 'go test -race -covermode=atomic -coverprofile=coverage.tmp -coverpkg $(go list ./... | grep -v /vendor | tr "\n" ",") {} && tail -n +2 coverage.tmp >> coverage.txt || exit 255' && rm coverage.tmp

However, upon analysis of the command, it doesn't appear there's any mention of adding the -race flag to go test.

Is this expected? Can it be documented as such?

Fails to install with "too many errors" message

$ go get github.com/ory/go-acc
# golang.org/x/sys/unix
go/src/golang.org/x/sys/unix/fcntl_darwin.go:11:9: undefined: fcntl
go/src/golang.org/x/sys/unix/fcntl_darwin.go:16:12: undefined: fcntl
go/src/golang.org/x/sys/unix/ioctl.go:20:9: undefined: ioctl
go/src/golang.org/x/sys/unix/ioctl.go:29:9: undefined: ioctl
go/src/golang.org/x/sys/unix/ioctl.go:38:9: undefined: ioctl
go/src/golang.org/x/sys/unix/ioctl.go:48:9: undefined: ioctl
go/src/golang.org/x/sys/unix/ioctl.go:60:9: undefined: ioctl
go/src/golang.org/x/sys/unix/ioctl.go:66:9: undefined: ioctl
go/src/golang.org/x/sys/unix/syscall_bsd.go:630:10: undefined: mmap
go/src/golang.org/x/sys/unix/syscall_bsd.go:631:10: undefined: munmap
go/src/golang.org/x/sys/unix/ioctl.go:66:9: too many errors

This is with macOS HighSierra.

Differentiate between ignored testPackages and ignored coverPackages

In my project setup I have test packages like api_test which contain only tests for the package api. I want to execute these but I do not want to have coverage for the api_test package.
Furthermore I have a package test with test helpers functions. Tests from here could be executed, but yet again I do not want to include coverage data for this package.

With the --ignore flag packages are both excluded from the go test <packages> command, as well as the -coverpkg=<packages> option.

The ..._test package construct is common in Go, so I think it would be helpful to have different ignores for testPackages and coverPackages.

[README.md] relationship between go-acc and Go 1.20 GOCOVERDIR feature

For several years, go-acc has me served very well in my admittedly simple case of aggregating coverage data from root as well as a non-root test runs. A big thank-you for your work from a small fish.

Now, Go 1.20 is finally another interesting fish in the sea in this respect, with its new feature of aggregating multiple test and integration test runs. Aggregating multiple unit test runs can be achieved using go test ... -args -test.gocoverdir="GOCOVERDIR" (as "documented" here) .

How does this Go 1.20 feature relate to go-acc? Does it now the same or where are differences?

Do you think such a feature comparison to be useful to devs either using go-acc or thinking about using it?

Build tags should be passed into the `go list` call

For projects that use build tags, the tag argument is not currently passed into the go list call; causing some tests to not be listed. Instead of only allowing a passthrough argument; the tool should probably allow a --tags argument.

Coverage only on new code.

While raising a pull request there is a need for this feature where we can find the coverage for the changes it contains. Do suggest some workaround for this?

Can't install due to coreos/systemd dependency

When trying to install, I get

...
go get: upgrading github.com/coreos/[email protected]: unexpected status (https://proxy.golang.org/github.com/coreos/go-systemd/@v/list): 410 Gone
...
go get: error loading module requirements

I get this when running go get -u github.com/ory/go-acc inside of a Dockerfile with golang:1.12.4 as the base image.

go-acc ./... fails with message `exec: already started`

exec: already started
Exited with code exit status 1

Issue stared about 30 min ago

Likely related to this commit:
e95c7e3

reproducible sample - https://circleci.com/gh/dharnitski/go-coverage-sample/9

Line 155 should be removed. It is duplicate

../../ory/go-acc/cmd/root.go:182:2: missing return at end of function

go get -u github.com/ory/go-acc
go-acc ./...

github.com/ory/go-acc/cmd

../../ory/go-acc/cmd/root.go:182:2: missing return at end of function

[README.md] documentation of --tags CLI flag

Currently, the README.md shows a seemingly outdated and somehow less useful help for a --toggle flag:

  -t, --toggle             Help message for toggle

A moderately recent go-acc doesn't support --toggle, but instead --tags:

      --tags strings       Tags to include

How to ignore particular file(s)?

The --ignore seems only accepts packages to be ignored. How do I tell go-acc to ignore a single go file?

there is something wrong when i run go get github.com/ory/go-acc

hi guys ,you do great job,
but when i run go get github.com/ory/go-acc, it show me the code like below:

package golang.org/x/text/transform: unrecognized import path "golang.org/x/text/transform" (https fetch: Get https://golang.org/x/text/transform?go-get=1: dial tcp 216.239.37.1:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.)
package golang.org/x/text/unicode/norm: unrecognized import path "golang.org/x/text/unicode/norm" (https fetch: Get https://golang.org/x/text/unicode/norm?go-get=1: dial tcp 216.239.37.1:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.)

and when i try the url in the error code, i find the transform is move to https://godoc.org/golang.org/x/text/transform

i am new in go lang, wish you guys can fix it

Document CLI option --ignore expected format

Please explain the expected format for the --ignore flag arguments in the usage message. I'd like to give multiple package names, do I separate them with commas, pipes or? I tried to use the force and read the source but I'm not a go expert.

I see the following when I run go-acc with no arguments:

$ go-acc
Error: requires at least 1 arg(s), only received 0
Usage:
  go-acc <flags> <packages...> [flags]

Examples:
$ go-acc github.com/some/package
$ go-acc -o my-coverfile.txt github.com/some/package
$ go-acc ./...
$ go-acc $(glide novendor)

You can pass all flags defined by "go test" after "--":
$ go-acc . -- -short -v -failfast


Flags:
      --covermode string   Which code coverage mode to use (default "atomic")
  -h, --help               help for go-acc
      --ignore strings     Will ignore packages that contains any of these strings
  -o, --output string      Location for the output file (default "coverage.txt")
  -t, --toggle             Help message for toggle
  -v, --verbose            Does nothing, there for compatibility

requires at least 1 arg(s), only received 0

I'm using go 1.12.16 and I installed go-acc by doing go get -v github.com/ory/go-acc. I don't know how to report the version to you, you might consider adding a "--version" or similar flag so people like me can report more accurately! Thanks for listening.

Resolve build issues on Windows

See mattn/go-sqlite3#940

fix: correctly handle empty params

Hi again;

It seems that I accidentally introduced a rather insidious bug with the PR #33.

If the --tags argument is not passed; the line below:

c := exec.Command("go", "list", tagsArg, a)

is executed as go list "" ./...; not go list ./.... It therefore tries to list the current folder. If there are no Go files directly in the current folder; this will cause the go list command to fail. Therefore we must only add tagsArg if it is specified.

The same bug already existed in a more hidden way, with how go list command can be replaced using GO_TEST_BINARY. If the new command is only one word; then this part would insert an empty string for the second word:

if len(gt) != 2 {
   gt = append(gt, "")
}

which is how I noticed this bug in the first place. Of course, I opened a PR that excludes empty strings in both cases: #36.