dnaka91 / llvm-cov-pretty

More beautiful HTML reports for llvm-cov/cargo-llvm-cov

💄 llvm-cov-pretty

More beautiful HTML reports for llvm-cov (cargo-llvm-cov specifically).

• Dark theme support (switches automatically based on your browser setting).
• Syntax highlighting.
• Reduced clutter from instantiation annotations.\
• More colors, because why not.
• Progress bars (like in grcov).

A note on stability

This tool is still in its early stages, and the JSON output of llvm-cov has several quirks and complexities. Therefore, for the time being, always generate the regular HTML report as well and compare the output.

Please file an issue if you find any differences.

Installation

Make sure you have cargo-llvm-cov installed.

From source

To build the project from source, you need a recent installation of Rust. The recommended installation method is through rustup. Then open up a terminal and install the project as follows:

cargo install llvm-cov-pretty

The binary will be installed into your $HOME/.cargo/bin folder by default. Please make sure it is available from your $PATH.

From AUR

Arch linux users can use an AUR helper to install the project from the AUR. For example, using paru:

paru -S llvm-cov-pretty

Pre-build binaries

If you don't want to build the project yourself, you can download pre-build binaries instead. Have a look at the release page and find the appropriate file for your platform.

Requirements

Before you can use this tool, make sure the following steps are done:

• In addition to installing cargo-llvm-cov, make sure you've run it at least once on your machine in any Rust project. It will jump into interactive mode and install any missing tools that it needs itself.
• It is expected that you are in the context of a Rust project. Ensure you invoke the program while located in a Rust project in your terminal.
  • However, you can pass the --manifest-path argument, but make sure you pass the same path to both cargo-llvm-cov and llvm-cov-pretty.

Usage

The tool operates on the JSON output from llvm-cov. As cargo-llvm-cov directly prints those to the standard output, you can pipe the programs together like so:

cargo llvm-cov --json | llvm-cov-pretty

Of course you can do this in two steps as well:

cargo llvm-cov --json > coverage.json
llvm-cov-pretty coverage.json

There are a few extra commands (like generating shell completions) and options (like disabling instantiation annotations) as well. Have a look at the output of llvm-cov-pretty --help.

Development

To start developing on the project, you'll need a few extra step in addition to the ones described in the From source section.

• Install a recent version of Node.js and PNPM.
• Ensure all submodules are initialized with git submodule update --init.

Then open up a new terminal and execute pnpm run watch. It will watch the project files and rebuild the stylesheet at assets/style.css when needed. With that you're ready to hack along.

Before you create a custom build, it's recommended to stop the watch job again and run pnpm run build once, which will create a minified and optimized version of the stylesheet. It is compiled into the binary during builds.

License

This project is licensed under the AGPL-3.0 License (or https://www.gnu.org/licenses/agpl-3.0.html).

llvm-cov-pretty's Issues

"80% function coverage" but you can't tell where the problem is

Describe the bug

I'm attaching three HTML files to this report. The first is llvm-cov show -format=html output for one file of a project I'm working on, and the other two are llvm-cov-pretty output for the same file, one generated with --skip-function-coverage and the other without. (All files have been put through htmltidy and then renamed with a .txt extension so Github will let me upload them.)

In the basic llvm-cov output, one can easily see that there is only one coverage region that was missed in testing: the lambda on line 73 was never called.

In llvm-cov-pretty output, the summary report at the top agrees with the basic llvm-cov output, but it's impossible to tell where the uncovered region is. In the file generated with --skip-function-coverage, every line of code is marked as fully covered (green background). In the file generated without that flag, line 73 is marked as partially covered but so are 17 other lines. Furthermore, the "jump to first uncovered line" link does not work (it has no href attribute).

(I happen to know, because I've been tinkering with coverage reports for this project for a while now, that those 17 other lines would appear to be partially covered in llvm-cov output as well if I had used -show-instantiations=true. What's going on is, except for line 73, each region is reached at least once in at least one of the monomorphizations of the generic functions in this file, but not in every monomorphization. Achieving 100% coverage of every monomorphization is not nearly as important as ensuring that there aren't any code paths that are never tested at all, so it's really important to be able to distinguish these two cases.)

I'm also attaching the .json file I used as input to llvm-cov-pretty.

Expected behavior

In llvm-cov-pretty output:

1. Line 73 should be visibly different from all other lines in this file.
2. The "jump to first uncovered line" link should take me to line 73.
3. Preferably, line 73 should have its uncovered regions highlighted, more or less the same as how llvm-cov does it.
4. It would be nice if each region of a generic function, that is reached at least once in some monomorphization of that function, would be treated as fully covered but there would be some kind of unobtrusive indication that it's not reached in all monomorphizations, and you could click on it to see details (the <summary> and <details> tags can do this without needing JS).

To reproduce

Please let me know if you need me to pull together a minimized test case for this problem.

What operating system are you using?

Linux

---

llvm-cov show -format=html output
llvm-cov-pretty output
llvm-cov-pretty --skip-function-coverage output
coverage.json.gz

Coverage info doesn't match 100% with `llvm-cov`'s report

Describe the bug

I noticed that the displayed coverage isn't always the same as seen in the regular llvm-cov output. Already fixed partially locally, but especially the regions for missing instantiations are not correct.

Expected behavior

The report should always match 100% with what is visible in llvm-cov. Some parts may of course be different in how they're displayed, but the information about what lines are not covered should always be correct.

To reproduce

No response

What operating system are you using?

None

Terminal output

No response

Additional context

This will need some more tinkering, as I didn't find the pattern in the JSON data yet, that would allow me to match the instantiation messages 100% with llvm-cov.

But at least the basic coverage data is already fixed, and a new version will be released soon.

No files listed in the report

Describe the bug

I ran cargo install llvm-cov-pretty, and then tried with a project without a workspace as well as a project with a workspace. In both, the report generated contains the correct overview information, but there are no files listed to poke into. The target folder only contains index.html, style.css, and syntax.css.

Expected behavior

I expected beautiful results like you shared in discord! :)

To reproduce

1. Clone https://github.com/khonsulabs/alot
2. Execute cargo llvm-cov --json | llvm-cov-pretty
3. Open the report

What operating system are you using?

None

Terminal output

No response

Additional context

Because my output for llvm-cov may be different (why? I don't know!) I've uploaded the json output in this gist.

missing field `cargo_args`

Describe the bug

I'm getting the error below when running this command:

cargo llvm-cov --features mocks --json | llvm-cov-pretty

Error: 
   0: failed parsing report data from STDIN
   1: missing field `cargo_args` at line 1 column 44038817

Note: JSON path: cargo_llvm_cov

Expected behavior

Report should be generated without any errors

To reproduce

No response

What operating system are you using?

Linux

Terminal output

No response

Additional context

llvm-cov-pretty 0.1.8
cargo-llvm-cov 0.5.33

Everything is highlihted as uncovered

Describe the bug

Hi,

I just installed llvm-cov-pretty, and ran it on a project.

At first glance the metrics seem to match the cargo llvm-cov html report.

However in the file source view, everything single executable line is highlighted as uncovered (wrapped in a <td class="uncovered">). That's a lot of red.

Expected behavior

Lines should be highlighted correctly.

What operating system are you using?

Linux

Additional context

Here's the formatted JSON object for a small file showing 100% line coverage:

{
  "branches": [],
  "expansions": [],
  "filename": "/path/to/mod.rs",
  "segments": [
    [21, 5, 22, true, true, false],
    [23, 6, 0, false, false, false],
    [26, 5, 22, true, true, false],
    [28, 6, 0, false, false, false],
    [38, 5, 1, true, true, false],
    [40, 6, 0, false, false, false],
    [44, 5, 6, true, true, false],
    [46, 6, 0, false, false, false]
  ],
  "summary": {
    "branches": {
      "count": 0,
      "covered": 0,
      "notcovered": 0,
      "percent": 0
    },
    "functions": {
      "count": 4,
      "covered": 4,
      "percent": 100
    },
    "instantiations": {
      "count": 70,
      "covered": 29,
      "percent": 41.42857142857143
    },
    "lines": {
      "count": 12,
      "covered": 12,
      "percent": 100
    },
    "regions": {
      "count": 4,
      "covered": 4,
      "notcovered": 0,
      "percent": 100
    }
  }
}

Add --output-dir option and document default location for output

Is your feature request related to a problem? Please describe.

I had to dig into the source code to learn that llvm-cov-pretty writes its output to the directory target/llvm-cov-pretty (where target is the Cargo target directory for the project). There doesn't seem to be any way to override this location.

Describe the solution you'd like

1. Please document the default output directory in the top-level README and the output of llvm-cov-pretty --help, and in the manual when there is one.
2. Please add a command line option (I suggest --output-dir, matching cargo-llvm-cov) that changes the output directory. This option should also be documented in the top-level README and in llvm-cov-pretty --help.