nextest-rs / datatest-stable Goto Github PK

I'm not sure if this is just some kind of user error, but I can't see to get coverage information generated for my datatests.

Coverage information is being generated, but it only seems to contain the hits from the non-datatest tests. When I clear the '*.profraw' files and run just the datatest tests, there is 0% coverage.

Initially I tried running cargo llvm-cov --html, then I tried following the steps for external tests but that didn't work either.

What might make my use-case a bit unusual is the fact that the datatest tests are in their own crate in the workspace, and I'm expecting coverage information to be generated for another crate in the workspace.
I thought this structure could be part of the problem, so I tried replacing the datatest_stable::harness! macro with a main function that directly calls a function from the other crate, and then coverage information was successfully generated.

Any help on this issue would be greatly appreciated.

Let's say you have 10k files. Datatest-stable iterates over all files currently to build the list of tests. Since nextest has a process-per-test model, each test will iterate over every file. That's not great.

Instead, if --exact is passed in, we should be able to pinpoint the exact test and just use that.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

Renovate tried to run on this repository, but found these problems.

• WARN: Error updating branch: update failure

Errored

These updates encountered an error and will be retried. Click on a checkbox below to force a retry now.

• chore(deps): pin dependencies (cloudflare/pages-action, taiki-e/install-action)

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update codecov/codecov-action digest to e28ff12
• chore(deps): update taiki-e/install-action digest to 7277a40
• chore(deps): lock file maintenance
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

Hi,

I have been using this harness with Great Success (tm), apart from a minor inconvenience.

Background:

To reduce boilerplate with tests separated into modules, I made the following wrapper:

macro_rules! generate_harness{
    ($($test:ident),*) => {
        datatest_stable::harness!(
            $(
                $test::run_test,
                $test::test_dir(),
                $test::FILE_PATTERN,
            )*
        );
    }
}

generate_harness!(test_pdf_parser, test_structure_parser, test_reference_parsing);

Problem

Unfortunately if I modify the wrapper to use paths:

macro_rules! generate_harness{
    ($($test:path),*) => {
        datatest_stable::harness!(
            $(
                $test::run_test,
                $test::test_dir(),
                $test::FILE_PATTERN,
            )*
        );
    }
}

generate_harness!(pdf::test_pdf_parser, structure::test_structure_parser, grammar::test_reference_parsing);

Cause

It suddenly doesn't work, because rustc cannot properly comprehend $test::run_test if $test a path and not an ident.

This is due to rust-lang/rust#48067.

Solution?

Apparently, one possible workaround is using $Va : ident $( :: $Vb : ident )* instead of path, but I have not tested this.

Loosening $name to expr would also solve the problem I think.

I'd also be happy with any other idea. What I currently do is just import everything into the crate namespace, but that's going to be problematic sooner or later.

Use fancy_regex crate instead of standard regex crate.

Thank you very much.

Sometimes
developers want to skip some tests files for run and even want to place such test files in codebase skipping tests for them entirely.
or want to just run only one test file to save time.