outputDir point to the full path about nf-test

[byb121]

This answer can help as well, and probably more flexible.

[lukfor]

We are considering the following approach:

• We will leave $outputDir unchanged to maintain compatibility.
• Users can continue using this pre-created folder, ensuring existing tests remain unaffected.
• Instead, we will introduce $launchDir, which will point to the directory where the Nextflow process was initiated.
• As a result, the following variables will be available:
  • $baseDir -> /home/maxime/workspace/nf-core_sarek (we will also support $projectDir as an alias like in Nextflow)
  • $launchDir -> /home/maxime/workspace/nf-core_sarek/.nf-test/tests/6b4fc92d7235947cf7ae249c9cd9ebf
  • $outputDir -> /home/maxime/workspace/nf-core_sarek/.nf-test/tests/6b4fc92d7235947cf7ae249c9cd9ebf/output

Since the Nextflow process is now started in the directory $launchDir, you can set param.outdir = "results" in the when block and access the folder in the then closure using path("$launchDir/results").

Your example:

nextflow_workflow {

    name "Test workflow in main.nf"
    script "main.nf"
    workflow "MAIN"

    test("MAIN: default test") {
        when {
            workflow {
                """
                """

            }
            params {
                outdir = "results"
            }
        }
        then {
            assertAll(
                { assert workflow.success },
                { assert snapshot(path("$launchDir/results").list()).match("outputPath") }
            )
        }
    }
}

The content of themd5sum.csv file:

cat results/csv/md5sum.csv
results/test_1.fastq.gz.md5

We believe that this approach and the chosen variable names are consistent with Nextflow. Happy to hear your feedback.

[maxulysse]

BTW, solved it for now with:

            params {
                outdir = baseDir.toURI().relativize(outputDir.toURI()).toString()
            }

EDIT:

> cat /home/maxime/workspace/nf-core_sarek/.nf-test/tests/6b4fc92d7235947cf7ae249c9cd9ebf/meta/params.json
{"baseDir":"/home/maxime/workspace/nf-core_sarek","outputDir":"/home/maxime/workspace/nf-core_sarek/.nf-test/tests/6b4fc92d7235947cf7ae249c9cd9ebf/output","outdir":".nf-test/tests/6b4fc92d7235947cf7ae249c9cd9ebf/output"}%                                                                                                                                                                                                                                                                 

[maxulysse]

We could probably update the docs:
This part lead me to believe the path would be relative:

This variable points to the directory within the temporary test directory (.nf-test/tests/<test-dir>/output/).

[maxulysse]

Hi,
Here's a tiny reproducible example: https://github.com/maxulysse/nf-test_66

When I run locally with nextflow run . --outdir results

I get:

$ cat results/csv/md5sum.csv 
results/test_1.fastq.gz.md5

When I run with nf-test, since I can't use a relative outdir, I get:

cat .nf-test/tests/ed6d3e4396651c8a5a3aca14e4dce26f/output/csv/md5sum.csv 
/home/max/workspace/nf-test_paths/.nf-test/tests/ed6d3e4396651c8a5a3aca14e4dce26f/output/test_1.fastq.gz.md5

[maxulysse]

wonderful, I love it

[lukfor]

I would also add $moduleDir (which points to the location of the module script) and $moduleTestDir (which points to the location of the test script).

Potential use case: If a module contains its own test data in the modules folder, it would be possible to load the data via:

inputs[0]=file("$moduleTestDir/sample.fastq.gz"). // --> modules/local/my_module/tests/sample.fastq.gz

or

inputs[0]=file("$moduleDir/sample.fastq.gz"). // --> modules/local/my_module/sample.fastq.gz

[nvnieuwk]

Can you remove the baseDir from the outputDir? This would have the desired effect, right?

[maxulysse]

Hi,
Any update on that issue?
My tiny fix solves some issues for me, but not all of them

[lukfor]

Thanks 🙏

To be sure I got it right. the following output would fit?

cat .nf-test/tests/ed6d3e4396651c8a5a3aca14e4dce26f/output/csv/md5sum.csv 
.nf-test/tests/ed6d3e4396651c8a5a3aca14e4dce26f/output/test_1.fastq.gz.md5

or did you prefer:

cat .nf-test/tests/ed6d3e4396651c8a5a3aca14e4dce26f/output/csv/md5sum.csv 
output/test_1.fastq.gz.md5

ATM I am not sure what the best solution is: (a) update outputDir to relative path or (b) add a new variable with the relative path. If (a) breaks existing testcases, it could be tricky for users to find the cause.

[maxulysse]

option a would be good if we're sure that the path is reproducible in a different environment (let's say my computer and GHA).
option b would be closer to what we have in reality.
We're using a --outdir params in nf-core to specify the output folder, so the issue was also to get --outdir to point to a relative path within the nf-test framework.

[maxulysse]

we usually have the test data in a separate repo, but I like where this is going

[lukfor]

We have just released RC1 of version 0.8.0 🎉
This is a pre-release and can be installed using the following command:

curl -fsSL https://code.askimed.com/install/nf-test | bash -s 0.8.0-rc1

If everything works as expected, please feel free to close this issue. Thanks for your support! 🙏

[maxulysse]

Works as expected, thank you so much