Comments (14)
Maybe a tag for nf-core? I think new issues should be made, rather than just trying to keep track of what we want in one long thread.
Thanks for your suggestion. We created a nf-core
label and it would help us if you can create separate issues.
from nf-test.
@lukfor Can we change the color of the label to #24B064
?
from nf-test.
Hi @nvnieuwk!
We just released nf-test 0.7.0. This release contains now snapshot testing. This could be helpful for nf-core:
https://code.askimed.com/nf-test/assertions/snapshots/
Happy to hear your feedback!
Best, lukas.
from nf-test.
Awesome stuff! I'm really excited to get this into nf-core/modules! Do you have anything you can share about the road map for tags? I know we can use pytest-workflow to run nf-test just like we do nextflow right now, but I'd love to cut out the extra dependancy and migrate to nf-test in one move.
So I guess I'm just asking is it a few weeks off, before 2023, before or after March 2023 (our next hackathon which would be a good time to migrate).
Thanks. We plan to release this feature in the next version in about 2-3 weeks.
from nf-test.
Tags moved to #49
from nf-test.
Hi,
- Could it be possible to supply a
.yaml
file with the assertions (a bit similar to how it is done inpytest
=> e.g. this file).
Thanks for the suggestion and the thoughts. I understand that this would be quite useful for nf-core, but @seppinho and I are not sure if this also makes sense in the context of nf-test. nf-test is highly expressive and benefits by using the power assertions provided from Groovy and Nextflow specific extensions. However, we plan to integrate some kind of snapshot testing for output channels (similar to https://jestjs.io/docs/snapshot-testing). I think this could also be very helpful for you.
- Could it also be possible to add a
tag
system (similar topytest --tag
). This would make it easier to run the needed tests on Github Actions instead of having to supply the whole file path.
Tags are a very good idea and I see a lot of useful use cases! 👌 We will definitifly implement them in one of the next releases!
Best, lukas.
from nf-test.
Thanks for the suggestion and the thoughts. I understand that this would be quite useful for nf-core, but @seppinho and I are not sure if this also makes sense in the context of nf-test. nf-test is highly expressive and benefits by using the power assertions provided from Groovy and Nextflow specific extensions. However, we plan to integrate some kind of snapshot testing for output channels (similar to https://jestjs.io/docs/snapshot-testing). I think this could also be very helpful for you.
I like this idea! Could it also be possible to add snapshot testing to the files inside of the output directory? We mainly test those files instead of output channels. Thanks for the feedback!
Tags are a very good idea and I see a lot of useful use cases! ok_hand We will definitifly implement them in one of the next releases!
Awesome! Thanks for all the good work!
Cheers,
Nicolas
from nf-test.
Awesome stuff! I'm really excited to get this into nf-core/modules! Do you have anything you can share about the road map for tags? I know we can use pytest-workflow to run nf-test just like we do nextflow right now, but I'd love to cut out the extra dependancy and migrate to nf-test in one move.
So I guess I'm just asking is it a few weeks off, before 2023, before or after March 2023 (our next hackathon which would be a good time to migrate).
from nf-test.
Thanks @lukfor! I'll have a look when I got some free time and will surely let you know if we encounter any problems. Thanks for the nice addition 🥳
from nf-test.
Hi, I've been trying out the new snapshots
feature for a little bit and found one small issue. It won't run the snapshot test in the following example and I get no warning or error telling me what I did wrong.
nextflow_pipeline {
name "Tests of the pipeline with all optional parameters on default"
script "main.nf"
test("Success") {
expect {
assert workflow.success
assert snapshot(
workflow,
path("${outputDir}/families/Proband_12345/reports/Proband_12345.TsTv.count"),
path("${outputDir}/families/Proband_12345/reports/Proband_12345.bcftools_stats.txt"),
file("${outputDir}/families/Proband_12345/reports/Proband_12345.TsTv.qual").exists(),
path("${outputDir}/families/Proband_12345/reports/Proband_12345.FILTER.summary"),
path("${outputDir}/families/Proband_12345/Proband_12345.vcf.gz").linesGzip[68],
file("${outputDir}/families/Proband_12345/Proband_12345.vcf.gz.tbi").exists(),
file("${outputDir}/multiqc_reports/multiqc_report.html").exists(),
path("${outputDir}/individuals/NA24385/NA24385.g.vcf.gz").linesGzip[53],
file("${outputDir}/individuals/NA24385/NA24385.g.vcf.gz.tbi").exists(),
path("${outputDir}/individuals/NA12878/NA12878.g.vcf.gz").linesGzip[53],
file("${outputDir}/individuals/NA12878/NA12878.g.vcf.gz.tbi").exists()
)
}
}
}
This test checks for:
- md5sum of files that stay consistent during multiple runs
- the existence of files that differ during multiple runs and cannot be read using
linesGzip
- the presence of a line in a file if the file is variable but has some constants
Is it possible to implement a bit of error handling if invalid snapshots creation is attempted? I'll be happy to put this in a different issue if you would prefer it.
Cheers,
Nicolas
from nf-test.
You created the snapshot
but you forgot to execute the match()
method to compare it with the saved snapshot.
However, I agree with you that we need a better error reporting for that case.
from nf-test.
I'm not sure if we should keep a thread of all the nf-core requests here rather than seaprate issues, but just in case x-ref: #43
from nf-test.
Maybe a tag for nf-core? I think new issues should be made, rather than just trying to keep track of what we want in one long thread.
from nf-test.
You created the
snapshot
but you forgot to execute thematch()
method to compare it with the saved snapshot.However, I agree with you that we need a better error reporting for that case.
Thanks! 🥳
from nf-test.
Related Issues (20)
- Weird behaviour when Gzipped files are empty HOT 4
- [Docs] Broken link for navigating to getting started HOT 1
- Automatic staging of `assets` `bin` and `lib` folders broken HOT 1
- Repeated 'permission denied' when running `nf-test clean` or `rm -rf .nf-test` HOT 2
- Add --target option to nf-test list HOT 2
- is there a way we can allow for deliberate snapshot matching? HOT 3
- Only run tests for changed files HOT 3
- Test file not created for workflow when workflow.onComplete is in main.nf HOT 1
- Can snapshots entries be made unique by `test-name + id` instead of just the `id` ?
- Problem when pipeline param is the same as nf-test param
- Broken link HOT 2
- bug: When an input is passed to output with no modification it causes the snapshot to have the absolute path HOT 1
- Access to output channels of unnamed/implicit workflow
- Keyword Arguments
- Allow capture and assertion of `.command.{err,out,log}`
- Conda environment reuse HOT 1
- Need the ability to skip md5 check for some files in snapshot HOT 1
- All tests' meta/output/work directories are deleted and recreated unnecessarily
- Caching of setup fixtures' results HOT 1
- nextflow.config required in my nf-test directory
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nf-test.