becheran / ntest Goto Github PK
View Code? Open in Web Editor NEWTesting framework for rust enhancing the built-in library with some useful features.
License: MIT License
Testing framework for rust enhancing the built-in library with some useful features.
License: MIT License
Describe the bug
When I originally started using ntest::timeout
it was because I had tests that would infinitely loop and never complete. Back on version 0.3.4
this worked well, and I happened to never have updated in that project. When trying to use it again in a new project when diagnosing a stuck test I noticed newer versions would no longer deal with this.
To Reproduce
Create a crate containing
#[test]
#[ntest::timeout(1000)]
fn foo() {
loop {}
}
run cargo test
Expected behavior
The test to fail after 1 second
Support negative numbers for test cases (float and int)
Currently the timeout macro does not work when you add the timeout attribute to a unit test which returns a result.
e.g.
#[test]
#[timeout(100)]
fn no_timeout() -> Result<(), String> {
let fifty_millis = time::Duration::from_millis(50);
thread::sleep(fifty_millis);
Ok(())
}
Would it be possible for you to enhance the macro so that it can wrap such a function and return the correct type
zbus CI is failing because ntest seems to be using syn::export
that is not present/pub in the latest syn
release. Seems this API was always internal-only and shouldn't have been used.
Right now test cases will alwas generate [test] attributes. Need to generate #[tokio::test] cases https://docs.rs/tokio/0.2.9/tokio/
Description
VS Code has a pair of handy links for running or debugging individual tests. But when #[ntest::timeout(1000)]
is placed, they are removed.
To Reproduce
Put #[ntest::timeout(1000)]
over the testing function.
Expected behavior
"Run" and "Debug" links should remain.
Screenshots
example
Desktop (please complete the following information):
When using struct/enum or any other pascal case type as an argument for test_case
macro rust-analyzer gives a warning
Function
function_Name
should have snake_case name, e.g.function_name
(rust-analyzer incorrect-ident-case)
ntest version: 0.9.0
Rust Analyzer: v0.3.1481
rustc: 1.67.1
#[test_case(Environment::Production)]
fn check_env(env: Environment) {
...
}
Macro could compile function name into lower case variant
OS: Ubuntu 22.04.1 LTS
Heya, I see that ntest
had a 0.8.1 release on crates.io (https://crates.io/crates/ntest) but that release is not tagged on github, and it looks like the master branch is still on 0.8.0 (at least, the ntest crate's Cargo.toml claims to be version 0.8.0). According to the source uploaded to crates.io, version 0.8.1 was made from git commit a858d8a, but that commit does not exist on github.
Could you publish the latest changes?
Thanks for the great crate :)
Describe the bug
I'm building ntest 0.7.3 crate for Guix System. The crate itself builds fine. However, when I run the cargo test release
the process exits with the SIGILL: illegal instruction
.
$ cargo test --release
Fresh unicode-xid v0.2.1
Fresh ntest_proc_macro_helper v0.7.3
Fresh proc-macro2 v1.0.27
Fresh serde v1.0.123
Fresh quote v1.0.9
Fresh toml v0.5.8
Fresh syn v1.0.72
Fresh proc-macro-crate v0.1.5
Fresh ntest_test_cases v0.7.3
Fresh ntest_timeout v0.7.3
Fresh ntest v0.7.3 (/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3)
Finished release [optimized] target(s) in 0.04s
Running `/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3/target/release/deps/ntest-2de355d2c3fb1823`
running 18 tests
test tests::about_equal_f32 ... ok
test tests::assert_false ... ok
test tests::about_equal_f64 ... ok
test tests::assert_false_fails ... ok
test tests::assert_false_trailing_comma ... ok
test tests::assert_panics_fails ... ok
test tests::assert_true ... ok
test tests::assert_panics ... ok
test tests::it_should_not_panic_if_values_are_approx_equal ... ok
test tests::assert_true_fails ... ok
test tests::compare_with_epsilon ... ok
test tests::assert_true_trailing_comma ... ok
test tests::fail_with_epsilon ... ok
test tests::assert_panics_trailing_comma ... ok
test tests::vector_trailing_comma ... ok
test tests::vector_fails ... ok
test tests::vector ... ok
test tests::vector_trailing_comma_with_epsilon ... ok
test result: ok. 18 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
Running `/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3/target/release/deps/integration-364ecb9c23d58916`
running 6 tests
error: test failed, to rerun pass '--test integration'
Caused by:
process didn't exit successfully: `/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3/target/release/deps/integration-364ecb9c23d58916` (signal: 4, SIGILL: illegal instruction)
To Reproduce
Steps to reproduce the behavior:
guix import crate -r ntest
<guix_repo>/guix/packages/crates-io.scm
guix build -L <guix_repo> rust-ntest
Expected behavior
The test would pass.
Screenshots
Here is a backtrace of running in gdb
(gdb) backtrace
#0 0x00005555555703d0 in core::ops::function::Fn::call ()
#1 0x0000555555569442 in std::sys_common::backtrace::__rust_begin_short_backtrace::h014a162e230a3962 ()
#2 0x00005555555712e8 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#3 0x00005555555c2a3a in std::sys::unix::thread::Thread::new::thread_start ()
#4 0x00007ffff7facf64 in start_thread ()
from /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libpthread.so.0
#5 0x00007ffff7ec59af in clone ()
from /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libc.so.6
Also as the tests run in parallel I tried to narrow down the issue with running them sequentially:
RUST_TEST_THREADS=1 cargo test --release
running 6 tests
test no_timeout ... ok
test test_function_10 ... ok
test test_function_200 ... ok
test timeout ... ok
test timeout_inf_loop ... error: test failed, to rerun pass '--test integration'
Caused by:
process didn't exit successfully: `/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3/target/release/deps/integration-364ecb9c23d58916` (signal: 4, SIGILL: illegal instruction)
``
**Desktop (please complete the following information):**
- Guix System
Describe the bug
When the timeout
attribute is put on a test and that test panics, the test reports that the test timed out, even though it didn't.
To Reproduce
Run this code:
#[cfg(test)]
mod tests {
use ntest::timeout;
#[test]
#[timeout(20000)]
fn panic() {
assert!(false);
}
}
Results in this output:
thread '<unnamed>' panicked at 'assertion failed: false', src/lib.rs:8:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tests::panic' panicked at 'timeout: the function call took 0 ms. Max time 20000 ms', src/lib.rs:6:5
The output should instead be this:
thread 'tests::panic' panicked at 'assertion failed: false', src/lib.rs:7:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Additional context
The fix is likely in ntest_proc_macro_helper
, which needs to be changed to handle test panics.
When using two attributes such as test case and timeout they will check for the next attribute to be valid.
#[test_case(200)]
#[ntest::timeout(100)]
Will panic:
error: custom attribute panicked
--> ntest/tests/integration.rs:5:1
|
5 | #[test_case(200)]
| ^^^^^^^^^^^^^^^^^
|
= help: message: Expected identifier!
Hi, are there any plans to release new version of ntest?
Last release was in 2022, since then there was a PR that afaik prevented rust-analyzer not working with ntest + tokio::test: #23
It was merged in July but is still not part of any release :(
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.