dzejkop / catchr

A testing framework for Rust inspired by Catch!

Catchr

Experimental: Might eat your laundry!

A testing framework for Rust inspired by Catch for C++.

Quickstart

Add catchr = "0.3.0" to your Cargo.toml

Write tests:

#[cfg(test)]
mod tests {
    catchr::describe! {
        section "my tests" {
            given "x is equal to 1" {
                let mut x = 1;

                when "1 is added to x" {
                    x += 1;

                    then "x should equal 2" {
                        assert_eq!(2, x);
                    }
                }

                when "2 is added to x" {
                    x += 2;

                    then "x should equal 3" {
                        assert_eq!(3, x);
                    }
                }

                // for all code paths
                assert!(x >= 2);
            }
        }
    }
}

cargo test

running 2 tests
test tests::section_my_tests::given_x_is_equal_to_1::when_2_is_added_to_x::then_x_should_equal_3 ... ok
test tests::section_my_tests::given_x_is_equal_to_1::when_1_is_added_to_x::then_x_should_equal_2 ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Sections

Each test section consists of a keyword, a description and a body.

keyword "description" {
    // body
}

For the moment the following keywords are supported: section, case, when, then, given.

Sections without any nested section will become test cases. Sections function like scopes - that is statements from the outer section are available in the inner section:

when "something" {
    let x = 1;

    then "anything" {
        let y = 1;
        assert_eq!(x, y);
    }

    then "whatever" {
        assert!(true);
    }
}

The let x = 1; can be used in the then "anything" section. But let y = 1; from the then "anything" section, cannot be used in the then "whatever" section.

Furthermore the scoping rules are preserved, so that inner test cases can borrow mutably without violating the borrow checker rules.

Consider the following example:

case "a" {
    let mut tmp = tempfile().unwrap();

    case "should write some data" {
        let mut writer = BufWriter::new(&mut tmp);
        writer.write_all(&[1, 2, 3]).unwrap();
    }

    tmp.seek(SeekFrom::Start(0)).unwrap();
    let bytes_in_tmp_file = tmp.seek(SeekFrom::End(0)).unwrap();

    assert_eq!(bytes_in_tmp_file, 3);
}

if the test case was expanded without scoping, we'd get

let mut tmp = tempfile().unwrap();

let mut writer = BufWriter::new(&mut tmp);
writer.write_all(&[1, 2, 3]).unwrap();

tmp.seek(SeekFrom::Start(0)).unwrap();
let bytes_in_tmp_file = tmp.seek(SeekFrom::End(0)).unwrap();

assert_eq!(bytes_in_tmp_file, 3);

which fails to compile!

so catchr will expand this test case into

let mut tmp = tempfile().unwrap();

{
    let mut writer = BufWriter::new(&mut tmp);
    writer.write_all(&[1, 2, 3]).unwrap();
}

tmp.seek(SeekFrom::Start(0)).unwrap();
let bytes_in_tmp_file = tmp.seek(SeekFrom::End(0)).unwrap();

assert_eq!(bytes_in_tmp_file, 3);

Async support

You can also use the describe_tokio macro to generate async tests that work with the tokio runtime. First you need to make sure that you have tokio in your dependencies and it has the required features - my recommendation is to use features = ["full]".

Next just write

catchr::describe_tokio! {
    when "Something" {
        then "Something" {
            // it's possible to `.await` here
            assert!(true);
        }
    }
}

How does it work?

The code from the Quickstart section will expand into something like this:

#[cfg(test)]
mod tests {
    mod section_my_tests {
        use super::*;

        mod given_x_is_equal_to_1 {
            use super::*;

            mod when_1_is_added_to_x {
                use super::*;

                #[test]
                fn then_x_should_equal_2() {
                    {
                        let mut x = 1;
                        {
                            x += 1;
                            {
                                assert_eq!(2, x);
                            }
                        }
                        assert!(x >= 2);
                    }
                }
            }

            mod when_2_is_added_to_x {
                use super::*;

                #[test]
                fn then_x_should_equal_3() {
                    {
                        let mut x = 1;
                        {
                            x += 2;
                            {
                                assert_eq!(3, x);
                            }
                        }
                        assert!(x >= 2);
                    }
                }
            }
        }
    }
}

catchr's Issues

Testing async-infected code

Currently the catchr uses #[test] to mark tests.

Which is good for usual tests, but async is so contagious that even tests are affected.

For async code (with tokio in my case) common way is to use #[tokio::test] instead of #[test] (and other runtimes have similar, eg actix_rt::test).

Can we have support for that?

Feature: Pretty messages

Catch does this

Scenario: vectors can be sized and resized
     Given: A vector with some items
      When: more capacity is reserved
      Then: the capacity changes but not the size

catchr will show you the path to the test, but paths cannot contain any special characters. It'd be nice to be able to do something like:

section "Very descriptive comment" {
	case "-- special chars ???" {
		assert!(false, catchr_msg!());
	}
}

and for it to output:

running 1 test
test very_descriptive_comment::special_chars ... FAILED

failures:

---- very_descriptive_comment::special_chars stdout ----
thread 'very_descriptive_comment::special_chars' panicked at '
Section: Very descriptive comment
	Case: -- special chars ???
', src/lib.rs:4:9

or something simmilar

Feature: Case enumeration

Syntax:

section "whatever" {
  each msg in ["Hello", "World"] {
    then "should be true" { 
      assert!(false, msg);
    }
  }
}

Should expand into

mod section_whatever {
    use super::*;
    
    mod msg_is_hello {
        use super::*;
        
        #[test]
        fn then_should_be_true() {
			{
            	let msg = "Hello";
            	assert!(false, msg);
			}
        }
    }
    
    mod msg_is_world {
        use super::*;
        
        #[test]
        fn then_should_be_true() {
            { 
                let msg = "World";
                assert!(false, msg);
            }
        }
    }
}

Notes:

1. in each item in items item should be a pattern and items a list of pattern.
2. item should inserted as the first item in the respective scope

Skip empty scopes when quoting

Right now this code

describe! {
	case "one" {
		case "two" {
			case "three" {
				assert!(true);
			}
		}
	}
}

expands to

mod one {
	use super::*;

	mod two {
		use super::*;

		#[test]
		fn three() {
			{ { { { assert!(true); } } } }
		}
	}
}

Ideally we'd want

mod one {
	use super::*;

	mod two {
		use super::*;

		#[test]
		fn three() {
			assert!(true);
		}
	}
}