cohenarthur / ftf

Parallel Functional Testing framework, written in Rust

ft

Parallel Functional Testing framework, written in Rust

Why ?

ft provides an alternative to constantly rewriting the same functional test framework for every project you start. It is written in pure Rust, and allows parallel testing, as well as an important set of input/output formats.

How ?

Look at the syntax to get started with creating your own tests. Then, simply run ft on the files you created.

Features

• Parallel execution
• Timeout
• Assert on stdout
• Assert on stderr
• Assert on exit-code
• Input from YAML
• Input from JSON
• Input from TOML
• YAML output
• JSON output
• TOML output

Installation

If you have rust and its build system, cargo, installed, simply run:

> cargo install --git https://github.com/cohenarthur/ft

To install rust, first visit the getting started page

You can also download a release of ft from the release page. The binaries are compiled for x86_64 systems.

ftf's Issues

Add parallelism

Add support for TAP protocol

TAP is parsed by almost every build system and ft would greatly benefit from supporting the TAP format

See: https://en.wikipedia.org/wiki/Test_Anything_Protocol

Add default values

Stuff like default arguments, default binary name, default environment values... Which one could skip with no_defaults or equivalent

Add json formatting

Add report on total amount of tests failed and passed

Output reports following junit format

This will allow some repository manager such as gitlab to parse and integrate the tests output in there review process.

Gitlab: https://docs.gitlab.com/ee/ci/unit_test_reports.html

Some crates that may be used:

• https://github.com/bachp/junit-report-rs
• https://docs.rs/junit-report/latest/junit_report/index.html

Add clippy to CI

Cratify `ft`

Make ft available as a library on top of a binary

Change formatting to make it easier for parsing

The format should be
[OK] <test_name>

instead of
<test_name> [OK]

Add piping functionality

Add stdin as option

ftf ignores timeout

Add formatting for inside C/C++/Rust comments

Rename repository to ftf in order to release on crates.io

Cmd is misleading

cmd have for description "Binary to test" in SYNTAX.md.
I found this misleading, maybe cmd should be named binary or the description should "A sheel command to execute"

Fix README regarding parallel execution

Since we can now support parallel execution using -j|--jobs, we should fix the README

Switch to `clap` instead of `structopt`

Add more directives

contains, ignore, regex...

Show file name on failure

Output easy to parse format for testsuite results

Something simple like <cases>, <successes>, <failures> or equivalent. Or take a "format" argument?

Add TOML formatting