Parallel execution of tests causes flakiness when multiple inline_c tests use the same names for environment variables

#22 · open · 4 comments

View on GitHub ↗

Vibha-Malhotra

When multiple inline_c tests in a rust file use the same names of environment variables, flakiness is observed during parallel execution of tests. For example, if all tests in a rust file need an input directory and an output directory, and environment variables `INLINE_C_RS_INPUT_PATH` and `INLINE_C_RS_OUTPUT_PATH` are used to pass these values to `assert_c` macro, cases fail randomly upon running `cargo test`. To get around this, one needs to use unique environment variable names in each test. This should be documented.

Comments

Hywan

Hi, You're right, that's indeed a dramatic limitation… I am not sure there is a way to workaround that.

camio

Starting a subprocess with environment variables separate from the parent process should be fine. I think the issue is that environment variables are used to communicate information to the inline-c subsystem. Perhaps `assert_c!` could take in additional "parameters"?

Hywan

`inline-c` doesn't control the processes being run. They are managed by `cargo test` (or [`nextest`](https://nexte.st/)). Or did I miss understood your comment?

camio

I'm talking about [this code](https://github.com/Hywan/inline-c-rs/blob/e24acb7036bbe7254a0151fa1cdf1a86ace16bab/src/run.rs#L26) which extracts environment variables from the containing process and then [injects them](https://github.com/Hywan/inline-c-rs/blob/e24acb7036bbe7254a0151fa1cdf1a86ace16bab/src/run.rs#L105) into the subprocess.