9 tests rely on commands like `ln -s` making copies instead of symlinks on Windows about gitoxide

[Byron]

I think the reason for these 9 tests failing is that they don't have archives to begin with, so the scripts are always executed and thus affected by MSYS settings.

[Byron]

Thanks a lot for reporting!

My guess, however, is that calls to ln -s in gitoxide's test suite are not deliberately intended to copy a file rather than create a symlink.

It's true, the intend is for it to actually create a symlink, even on Windows. It was pretty clear that this didn't happen though, so I started to rely on using archives instead. These are also known to not create symlinks on Windows, I think, and it's strange that MSYS=winsymlinks:nativestrict cargo nextest run --all --no-fail-fast shows any difference. That variable shouldn't affect anything at all as the untar of archives is done in a Rust crate which at least in theory should be able to create symlinks. It definitely shouldn't be affected by the MSYS environment variable. But apparently, it is, which I consider a feature even as I always wanted symlinks to be created on Windows as well.

9 failing tests

I looked into one of the failing tests, gix-archive/tests/archive.rs and noticed that it clearly assumed there are no symlinks on WIndows. Fixing this would be easy, and probably the other tests are tuned like this as well.
In theory, this behaviour should be consistent even with archives being ignored.

In theory, testing with archives should be the same as testing without, and I'd hope that this MSYS flag is the key to (nearly) getting there as it should allow for feature-parity between the platforms.

Testing without archives on Windows

There is a way to partition tests and only run a subset of them in a deterministic fashion. To keep Windows runtimes in check and not let them pass 25 minutes, one could use this with a controlling script that runs random, non-repeating partitions until the time runs out. The partitioning should help to over time run all tests, instead of running only the first X tests each time.

https://nexte.st/docs/ci-features/partitioning/

Maybe this is something you could establish once all tests are fixed.

[EliahKagan]

I think the reason for these 9 tests failing is that they don't have archives to begin with

I'm relieved that the reason--even if not the needed fix--is straightforward.

It was pretty clear that this didn't happen though, so I started to rely on using archives instead.

It seems to me that pre-generated archives actually have an advantage beyond working around known platforms limitations and speeding up test runs: they provide some protection against unknown platform-specific problems that would cause fixtures to generate the wrong thing, especially if the tests are being run fairly regularly both with and without using the committed archives.

There is a way to partition tests and only run a subset of them in a deterministic fashion. To keep Windows runtimes in check and not let them pass 25 minutes, one could use this with a controlling script that runs random, non-repeating partitions until the time runs out.

I wonder if there is also a way to effectively compare the archives that get generated (though what kinds of variations are correct would differ across tests that use them).

[Byron]

I wonder if there is also a way to effectively compare the archives that get generated (though what kinds of variations are correct would differ across tests that use them).

I think I forgot to mention that the idea would be to use nextest as runner but without using test archives to force actually executing the scripts to assure they work. Maybe, instead of complicating the thing with partitioning, the job could also be run but setup so nobody has to wait for it. Ideally, failures get communicated by email but otherwise are not blocking.