Car AI tests about egregoria

[skairunner]

We don't need to make it necessarily fail the build, but could make an informational webhook kinda like some implementations of coverage

…

On Mon, Aug 10, 2020, 16:12 Pâris DOUADY ***@***.***> wrote: I agree on your points except for CD (3), this is merely an indicator of how well the car AI performs but shouldn't be required for CI to pass. Random might be used for car AI which would lead to flaky tests, also car AI is floating-point heavy which leads to further reproducibility problems. However, this applies to complicated tests (lots of cars doing hard interactions). I agree that some simple tests (<= 2 cars, simple goals) could be used to test for regressions. About scenarios (1), I'm happy I got Lua working and I think it could be a cool application, and would further extend the modding possibilities. (2) shouldn't be too hard either since the core engine egregoria is separated from the render module wgpu-renderer so having a headless client using the mods module for running scenarios shouldn't be too hard. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#19 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA22VOZVHN5C35JKFAU4B33R7755XANCNFSM4PWOHGIA> .

[skairunner]

As far as I can tell, these steps would be required:

(1) Ability to generate and load test scenarios (w/map, vehicle, and success conditions)
(2) Framework to headlessly start & run a scenario and check for completion
(3) Adding this as part of the CI/CD

[Uriopass]

I agree on your points except for CD (3), this is merely an indicator of how well the car AI performs but shouldn't be required for CI to pass.
Random might be used for car AI which would lead to flaky tests, also car AI is floating-point heavy which leads to further reproducibility problems. (note: I tried once to have reproducible runs for easier debugging but it was so hard I gave up.)

However, this applies to complicated tests (lots of cars doing hard interactions). I agree that some simple tests (<= 2 cars, simple goals) could be used to test for regressions.

About scenarios (1), I'm happy I got Lua working and I think it could be a cool application, and implementing this would further extend the modding possibilities which are non-existent at the moment.

(2) shouldn't be too hard either since the core engine egregoria is separated from the render module wgpu-renderer so having a headless client using the mods module for running scenarios shouldn't be too hard.

[Uriopass]

Ah ! Interesting, I'm not sure how this information could be presented but then I definitely agree.

[Uriopass]

Got some work done, a basic headless runner, a scenario player in wgpu-renderer and some lua bindings for the world to modify it. See the test script here

[Uriopass]

Got scenarios up and working, even in the wgpu-renderer accessible from the GUI. Not entirely complete but closing. (Not running on CI yet)