[CI] Add more unit test to ensure the the outputs are reasonable about llamasharp

[martindevans]

We could also hardcode the expected responses in the unit tests. For example in this test it generates two completions of "Question. what is a cat?\nAnswer:" and assets that they are the same. We could assert the exact response too.

Of course this would only work with temp=0 and a specific model (even a specific quantisation), but it might save a few OpenAI calls!

[SignalRT]

In my opinion it would be easier the alternative that Martin proposes. We can not run all the test in CI y we should verify all the test locally.

[AsakusaRinne]

We could also hardcode the expected responses in the unit tests.

Yes, I also want to save the tokens where this approach works! I'll only consider using OpenAI API when necessary.

We can not run all the test in CI. we should verify all the test locally.

I tend to view things a bit differently. The workflows and unit test are responsible for reducing the risk when we merge the PRs. As long as the workflows pass, it should be equal to saying that terrible behaviors won't appear if we merge the PR.

However, due to the GPU backends, it's indeed hard for us to cover all the cases in the workflows. I can provide a machine with Nvidia GPU and Linux OS to run the workflows, but no idea for Windows yet. :)