Access Violation in SafeLlamaContextHandle.Decode about llamasharp

[martindevans]

Is this happening all the time, or just in specific situations?

[PaulaScholz]

All the time in a console program.

[PaulaScholz]

If I go back to LlamaSharp 0.10, the issue disappears.

[martindevans]

Can you give us some details on what model you're using, what calls you're making etc?

[PaulaScholz]

Sure, I'm just using the Semantic Kernel chat program sample. As for model, llama-2-13b-chat.Q5_K_M.gguf is the file. It works with 0.10, same hardware.

[martindevans]

It's odd no one else has reported this! It doesn't look like you're doing anything particularly unusual.

If the error is caused by trying to decode an empty batch that is at least easy to avoid.

[martindevans]

I've just pushed up #691, which skips decoding if the batch is empty. Could you pull that branch and try it to see if it fixes your issue?