Exception "EmbeddingMode must be true" after upgrading to v0.10.0 about llamasharp

[martindevans]

Can you post the full stack trace of the error? Thanks.

[LSXAxeller]

System.ArgumentException
  HResult=0x80070057
  Message=EmbeddingMode must be true (Parameter 'params')
  Source=LLamaSharp
  StackTrace:
   at LLama.LLamaEmbedder..ctor(LLamaWeights weights, IContextParams params, ILogger logger)
   at EGOIST.ViewModels.Pages.TextViewModel.<GenerationSwitchModels>b__52_0() in E:\Native\Projects\EGOIST\EGOIST\ViewModels\Pages\TextViewModel.cs:line 266
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)

I figured it out. It seems that when I use GenerationModelEmbedder = new(GenerationModel, GenerationModelParameters);, it requires EmbeddingMode = true in GenerationModelParameters. This wasn't necessary in previous versions. Sorry, I'm not very familiar with Visual Studio and detailed stack traces yet.

side question, how to use OpenCL backend ? I have a AMD RX580 GPU but even after installing OpenCL backend it still running on CPU

[martindevans]

One of the changes in 0.10.0 was that the LLamaEmbedder requires EmbeddingMode = true set. previously it would modify your config to set it, which wouldn't always be what you wanted (e.g. if the config is shared, that might get enabled without you realising it). Sounds like you've bumped into that.

how to use OpenCL backend

I'm not sure of the status, but I don't think OpenCL is available for LLamaSharp yet. Some initial work was done to add CLBLAST in 0.10.0, but (as far as I know) it's not shipped to nuget yet.

[martindevans]

Since you'vce figured out the problem I'll close this issue :)