LLamaWeights.LoadFromFile about llamasharp

[martindevans]

I know that llama.cpp "might" be thread safe in these cases, but the comments over there are not clear to me.

Unfortunately that's the best answer available right now - LLamaSharp is as threadsafe as llama.cpp is, but it's unclear how threadsafe that actually is :(

would it be safe to load LLamaWeights.LoadFromFile only once and pass it to both implementations?

I haven't worked on the Kernel Memory part of LLamaSharp, @Oceania2018 may have a better answer than I do. I think the reason the weights are not shared is because a model can either be loaded in embedding mode (in which case it can't be used for inference) or in inference mode (in which case it can't be used for embedding).

[martindevans]

I've closed this issue since there's been no more activity and the question was answered, but feel free to re-open it if you want to know more :)