This is an experiment at wrapping the excellent llama.cpp / ggml in go, and seeing whether that lets us make more dynamic use of multiple nodes than we get with the static configuration in llama.cpp's rpc server.
justinsb/kllama
README
Contributors
This is an experiment at wrapping the excellent llama.cpp / ggml in go, and seeing whether that lets us make more dynamic use of multiple nodes than we get with the static configuration in llama.cpp's rpc server.