rishi-11-2 / llama_cpp

A Fake OPEN AI Server Created using llama-cpp

We can run llama-cpp server on local machine at no cost

GGUF Models are used for inferencing with llama-cpp

in 'app.py' (mistral-7b-instruct-v0.1.Q2_K.gguf) is used for any stock data analysis. Mixtral model can also be used depending on the local computer's physical memory(RAM)

in 'multimodal.py' , user can enter images along with prompt and 'llava' model is used for inferencing.

Models used : mistral-7b-instruct-v0.1.Q2_K.gguf

llava-v1.5-7b-mmproj-Q4_0.gguf

llava-v1.5-7b-Q4_K.gguf