Use remote docker ollama server about twinny

[rjmacarthy]

Hey @jakern thanks for the report.

Actually, you can skip the server checks in the beginning and use the option to disable them which is persistent across sessions. Then, in the settings update the API URL and path to point towards your Docker instance and everything should just work.

Some docs would be good probably to explain this a little better.

Does that help?

Many thanks,

[jakern]

Thanks a bunch! I didn't see that option. I am able to get past the check and receive suggestions from the end point! closing.

[wchorski]

I too want to use Ollama running in a docker container (on my local machine). I figured the setup would be exactly like the native Ollama app, but when asking the Twinny chat, I get an endless progress spinner or Sorry, I don’t understand. Please try again.

I've got the container up and running with these instructions,

• Port 11434 exposed to host
• Installed the codellama:7b-instruct model
• able to execute docker exec -it ollama ollama run codellama:7b-instruct and interact with Ollama via terminal

Chatting with Ollama in terminal

Here are some logs of when I interact with Ollama docker container directly with the exec command

2024-03-04 15:43:01 [GIN] 2024/03/04 - 21:43:01 | 200 |    1.855333ms |       127.0.0.1 | POST     "/api/chat"
2024-03-04 15:43:20 [GIN] 2024/03/04 - 21:43:20 | 200 |   4.71799671s |       127.0.0.1 | POST     "/api/chat"
2024-03-04 15:43:33 [GIN] 2024/03/04 - 21:43:33 | 200 |  5.928768169s |       127.0.0.1 | POST     "/api/chat"
2024-03-04 15:43:50 [GIN] 2024/03/04 - 21:43:50 | 200 |  5.541841752s |       127.0.0.1 | POST     "/api/chat"

Chatting via Twinny

When chatting with Twinny this is what I saw in Ollama Docker container logs

2024-03-04 15:38:26 [GIN] 2024/03/04 - 21:38:26 | 200 | 53.482893024s |    192.168.65.1 | POST     "/v1/chat/completions"
2024-03-04 15:38:26 [GIN] 2024/03/04 - 21:38:26 | 200 | 27.580572721s |    192.168.65.1 | POST     "/v1/chat/completions"

but that didn't help link Twinny with Ollama. I'm assuming I'm not exposing the container correctly to the system...

Above Advice

I was a bit confused by the advice

skip the server checks in the beginning and use the option to disable them

Is this setting in the VSCode User settings? Could I get a screenshot?

update the API URL and path to point towards your Docker

from this I assumed I had to change

• hostname -> 127.0.0.1
• /v1/chat/completions -> /api/chat

[DenisBY]

I have the same issue. I'm able to chat via console, but Twinny continues to answer Sorry, I don’t understand. Please try again. I already changed /v1/chat/completions -> /api/chat as console shows this path.

I use the latest available ollama/ollama Docker image from Docker Hub