ollama modelinstance can permafail with "connection refused" about helix

[philwinder]

I've been seeing this again with the following setup: local dev, single luke@prime runner.

for i in `seq 1 10`; do curl --request POST \        
  --url http://localhost:8080/v1/chat/completions \
  --header 'Authorization: Bearer hl-95i-ORkQBZmUbVJHWIo04yNdylT8w3gBCLX_L3zPF7k=' \
  --header 'Content-Type: application/json' \
  --data '{
  "model": "llama3:instruct",
  "messages": [
    {
      "role": "user",
      "content": "Write a 1000 word essay on Yorkshire."
    }
  ]
}' & ; done

results in:

runner-1  | 2024-08-14T18:55:29Z ERR api/pkg/runner/llm_ollama_model_instance.go:156 > error processing request error="failed to get response from inference API: Post \"http://localhost:43851/v1/chat/completions\": dial tcp 127.0.0.1:43851: connect: connection refused" session_id=
runner-1  | 2024-08-14T18:55:29Z ERR api/pkg/runner/llm_ollama_model_instance.go:159 > detected connection refused, exiting and hoping we get restarted - see https://github.com/helixml/helix/issues/242

[philwinder]

Fixed in https://github.com/helixml/helix/releases/tag/0.10.2