Some issues with the self-hosting option about nllb-api

[winstxnhdw]

You'll have to clone the repo and build the Docker image without the timeouts. Otherwise, you can just set it to an arbitrarily large value. The model only supports 1024 tokens anyways, so at some point, it shouldn't be taking THAT long to perform its predictions.

[winstxnhdw]

I have removed the rate limiting module entirely in the latest commit.

[winstxnhdw]

That’s really strange. Let me take a look.

[winstxnhdw]

Hey, I am not able to replicate your issue. Is your request taking longer than 10 seconds to process?

[winstxnhdw]

You can try extending the cache timeout variable.

docker run --rm \
  -e APP_PORT=7860 \
  -e CACHE_TIMEOUT=100 \
  -p 7860:7860 \
  -v ./cache:/home/user/.cache \
  ghcr.io/winstxnhdw/nllb-api:main

[mlevin-dev]

We tested a clean installation on several different machines (MacOS, Ubuntu), the issue is the same across all of them. The only potential limitation that applies is the abysmal internet connection speed (< 5Mb/s). It seems to me like something is timing out.

Is your request taking longer than 10 seconds to process?

Probably yes, is there any way to disable all the timeouts?