Ability to use GPT 3.5 Turbo about gpt-pilot

[sander110419]

I've made a PR to enable the selection of the model.
We aware that GPT3.5 provides way worse code, you're better off using the Azure endpoint and using the gpt4-32k model.

[CyKiller]

Not sure how to fully map out handling prioritized model switching for gpt-pilot, but introducing a rate-limiting mechanism in llm_connection.py to manage API requests to both GPT-4 and GPT-3.5 models can be done. We can try implementing counters for Tokens Per Minute (TPM) and Requests Per Minute (RPM) / then add logic to switch between the two models. These counters would enable dynamic model switching, allowing the system to choose the most appropriate model based on current usage and rate limits. Additionally, we can incorporate an exponential mechanism strategy to handle "429: Too Many Requests" errors. This involves a while-loop that retries the API request, doubling the wait time with each subsequent retry, up to a maximum number of retries. The contributor's input is better on how to target this, it's good for emulating instant gratification but in no form is it a true model switching formula that could be a nice fit for gpt-pilot.

[coolxeo]

Just created a PR that could solve it in a simple way
#39

I recommend to use gpt-3.5-turbo-16k-0613 model as has high capacity

[coolxeo]

If you are having problems with rate limited you could try to downgrade to GPT-3 that doesn't have so many restrictions

To make the change, just setup your environment variable in your .env file to whatever model you want to use
OPENAI_MODEL=gpt-3.5-turbo-16k-0613

Reference of GPT-3 models
https://platform.openai.com/docs/models/gpt-3-5

Thanks @zvone187 for merging my PR 🤗

[zvone187]

Happy to do so - thank you for the PR 🙏