Comments (5)
I've made a PR to enable the selection of the model.
We aware that GPT3.5 provides way worse code, you're better off using the Azure endpoint and using the gpt4-32k model.
from gpt-pilot.
Not sure how to fully map out handling prioritized model switching for gpt-pilot, but introducing a rate-limiting mechanism in llm_connection.py to manage API requests to both GPT-4 and GPT-3.5 models can be done. We can try implementing counters for Tokens Per Minute (TPM) and Requests Per Minute (RPM) / then add logic to switch between the two models. These counters would enable dynamic model switching, allowing the system to choose the most appropriate model based on current usage and rate limits. Additionally, we can incorporate an exponential mechanism strategy to handle "429: Too Many Requests" errors. This involves a while-loop that retries the API request, doubling the wait time with each subsequent retry, up to a maximum number of retries. The contributor's input is better on how to target this, it's good for emulating instant gratification but in no form is it a true model switching formula that could be a nice fit for gpt-pilot.
from gpt-pilot.
Just created a PR that could solve it in a simple way
#39
I recommend to use gpt-3.5-turbo-16k-0613
model as has high capacity
from gpt-pilot.
If you are having problems with rate limited you could try to downgrade to GPT-3 that doesn't have so many restrictions
To make the change, just setup your environment variable in your .env
file to whatever model you want to use
OPENAI_MODEL=gpt-3.5-turbo-16k-0613
Reference of GPT-3 models
https://platform.openai.com/docs/models/gpt-3-5
Thanks @zvone187 for merging my PR 🤗
from gpt-pilot.
Happy to do so - thank you for the PR 🙏
from gpt-pilot.
Related Issues (20)
- [Bug]: When selecting "Load App", I get a spin wheel. HOT 4
- [Bug]: Crash when running python main.py: MissingGreenlet: greenlet_spawn has not been called HOT 3
- [Bug]: AttributeError: 'SpecWriter' object has no attribute 'telemetry' HOT 2
- [Enhancement]: MS-SQL Support
- [Howto]: i accidently closed the app tab
- [Bug]:After loading the app, the front-end reported an error: Error connecting to the LLM: API error: Error code: 400 - {'error': {'message': 'Unrecognized request argument supplied: stream_options (request id: 20240617162225973326812zW5O0B98)', 'type': 'invalid_request_error', 'param': '', 'code': None}}
- Error connecting to the LLM: Error parsing response: Expected a single code block, got 0[Pythagora] Would you like to retry the failed request?
- [Bug]: KeyError: 'user_feedback_qa' HOT 9
- [Bug]: New claude sonnet adds comments/descriptions into code files, making them unusable HOT 3
- [Bug]: New Claude Sonnet CodeMonkey describes every file in env
- [Bug]: API error: Error code: 400 - {'error': {'message': "Unknown parameter: 'stream_options' HOT 4
- [Bug]: ValueError: Empty message content HOT 3
- [Bug]: HOT 2
- [Bug]: Added my own openai key but still won't let me proceed.
- [Howto]: I have deployed GPT Pilot on a new computer. How to synchronize the app to the new computer HOT 1
- [Bug]: node_sqlite3.node is not a valid Win32 application HOT 1
- [Enhancement]: Gemini Support HOT 1
- [Howto]: set up the llm studio Meta-Llama-3-8B-Instruct-GGUF model in gpt pilot 0.2 in local system
- [Howto]: Complete The New Step Six? HOT 2
- [Bug]: after selecting "Load App", my existing apps are not shown
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gpt-pilot.