Comments (2)
Thank you for nice detailed explanation with pictures, think we can close this now.
from async-openai.
Did you get that response using the legacy "Completions" api or the "Chat Completions" api? Before they went to the chat completions api, the whole interaction was sort of a long string. If I go to the playground and choose the legacy completions api, and write a prompt, and I make sure that I just enter my prompt and don't include any newline characters of my own, you will see that it is going to prepend two newline characters before the text. If you reset it and type the same prompt and hit enter twice so there is a blank line above the cursor, and then submit the request, the response text will start right where the cursor is (again).
It is predicting next tokens, so if its training data included a lot of prompt/completion pairs where the completion starts with newlines, that is what you will see. If you end without newlines, it thinks the next token needs to be the double newline token. If your prompt ends with a single newline character, it is going to prepend a single newline. You can try this yourself: 0, 1, or 2 newlines after your prompt should yield the same spacing between the prompt and response text under the older models.
Another way to see the difference clearly is to go in the legacy playground and give it a very incomplete prompt. You will see it carries on right where you stopped. In the new api's they've stopped that madness.
from async-openai.
Related Issues (20)
- how to change base_url? HOT 2
- text to speech - flac format does not seem to work HOT 9
- Auto Generation from OpenAPI spec HOT 3
- Add support for llama.cpp completions HOT 2
- impls.rs: AssistantTools HOT 3
- query for client files list seems to be missing HOT 1
- Changing base url - Usage with open source LLM - Invalid status code: 404 Not Found HOT 3
- Stabilizing wasm target HOT 2
- Support text-to-speech (TTS) streaming functionality HOT 12
- Advanced error handling for OpenAI-like LLM APIs
- Test compilation failed HOT 2
- Support variable length embeddings HOT 3
- OpenAI transcriptions api language field is invalid ! HOT 1
- Embedding deserialization error when using encoding_format `EncodingFormat::Base64`
- Ollama AI Chat Completion Example / OpenAI Compatibility HOT 3
- Sync OpenAPI spec changes after 4th Jan 24 HOT 1
- Initialize `Client` with custom HTTP client, config, and backoff HOT 3
- Usage of std::thread::sleep inside async function in assistants example HOT 1
- Add retry on 504 Gateway Time-out and "connection closed before message completed" HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from async-openai.