Comments (2)
Thanks for the suggestion.
In the chat interaction, the limited context window of the model is filled with previous fragments of conversations, creating the illusion that the chatbot has a memory. In reality, you need to send the entire conversation back and forth.
RAG is different from regular chat. All the context window memory of the model is used to fill it with document extracts to ground the answer, rather than relying on previous conversations.
I am afraid it is not feasible with the current context window lengths, unless you are using expensive, extremely long context window models. While it will remember some of the previous requests, it will have less factual data to provide the answer.
It might become possible in the future as the context length of the models increases.
from llm-search.
thank you for the response. Not much we can do about this at this time then. It's not even a big deal. It's a nice to have, but not really something that can't be mitigated with prompting.
I'll close this for now.
from llm-search.
Related Issues (20)
- Device map passed multiple times into the llm HOT 2
- Passing history in context to remember conversation HOT 3
- CSV data parsing HOT 3
- add support for using unstructured SaaS API for handling unstructured data pre-processing HOT 7
- GPU not being used during inference HOT 5
- Any examples or videos? HOT 9
- Plans for RRF? HOT 3
- Support several CUDA devices HOT 2
- Support openai for Embedding HOT 1
- Reduce logging HOT 2
- update requirements.txt legacy langchain, chroma, openai HOT 1
- Model params for GPU setup HOT 1
- CMake failed HOT 1
- support for cpp and hpp source code search? Generic text search? HOT 4
- Can't change embedding model HOT 2
- Can we return source document with LLM response? HOT 8
- Can we share web app publicly with Streamlit? HOT 1
- Re-ranking model too slow HOT 10
- License HOT 1
- Feature request: Adding Llamaparse support for unstructured PDFs HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llm-search.