ray-project / langchain-ray Goto Github PK
View Code? Open in Web Editor NEWExamples on how to use LangChain and Ray
License: Apache License 2.0
Examples on how to use LangChain and Ray
License: Apache License 2.0
I followed the example "https://github.com/ray-project/langchain-ray/tree/main/open_source_LLM_retrieval_qa", with the "Building the vector store index" and "Serving" commands as below without any problems:
python build_vector_store.py
serve run serve:deployment
But when trying "Querying"
python query.py 'What is the difference between SERVE and PACK placement groups?'
I get the following error:
Traceback (most recent call last):
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection
raise err
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 1286, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 1332, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 1281, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 1041, in _send_output
self.send(msg)
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/http/client.py", line 979, in send
self.connect()
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connection.py", line 205, in connect
conn = self._new_conn()
^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f06d13f2950>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /?query=What%20is%20the%20difference%20between%20SERVE%20and%20PACK%20placement%20groups? (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f06d13f2950>: Failed to establish a new connection: [Errno 111] Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/oia2ml/Llama-2/langchain-ray/open_source_LLM_retrieval_qa/query.py", line 6, in <module>
response = requests.post(f"http://localhost:8000/?query={query}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/oia2ml/anaconda3/envs/ray/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /?query=What%20is%20the%20difference%20between%20SERVE%20and%20PACK%20placement%20groups? (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f06d13f2950>: Failed to establish a new connection: [Errno 111] Connection refused'))
Tried to troubleshoot with some different changes (IP and firewall) but could not solve out the problem.
Could you please advise on this matter?
Thank you.
On my Ubuntu 20.04 machine, I had to add the following dependencies to get the demo to work:
$ python -m pip install beautifulsoup4
$ python -m pip install accelerate
I also found that the versions of langchain and wandb specified in requirements.txt are not compatible, as indicated by this error message:
ValueError: The Weights & Biases Langchain integration does not support versions 0.0.187 and lower. To ensure proper functionality, please use version 0.0.188 or higher.
The simplest solution was to eliminate the small number of references to wandb:
serve.py: remove the import of wand, and the one call to initialize it
local_pipeline.py: remove import of wand
With these changes, the demo runs.
(ReadBinary->FlatMap->FlatMap pid=17023, ip=172.31.69.122) OSError: When reading information for key '2205.13708v1.HiJoNLP_at_SemEval_2022_Task_2_Detecting_Idiomaticity_of_Multiword_Expressions_using_Multilingual_Pretrained_Language_Models.pdf' in bucket 'ray-llm-batch-inference': AWS Error [code 15]: No response body. [repeated 157x across cluster]
(ReadBinary->FlatMap->FlatMap pid=27798, ip=172.31.75.31) invalid pdf header: b'<?xml'
(ReadBinary->FlatMap->FlatMap pid=27798, ip=172.31.75.31) EOF marker not found
Getting below error while loading the localhgembeddings:
error:
huggingface_hub.utils._errors.HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/sentence-transformers/multi-qa-mpnet-base-dot-v1
Looks like the embeddings are not more available. Do we have any alternatives ?
Hi, thanks for the great work in the open-source space. I am facing the below error:
index = faiss.IndexFlatL2(len(embeddings[0])) IndexError: index 0 is out of bounds for axis 0 with size 0
The faiss index is empty. There are no embeddings?
Can you help me debug this? I really appreciate any help you can provide.
Maybe I was wrong, but this line (line 72):
result = self.chain({"input_documents": search_results, "question": query})
in example open_source_LLM_retrieval_qa/serve.py
input_documents
should be context
, for context
is the placeholder to use.
I have problems to run this demo:
Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.