pinecone-io / canopy Goto Github PK
View Code? Open in Web Editor NEWRetrieval Augmented Generation (RAG) framework and context engine powered by Pinecone
Home Page: https://www.pinecone.io/
License: Apache License 2.0
Retrieval Augmented Generation (RAG) framework and context engine powered by Pinecone
Home Page: https://www.pinecone.io/
License: Apache License 2.0
from canopy.knowledge_base import KnowledgeBase
kb = KnowledgeBase(index_name="{{TestIndex}}",namespace="{{TestNamespace}}")
kb.connect()
Similar to chain using namespace in vector stores, namespaces should be supported in Chat Engines
from canopy.knowledge_base import KnowledgeBase
from canopy.chat_engine import ChatEngine
kb = KnowledgeBase(index_name="demo")
kb.connect()
ContextEngine(kb,global_metadata_filter={'key': {'$eq': 'value'}})
chat_engine=ChatEngine(context_engine)
Balancing costs of using a new index for each set instead of multiple namespaces in an index. Metadata filtering works but is not backwards compatible with existing namespaces code.
No
No response
I can't use the CLI to do a canopy chat even though I am able to hit the server and conduct chat completion.
CLI to hit server and return chat completion like it did when I manually hit the server
The range of the Cybertruck is estimated to be between 250-340 miles (400-545 km) according to the EPA. However, with the optional range extender battery, the range can be extended to 440-470 miles (710-755 km).\n(Source: /Users/andrewnada/Downloads/tesla_stats.txt)
{"id": "0", "text": "Manufacturer Tesla, Inc.", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "1", "text": "Production 2023\u2013present", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "2", "text": "Assembly United States:\u00a0Austin, Texas\u00a0(Gigafactory Texas)", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "3", "text": "Designer Franz von Holzhausen, Sahm\u00a0Jafari,Ian\u00a0Kettle, WooTak\u00a0Kim, Julien\u00a0Bilodeau", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "4", "text": "Body and chassis Class Full-size\u00a0pickup truck Body\u00a0style 4-door\u00a0crew cab Layout Rear motor, rear-wheel drive Dual- or tri-motor, all-wheel drive Powertrain Transmission 1-speed fixed gear ratio", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "5", "text": "Battery 123 kWh: 816 V, 150 Ah +50 kWh (with range extender battery)", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "6", "text": "Electric\u00a0range 250\u2013340\u00a0mi (400\u2013545\u00a0km) EPA estim. 440\u2013470\u00a0mi (710\u2013755\u00a0km) (with optional range extender battery)", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "7", "text": "Plug-in\u00a0charging \u2264250 kW DC at 400V \u2264350 kW DC at 800V 11.5 kW AC (240V, 48A) V2H\u00a0and\u00a0V2V: 11.5 kW AC (240V, 48A) V2L: 9.6 kW AC (240V, 40A) continuous", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "8", "text": "Dimensions Length 223.7\u00a0in (5,680\u00a0mm) Width 80\u201395\u00a0in (2,000\u20132,400\u00a0mm) Height 70.5\u00a0in (1,790\u00a0mm)", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
{"id": "9", "text": "Curb\u00a0weight AWD: 6,603\u00a0lb (2,995\u00a0kg) Cyberbeast: 6,843\u00a0lb (3,104\u00a0kg)", "source": "/Users/andrewnada/Downloads/tesla_stats.txt", "metadata": {}}
server is just showing health checks
- **OS**: Mac OS sonoma 14.1.1 (23B81)
- **Language version**: Python 3.11.5
- **Canopy version**: pulled latest f372033
No response
Enabled Canopy to also allow uploading for images, or indexing of images, for the RAG operation. This also means that inference API from Canopy should be able to respond with a proper REST format (JSON + base64 enc for bin?) to return said produced images.
Considering creating something like this myself.
Multimodal RAG users who wish to use multimodality within the ChatCompletions API from OpenAI , for example.
yes, but I've just discovered canopy 10 minutes ago :)
No response
Can we use canopy with some other Vector Database apart from Pinecone ? I f not So can u please add this feature as well.
NA
NA
NA
NA
When upserting data with metadata fields or document ids that are parseable to datetime, the upert succeed. However, when trying to query or chat with this data, a data validation error raised for these values to be of time datetime, although expected to be str.
datatime values string in metadata and ids expected to be treated as strings. This is a known issue with Pinecone python client, but this repo explicitly raise an error for this while the client isn't, so this issue should be resolved here if it's not going to be resolved soon in the client side
Can try running https://gist.github.com/jamescalam/35ad30a0e0a7d20da7b77ce628b264e2
2023-10-16 13:49:35,611 - MainProcess - resin_cli.app [ERROR ]: Chat with question_id 50fb62c3-f202-4062-9931-35b63bd24f13 failed
Traceback (most recent call last):
File "/Users/amnoncatav/Pinecone/context-engine/resin_cli/app.py", line 53, in chat
answer = await run_in_threadpool(chat_engine.chat,
File "/Users/amnoncatav/Pinecone/context-engine/.venv/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
File "/Users/amnoncatav/Pinecone/context-engine/.venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/Users/amnoncatav/Pinecone/context-engine/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2106, in run_sync_in_worker_thread
return await future
File "/Users/amnoncatav/Pinecone/context-engine/.venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 833, in run
result = context.run(func, *args)
File "/Users/amnoncatav/Pinecone/context-engine/resin/chat_engine/chat_engine.py", line 130, in chat
context = self.get_context(messages)
File "/Users/amnoncatav/Pinecone/context-engine/resin/chat_engine/chat_engine.py", line 160, in get_context
context = self.context_engine.query(queries, self.max_context_tokens)
File "/Users/amnoncatav/Pinecone/context-engine/resin/context_engine/context_engine.py", line 58, in query
query_results = self.knowledge_base.query(
File "/Users/amnoncatav/Pinecone/context-engine/resin/knoweldge_base/knowledge_base.py", line 240, in query
results = [self._query_index(q, global_metadata_filter) for q in queries]
File "/Users/amnoncatav/Pinecone/context-engine/resin/knoweldge_base/knowledge_base.py", line 240, in <listcomp>
results = [self._query_index(q, global_metadata_filter) for q in queries]
File "/Users/amnoncatav/Pinecone/context-engine/resin/knoweldge_base/knowledge_base.py", line 285, in _query_index
KBDocChunkWithScore(id=match['id'],
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 8 validation errors for KBDocChunkWithScore
metadata -> published
str type expected (type=type_error.str)
metadata -> published
value is not a valid integer (type=type_error.integer)
metadata -> published
value is not a valid float (type=type_error.float)
metadata -> published
value is not a valid list (type=type_error.list)
metadata -> updated
str type expected (type=type_error.str)
metadata -> updated
value is not a valid integer (type=type_error.integer)
metadata -> updated
value is not a valid float (type=type_error.float)
metadata -> updated
value is not a valid list (type=type_error.list)
- mac M1
- python 3.10
No response
Support the new version of OpenAI python SDK (V1.2.0) released few days ago. Currently this library failing to work with this version due to breaking changes.
No response
Support this version could enable using new OpenAI capabilities like assistants, threads and many more. Also should enable to add Canopy to projects that already using the new OpenAI version
Yes
the current behavior is ambiguous error msg:
Failed to infer vectors' dimension from encoder due to error: Error code: 429 - {'error': {'message': 'You exceeded your current quota, please ...
The error should indicate that this is the embedding model issue
- **OS**: any
- **Language version**: any
- **Canopy version**: 0.2.0
I currently cannot set the source
value for each record metadata, meaning I can't set insightful sources returned to me in the chat — would be great it I could do that!
In my use-case I want to chat with arxiv docs. It's important for me to see where the info is coming from but all I see is (Source: Context)
. I assumed this is being set via the "source"
metadata field, as I earlier tried to set it but received an error about this being a protected field. Ideally I'd like to put a paper ID or URL in that space.
In the "metadata"
field of records, have a key named "source"
and then run resin upsert ./data.parquet
No response
- **OS**: MacOS
- **Language version**: 3.9.12
- **Pinecone client version**: 2.2.1
No response
Hi guys, I haven't found any community forum so asking here a few questions:
I'm using this documentation for my RAG project .
while index.describe_index_stats()['total_vector_count'] < len(docs):
time.sleep(0.05)
The same goes for delete()
please advise a proper way to validate that the upserted documents are indexed and available.
Thanks!
Start canopy, switch to new terminal to run "canopy chat" and it says that canopy isnt running
Canopy chat starts
Win environment
Occurs inside and outside virtual environment
Starting Canopy server on 0.0.0.0:8000
INFO: Started server process [6148]
INFO: Waiting for application startup.
2023-11-09 20:47:46,231 - MainProcess - canopy_server.app [INFO ]: Did not find config file. Initializing engines with default configuration
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
At same time:
Error: Canopy server is not running on http://0.0.0.0:8000.
please run `canopy start`
- Win:
- python3
I am not a coder, just hobbyist, so likely I am just missing something
All of my Azure OpenAI env vars are set, and I was able to both canopy upsert
and canopy start
using the Azure OpenAI setup.
When running canopy chat
, everything works as expected.
However when running canopy chat --no-rag
, after submitting the first prompt, an error is returned: "Error: No OpenAI API key provided. When using the --no-rag
flag You will need to have a valid OpenAI API key. Please set the OPENAI_API_KEY environment variable."
The expected behavior is that the --no-rag flag would work for the Azure OpenAI setup.
canopy start
)canopy chat --no-rag
The current process for one-time upload of a large dataset currently relies on naive batching, without any performance optimizations. In addition, there is almost no robustness to failures, and almost any intermediate failure.
The current KnowledgeBase.upsert()
method was designed to a small set of Documents - more suitable for "follow up" data updates.
We need to either refactor the existing method or add an additional bulk_upsert()
method that would be both more performant and more robust for failures.
Further analysis is required. Some of the processing steps are CPU-bound, while others are natively async.
In addition, some of the steps could benefit from different batch sizes than other (e.g. embedding model calls) - so we might consider doing the processing in some sort of a producer-consumer pipeline.
This would affect all users, especially new users or ones starting a new project.
No response
No response
The logic in the KnowledgeBase Module is hardcoded to add canopy--
before user defined index name. This causing the module to raise the following error.
f"The index {self.index_name} does not exist or was deleted. "
"Please create it by calling knowledge_base.create_canopy_index() or "
"running the `canopy new` command"
)
I have changed the INDEX_NAME_PREFIX = ""
to an empty string. Now, it able to connect to an existing index and verify the connection.
from canopy.knowledge_base import KnowledgeBase
kb = KnowledgeBase(index_name='<<index_name>>')
kb.connect()
No response
- **OS**: Windows 11
- **Language version**: Python 3.10.10
- **Pinecone client version**: 2.1.1
No response
Is local vector database support on the roadmap? Looking for a privacy-focused rag solution, and needing to send all my notes to a cloud service is a deal-breaker for me.
When I launch the command canopy new
to create a new index I receive the following error:
Error: Failed to create a new index. Reason:
Failed to infer vectors' dimension from encoder due to error: 'ascii' codec can't encode character '\u201c' in position 7: ordinal not in range(128). Please fix the error or provide the dimension manually
A new index is created
canopy new
Canopy is going to create a new index: canopy--merchants
Do you want to continue? [y/N]: y
Error: Failed to create a new index. Reason:
Failed to infer vectors' dimension from encoder due to error: 'ascii' codec can't encode character '\u201c' in position 7: ordinal not in range(128). Please fix the error or provide the dimension manually
### Environment
```markdown
- **OS**: Mac Os Sonoma 14.1
- **Language version**: Python 3.11.4
- **Pinecone client version**: Canopy, version 0.2.0
No response
"pip install -e ." fails with errors when using python 3.12 but it works with older versions.
Expected behavior: It should also work with python 3.12.
Steps to reproduce:
Relevant log:
...
Collecting numpy>=1.25.0 (from pandas-stubs<3.0.0.0,>=2.0.3.230814->canopy-sdk==0.2.0)
Downloading numpy-1.25.2.tar.gz (10.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.8/10.8 MB 10.4 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [33 lines of output]
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
backend = _build_backend()
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
obj = import_module(mod_path)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
File "<frozen importlib._bootstrap>", line 1304, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 929, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 994, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/tmp/pip-build-env-tb9_n15e/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
import setuptools.version
File "/tmp/pip-build-env-tb9_n15e/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
import pkg_resources
File "/tmp/pip-build-env-tb9_n15e/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2172, in <module>
register_finder(pkgutil.ImpImporter, find_on_path)
^^^^^^^^^^^^^^^^^^^
AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Environment:
(Note: I tried to use the bug template but the 'submit new issue' button stayed gray even though I had everything filled in)
If I try to upsert any data containing GPT special tokens like <|endoftext|>
I will get this error:
The reason being that tiktoken tokenizer does not allow encoding of the special tokens by default, however we can encode them as normal text (ie not as special tokens and avoiding errors) by setting disallowed_special=()
in the encode
method. See code here.
You can add this by modifying the /resin/knowledge_base/tokenizer/openai.py
file to contain:
import tiktoken
from typing import List
from .tokenizer import BaseTokenizer
from ...models.data_models import Messages, MessageBase, Role
class OpenAITokenizer(BaseTokenizer):
MESSAGE_TOKENS_OVERHEAD = 3
FIXED_PREFIX_TOKENS = 3
def __init__(self, model_name: str):
self._encoder = tiktoken.encoding_for_model(model_name)
def tokenize(self, text: str) -> List[str]:
return [self._encoder.decode([encoded_token])
for encoded_token in self._encoder.encode(text, disallowed_special=())]
def detokenize(self, tokens: List[str]) -> str:
if not isinstance(tokens, List):
raise TypeError(f"detokenize expect List[str], got f{type(tokens)}")
return "".join(tokens)
def token_count(self, text: str) -> int:
return len(self._encoder.encode(text, disallowed_special=()))
def messages_token_count(self, messages: Messages) -> int:
# Adapted from: https://github.com/openai/openai-cookbook/.../How_to_format_inputs_to_ChatGPT_models.ipynb # noqa
num_tokens = 0
for message in messages:
num_tokens += self.MESSAGE_TOKENS_OVERHEAD
for key, value in message.dict().items():
num_tokens += self.token_count(value)
num_tokens += self.FIXED_PREFIX_TOKENS
return num_tokens
@staticmethod
def test_messages_token_count(tokenizer):
messages = [MessageBase(role=Role.USER, content="hello"),
MessageBase(role=Role.ASSISTANT, content="hi")]
assert tokenizer.messages_token_count(messages) == 11
@staticmethod
def test_messages_token_count_empty_messages(tokenizer):
assert tokenizer.messages_token_count([]) == 0
Allow the special tokens to be encoded.
Run this notebook
No response
- **OS**: MacOS
- **Language version**: Python 3.9.12
- **Pinecone client version**: 2.2.1
No response
I get an error when I starting my canopy server and specify an index name that differs from my INDEX_NAME environmental variable.
The canopy server would start on the specified index name.
🚨 Note 🚨
For debugging only. To run the Canopy server in production, please use Docker with a Gunicorn server.
Error: Index name provided via --index-name 'canopy--test' does not match the index name provided via the INDEX_NAME environment variable 'test1'
- **OS**: Windows 11
- **Language version**: Python 3.10
- **Canopy version**: 0.6.0
- **Terminal**: Windows Powershell
In the canopy help options, it says:
--index-name TEXT Index name, if not provided already as an
environment variable.
So I assume I can't use a different index than the one specified in the environment variable. But I want to make sure this is a feature, not a bug.
I confirmed the index name in the log is the same as index I created.
Moving Canopy towards being more multi-modal, would be good to allow uploading (or simply linking to?) images and having them vectorised and queryable
No response
No response
No response
No response
When importing a file with canopy-cli (upsert) the f.read()
in canopy_cli\data_loader\data_loader.py
can throw a UnicodeDecodeError
exception.
This results in the CLI responding with a very uninformative error message like this:
Error: A unexpected error while loading the data from files in .\data\hello\test.txt. Please make sure the data is in valid `jsonl`, `parquet`, `csv` format or plaintext `.txt` files.
If I patch the code to print out the exception like this,
try:
text = f.read()
except UnicodeDecodeError as e:
print("Error reading file: ", e)
It prints the exception message:
'charmap' codec can't decode byte 0x9d in position 249: character maps to <undefined>
So, it decoding UTF-8 seems to be the root problem.
Either import the file, handling the error, or provide a better error message so the files can be fixed.
I am using Python 3.11 on Windows.
To reproduce create a text file test.txt
as a UTF-8 encoded text file containing e.g. a 0x9d byte.
canopy upsert test.txt
then fails:
Error: A unexpected error while loading the data from files in test.txt. Please make sure the data is in valid `jsonl`, `parquet`, `csv` format or plaintext `.txt` files.
No response
- **OS: Windows 11**:
- **Language version: Python 3.11.5**:
- **Canopy version: canopy-sdk==0.1.4**:
No response
I'm having trouble getting Canopy installed because the Pip doesn't want to install to my virtual environment. It seems to me this could all be avoided with an Extension for VSC. I've istalled many Extentions (Numpy, SQL Tools, Github Copilot) and have had universal success with them.
I've submitted a bug report about the Pip install of Canopy
The very large number of VSC users out there, and Copilot will learn about it and help many users.,
I wish I was that smart.
No response
When calling the chat completions API endpoint directly from a client like the browser, it makes a preflight CORS check using the OPTIONS HTTP request. This is not supported and results in the API call failing as it does not get past this check.
Unless the API endpoint is solely intended for proxy-like clients that entirely control the HTTP calls and headers made, I would expect the OPTIONS HTTP request being supported. This makes it for browser application clients possible to directly interface with the exposed canopy API.
I believe the exposed API is not protected and if this is done intentionally, I'm glad to close the issue :)
INFO: 127.0.0.1:55991 - "OPTIONS /v1/qanda-content/chat/completions HTTP/1.1" 405 Method Not Allowed
n/a
No response
The gunicorn app name is incoreect may:
gunicorn canopy_cli.app:app --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 --workers
gunicorn canopy_server.app:app --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 --workers
ModuleNotFoundError: No module named 'canopy_cli.app'
- **OS**: Ubuntu 20.04.6
- **Language version**: python3.10
- **Canopy version**: canopy-sdk==0.6.0
gunicorn canopy_cli.app:app --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 --workers
The ChatEngine
takes a max_prompt_tokens
argument, which allows limiting the prompt size (message history + system prompt + retrieved context).
Currently, this argument has an arbitrary hard-coded default value of 4096
. However - the real default value should be LLM dependent, as different LLMs have different context windows.
Each LLM
class should have a context_window
cached property (the value may change, according to model_name
).
If the user hasn't specified max_prompt_tokens
(the default should be None
) - the ChatEngine
will use llm.context_window
.
No response
No response
No response
Currently cannopy pydantic version in poetry is : pydantic = "^1.10.7"
When we want to work with applications using pydantic 2 -> it makes it very hard to work with the product and forces downgrades in other libraries
We are using now pydnatic 1 in our application, which cost us refactoring
every user trying to add canopy to a poetry project that contains requirement for updated pydantic
No response
No response
Just created a new venv, installed canopy, configured the environment variables and when i type canopy new testbase i have the following error:
Error: Failed to create a new index. Reason:
Failed to infer vectors' dimension from encoder due to error: You exceeded your current quota, please check your plan and billing details.. Please fix the error or provide the dimension manually
It happens with the free tier pinecone license and also on the Standard.
Should create the index
Env: Windows 11 with miniconda
No response
- **Windows**:
- **Python 3.11.5**:
- **Canopy cli 0.1.2**:
No response
Allow to filter the search in Pinecone using metadata for each call made with the API. A metadata parameter would be added to the endpoint .../?metadata=string
No response
No response
No response
No response
Our current application is running under fastapi 0.104.0, do I need to downgrade to 0.92.0 in order to install canopy? Can we loose the fastapi and uvicorn requirement here?
We do not need the server part of canopy, we want to use the library part to build our own service (I believe there should be many people have the same requirement?), is this possible?
No response
No response
No response
No response
(canopy4.0) perrykrug@Perrys-Pinecone-MacBook:~/canopy$ canopy start
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
🚨 Note 🚨
For debugging only. To run the Canopy server in production, run the command:
gunicorn canopy_server.app:app --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000 --workers <num_workers>
Starting Canopy server on 0.0.0.0:8000
INFO: Started server process [11591]
INFO: Waiting for application startup.
2023-12-11 18:17:51,493 - MainProcess - canopy_server.app [INFO ]: Did not find config file. Initializing engines with default configuration
ERROR: Traceback (most recent call last):
File "/Users/perrykrug/canopy/canopy4.0/lib/python3.11/site-packages/starlette/routing.py", line 671, in lifespan
async with self.lifespan_context(app):
File "/Users/perrykrug/canopy/canopy4.0/lib/python3.11/site-packages/starlette/routing.py", line 566, in aenter
await self._router.startup()
File "/Users/perrykrug/canopy/canopy4.0/lib/python3.11/site-packages/starlette/routing.py", line 648, in startup
await handler()
File "/Users/perrykrug/canopy/canopy4.0/lib/python3.11/site-packages/canopy_server/app.py", line 282, in startup
_init_engines()
File "/Users/perrykrug/canopy/canopy4.0/lib/python3.11/site-packages/canopy_server/app.py", line 338, in _init_engines
kb.connect()
File "/Users/perrykrug/canopy/canopy4.0/lib/python3.11/site-packages/canopy/knowledge_base/knowledge_base.py", line 240, in connect
self._connect_index()
File "/Users/perrykrug/canopy/canopy4.0/lib/python3.11/site-packages/canopy/knowledge_base/knowledge_base.py", line 202, in _connect_index
raise RuntimeError(
RuntimeError: The index canopy--canopy does not exist or was deleted. Please create it by calling knowledge_base.create_canopy_index() or running the canopy new
command
ERROR: Application startup failed. Exiting.
A better error message, or option to create index
Run canopy start
after a new installation, before creating canopy index.
No response
- **OS**:
- **Language version**:
- **Canopy version**:
No response
We currently designed canopy-server
and the LLM
class under the assumption that a developers would build a Chatbot service with Canopy which offers a single bot service. Under that assumption - all the LLM calls are done by a single pre-configured API key, using pre-configured model params (such as model_name
, max_generated_tokens
).
However, some developers might want to offer a multi-tenant bot service over the same knowledge base. In that case, the server would need to take the API key and the model params from each individual /chat.completion
call, and the LLM should allow passing these per call.
We should probably add a bool
in the LLM
's config which determines whether or not to allow to pass model params per call.
We can maybe add a separate config flag specifying whether or not to allow users to pass their own personal LLM API keys.
Theoretically, we can even do a more detailed allow_override
bool per each model param - but that's probably an overkill.
Developers who want to use Canopy to offer multi-tenant chatbot service.
No response
No response
I try to start an index :
canopy new test
and this is the error that I get :
I don't understand how is this going to change the initial variable definition, how should it be provided, I already provided anyscale api key and still cannot use the command :
canonopy new test
This is the error that I get :
(canopy-env) kingsize@Goncalos-MacBook-Pro-3 EdGenAI % canopy new test
Canopy is going to create a new index named canopy--test with the following initialization parameters:
{}
Do you want to continue? [y/N]: y
Error: Failed to create a new index. Reason:
Canopy has failed to infer vectors' dimensionality using the selected encoder: OpenAIRecordEncoder. You can provide the dimension manually, try using a different encoder, or fix the underlying error:
Failed to enconde documents using OpenAIRecordEncoder. Error: Your OpenAI account seem to have reached the rate limit. Details: You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.
I should expect th enormal behaviour after that command
Does anybody know how to fix this issue, maybe using anyscale, I already created an account and it works fine with the examples they provide. Thay also provide embeddings.
No response
- **OS**:
- **Language version**:
- **Canopy version**:
No response
A bit more complicated than images, Canopy would likely need to slice up the video (user configureable).
No response
No response
No response
No response
Hi, I see in the README that canopy
actually supports local files likes parquet, is there nay kind of DocumentLoader for online contents like:
... and does it support local pdf import ?
I gues almost any lazy people wanting to give a try and use canopy
No response
No response
Asking questions I ran into this streaming error. I think this is usually caused by nothing being returned in the stream, which is usually because the stream hasn't been closed correctly. I can't see anything in the streaming code that might trigger this, and I've only hit this error once so far, so may be a fluke — will update this issue if I see again.
No streaming errors. I would have expected this to have been caught by the try-except
statement in /context/chat/completions
endpoint.
Following this notebook
No response
- **OS**: MacOS
- **Language version**: 3.9.12
- **Pinecone client version**: 2.2.1
No response
Add --index-name
to canopy start
No response
No response
No response
No response
I am new to Python and its virtual environments, however, the Pip installer is ignoring my istructions on where to install canopy-env because it claims it is unwritable (it is writable) and defaults to an "AppData\Roaming" folder I want nothing to with (having to do with my OneDrive distrust). My System Path points to "C:\Program Files\Python312", "C:\Program Files\Python312\Lib", and "C:\Program Files\Python312\Scripts" and my user Path doesn't have a folder specified for Python.
Install Canopy in the folder I told it to.
Issue the following command:
PS C:\Users\ [username] \Documents\canopy-env> py -m pip install canopy-sdk
The following error will occur (note I am an administrator on this computer and "Trusted Installers" have also been given full permissions to the "C:\Users\[username]\Documents" and "C:\Program Files\Python312\" folders:
==============================================
Defaulting to user installation because normal site-packages is not writeable
Collecting canopy-sdk
Using cached canopy_sdk-0.5.0-py3-none-any.whl.metadata (15 kB)
Collecting fastapi<0.93.0,>=0.92.0 (from canopy-sdk)
Using cached fastapi-0.92.0-py3-none-any.whl (56 kB)
Collecting gunicorn<22.0.0,>=21.2.0 (from canopy-sdk)
Using cached gunicorn-21.2.0-py3-none-any.whl.metadata (4.1 kB)
Collecting jsonschema<5.0.0,>=4.2.0 (from canopy-sdk)
Using cached jsonschema-4.20.0-py3-none-any.whl.metadata (8.1 kB)
Collecting openai<2.0.0,>=1.2.3 (from canopy-sdk)
Using cached openai-1.6.1-py3-none-any.whl.metadata (17 kB)
Collecting pandas-stubs<3.0.0.0,>=2.0.3.230814 (from canopy-sdk)
Using cached pandas_stubs-2.1.4.231227-py3-none-any.whl.metadata (9.6 kB)
Collecting pinecone-client<3.0.0,>=2.2.2 (from canopy-sdk)
Using cached pinecone_client-2.2.4-py3-none-any.whl.metadata (7.8 kB)
Collecting pinecone-datasets<0.7.0,>=0.6.2 (from canopy-sdk)
Using cached pinecone_datasets-0.6.2-py3-none-any.whl.metadata (11 kB)
Collecting pinecone-text<0.8.0,>=0.7.1 (from canopy-sdk)
Using cached pinecone_text-0.7.1-py3-none-any.whl.metadata (9.0 kB)
Collecting prompt-toolkit<4.0.0,>=3.0.39 (from canopy-sdk)
Using cached prompt_toolkit-3.0.43-py3-none-any.whl.metadata (6.5 kB)
Collecting pydantic<2.0.0,>=1.10.7 (from canopy-sdk)
Using cached pydantic-1.10.13-py3-none-any.whl.metadata (149 kB)
Collecting python-dotenv<2.0.0,>=1.0.0 (from canopy-sdk)
Using cached python_dotenv-1.0.0-py3-none-any.whl (19 kB)
Collecting sentencepiece<0.2.0,>=0.1.99 (from canopy-sdk)
Using cached sentencepiece-0.1.99.tar.gz (2.6 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [31 lines of output]
Traceback (most recent call last):
File "C:\Users\gaopt\AppData\Roaming\Python\Python312\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
main()
File "C:\Users\gaopt\AppData\Roaming\Python\Python312\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\gaopt\AppData\Roaming\Python\Python312\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\gaopt\AppData\Local\Temp\pip-build-env-v_l6vtru\overlay\Lib\site-packages\setuptools\build_meta.py",
line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\gaopt\AppData\Local\Temp\pip-build-env-v_l6vtru\overlay\Lib\site-packages\setuptools\build_meta.py",
line 295, in _get_build_requires
self.run_setup()
File "C:\Users\gaopt\AppData\Local\Temp\pip-build-env-v_l6vtru\overlay\Lib\site-packages\setuptools\build_meta.py",
line 480, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "C:\Users\gaopt\AppData\Local\Temp\pip-build-env-v_l6vtru\overlay\Lib\site-packages\setuptools\build_meta.py",
line 311, in run_setup
exec(code, locals())
File "<string>", line 126, in <module>
File "C:\Program Files\Python312\Lib\subprocess.py", line 408, in check_call
retcode = call(*popenargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python312\Lib\subprocess.py", line 389, in call
with Popen(*popenargs, **kwargs) as p:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python312\Lib\subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Program Files\Python312\Lib\subprocess.py", line 1538, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 2] The system cannot find the file specified
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
- **Windows 10**:
- **Python312**:
- **canopy_sdk-0.5.0-py3-none-any.whl.metadata**:
No response
It would be good if the user could choose which port to run canopy on. This works although it's undocumented;
canopy start --port 8002
But canopy health
and canopy chat
both still look for port 8000.
No response
No response
No response
No response
RetryError
is shadowing the actual underlying error.
| 350/423 [25:13<05:07, 4.21s/it]Error: Failed to upsert data to index canopy--canopy-101. Underlying error: RetryError[<Future at 0x168f90f10 state=finished raised APIError>]
You can allow partial failures by setting --allow-failures.
The inner error should be printed
@retry
failsNo response
- **OS**:
- **Language version**:
- **Pinecone client version**:
No response
We will need async and streaming to integrate canopy into our app, do you have any timeline when those will be supported?
No response
No response
No response
No response
Allow for users to provide details of a fine tuned (OpenAI) model, to replace the default model used by Canopy.
None.
Anyone building RAG applications wanting to build both a knowledge base and develop the custom LLM to work with the information. It might avoid the need for very detailed prompts as the current status does require very specific prompting to generate good results.
No
No response
The server starts in the terminal. But when I do canopy chat
it doesn't do anything. It is just blank. It doesn't crash or do anything.
The terminal just stays blank.
This is what should happen
No response
- **OS**: Windows 11
- **Language version**: Python 3.11.4
- **Canopy version**: 0.5.0
No response
I sent a user message and got an error on the 'Without Context (No RAG)' section in canopy chat. Error code: 400. Basically, the model's maximum context length was exceeded from user messages results in a canopy chat thread.
With Context (RAG):
Yes, for a Dramatic clothing personality type in a casual wear setting with a Winter seasonal palette, purple can be a recommended color for sweaters or tops. The Winter palette includes colors like royal purple, which is a bright and bold shade of purple. Winter palettes strive to stay sharp and are best in clear and high-contrast colors, so a vibrant purple can be a great choice for a Dramatic personality type in casual wear. Source: 978-0345345462
Without Context (No RAG):
(OpenAI's response)
Yes, for a Dramatic clothing personality type in a casual wear setting with a Winter seasonal palette, purple can be a recommended color for sweaters or tops. The Winter palette includes colors like royal purple, which is a bright and bold shade of purple. Winter palettes strive to stay sharp and are best in clear and high-contrast colors, so a vibrant purple can be a great choice for a Dramatic personality type in casual wear. Source: 978-0345345462
Traceback (most recent call last):
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\canopy_cli\cli.py", line 443, in _chat
openai_response = client.chat.completions.create(
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\_utils\_utils.py", line 271, in wrapper
return func(*args, **kwargs)
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\resources\chat\completions.py", line 648, in create
return self._post(
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\_base_client.py", line 1167, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\_base_client.py", line 856, in request
return self._request(
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\_base_client.py", line 947, in _request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 4097 tokens. However, your messages resulted in 4135 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\J\AppData\Local\Programs\Python\Python310\Scripts\canopy.exe\__main__.py", line 7, in <module>
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\canopy_cli\cli.py", line 577, in chat
_ = _chat(
File "C:\Users\J\AppData\Local\Programs\Python\Python310\lib\site-packages\canopy_cli\cli.py", line 447, in _chat
err = e.http_body if isinstance(e, OpenAI_APIError) else str(e)
AttributeError: 'BadRequestError' object has no attribute 'http_body'
- **OS**: Windows 11
- **Language version**: Python 3.10
- **Canopy version**: 0.6.0
- **Terminal**: Windows Powershell
I used the "canopy chat --no rag". The "With Context (RAG):" response sent accurate and fully. The error occurred on the " Without Context (No RAG):" part.
I converted the JSONL file to text, and attached that for reference. I 'canopy upsert' the JSONL file to my canopy.
I'm using two instances of Windows Powershell; one for the server, one for the chat. Testing purposes.
OpenAI API key and Pinecone API key are set as Windows Environmental Variables.
I used my OpenAI API key. I haven't edited any OpenAI settings via Canopy (I don't know how).
As far as I can see, for the RAG generated response, there's no meta-data with regards to which vector documents it has used. This would be very useful for end-users so that they can get references as to which documents the response is referring to.
Edit Having dived into the code base, is this intended to be build perhaps by clients using this framework by building their own ContextBuilder?
No response
I believe there's a lot use cases that would want to prove that what they output is correct. The end user can manually verify or get more in-depth information based on meta-data coming back from RAG retrieved documents.
My Python knowledge is very limited, but I'll give it a go if maintainers direct me as to how they see the data object structure change for example.
Nothing at the moment, I think the feature is pretty straight-forward. I'd suggest the client supports a simple list of strings representing the meta-data properties they would like to retrieve from Pinecone, and include those in the RAG response.
The pip installer for canopy-sdk fails.
I expect the pip installer to be work.
In the command prompt run: python -m pip install canopy-sdk
Collecting sentencepiece<0.2.0,>=0.1.99 (from canopy-sdk)
Using cached sentencepiece-0.1.99.tar.gz (2.6 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [47 lines of output]
CMake Deprecation Warning at CMakeLists.txt:15 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
-- VERSION: 0.1.99
CMake Error at CMakeLists.txt:23 (project):
Generator
NMake Makefiles
does not support platform specification, but platform
x64
was specified.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLan
- **Windows 10**:
- **Python 3.12.1**:
- **Latest pip**:
- **CMake 3.28.1**:
My suspicion is that sentencepiece-0.1.99.tar.gz needs an earlier version of CMake., or sentencepiece is essentially (currently) incompatible with Python 3.12.1. See google/sentencepiece#932 .
The CLI uses the openai
client in canopy chat
to talk with the Canopy server, regardless of the LLM actually being used (which might be non-OpenAI).
But since openai
client verifies the presence of an Open AI API key, this errors out, even though the configuration is correct.
We need to change this line to pass some dummy api_key=
value.
canopy chat
should work with any valid LLM config
OPENAI_API_KEY
env var.canopy chat
No response
- **OS**:
- **Language version**:
- **Canopy version**:
No response
Current upsert behavior (on hitting resin upsert ./data.parquet
) seems to be encoding everything to embeddings first, then once all are done it upserts to Pinecone.
A couple of issues with this:
Default batch size of 100
that encodes 100 embeddings, upserts to Pinecone, and then moves onto the next batch.
Can try running https://gist.github.com/jamescalam/35ad30a0e0a7d20da7b77ce628b264e2
No response
- **OS**: MacOS
- **Language version**: Python 3.9.12
- **Pinecone client version**: 2.2.1
No response
When running canopy new
, the config file specified by the environment variable $CANOPY_CONFIG_FILE is not read. The config file must be manually specified using the -c
parameter.
The config file specified by $CANOPY_CONFIG_FILE is read.
Run canopy new
No response
- **Canopy version**: master branch
No response
It would be incredible if we could run local canopy with Mixtral 8x7b — which (afaik) would need GGUF quantized Mixtral via vLLM. This would open us up to integrations with things like formal grammars too (which again, afaik need local models, I don't think any API solutions accept it)
Holiday season is just around the corner and not sure if you guys got me anything so just putting this out there as an idea
No response
The world, but primarily open LLM devs. Would probably be less production use, but I'm sure having this and being able to run for free (Pinecone free tier + local LLM) would push forward more devs building with canopy imo
maybe yes
Requires around 30GB of memory using GGUF quantized Mixtral https://huggingface.co/TheBloke/Mixtral-8x7B-v0.1-GGUF — this would fit on Mac M1/M2 chips
When trying to upsert a relatively small dataset I'm hitting OpenAI rate limits:
For this to be handled with exponential backoff and/or a rate parameter we can set from CLI
Follow this notebook
No response
- **OS**: MacOS
- **Language version**: 3.9.12
- **Pinecone client version**: 2.2.1
No response
minor UX improvement to chat would be an immediate acknowledgement that the user input has been sent to the LLM. Right now, it hangs for a second or two and it's not clear what's happening (or if I remembered to hit Esc first or not)
No response
No response
No response
No response
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.