pymilvus.exceptions.ParamError: <ParamError: (code=1, message=invalid input, length of string exceeds max length. length: 68919, max length: 65535)> about langchain-chatchat

[dosubot]

To address the ParamError due to strings exceeding the maximum allowed length when inserting documents into Milvus, ensure your strings do not surpass the Milvus schema's limit for string fields. This error occurs when text data inserted into a field exceeds its maximum length.

To fix this, you can:

1. Validate or Truncate Strings: Before insertion, check the string lengths against the maximum allowed by your Milvus collection's schema. If any string is too long, truncate it to fit the limit or consider not inserting it.

2. Adjust the Schema: If truncating strings is not viable, you might need to adjust your collection's schema to accommodate longer strings, provided Milvus and your application design allow it.

Here's a code snippet to truncate strings to a specified maximum length:

max_length = 255  # Adjust based on your schema's requirements

for doc in docs:
    for k, v in doc.metadata.items():
        # Truncate strings to the maximum allowed length
        doc.metadata[k] = str(v)[:max_length]
    for field in self.milvus.fields:
        doc.metadata.setdefault(field, "")
    doc.metadata.pop(self.milvus._text_field, None)
    doc.metadata.pop(self.milvus._vector_field, None)

This approach ensures all strings are within the allowed length before attempting insertion, preventing the ParamError. For more detailed guidance, refer to the modifications in the Langchain-Chatchat repository.

Details

Sources

• server/knowledge_base/kb_service/milvus_kb_service.py
• configs/kb_config.py.example

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.