milvus-io / milvus-sdk-node Goto Github PK

Describe the bug:

The createCollection function fails if supplying a primary key field with a string type. According to the docs this should be supported: https://milvus.io/docs/create_collection.md

Steps to reproduce:

1. await client.createCollection({
     collection_name: 'coll',
     fields: [{ name: 'id', data_type: DataType.String, is_primary_key: true }],
   });

Gives

Error: The `data_type` for the primary key field must be DataType.Int64.

    at checkCollectionFields (node_modules/.pnpm/@[email protected]/node_modules/@zilliz/milvus2-sdk-node/milvus/utils/Validate.ts:82:11)
    at MilvusClient.<anonymous> (node_modules/.pnpm/@[email protected]/node_modules/@zilliz/milvus2-sdk-node/milvus/grpc/Collection.ts:110:26)
    at Generator.next (<anonymous>)
    at node_modules/.pnpm/@[email protected]/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/grpc/Collection.js:8:71
    at new Promise (<anonymous>)
    at Object.<anonymous>.__awaiter (node_modules/.pnpm/@[email protected]/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/grpc/Collection.js:4:12)
    at MilvusClient._createCollection (node_modules/.pnpm/@[email protected]/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/grpc/Collection.js:74:16)
    at MilvusClient.<anonymous> (node_modules/.pnpm/@[email protected]/node_modules/@zilliz/milvus2-sdk-node/milvus/MilvusClient.ts:82:25)
    at Generator.next (<anonymous>)
    at fulfilled (node_modules/.pnpm/@[email protected]/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/MilvusClient.js:5:58)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

Milvus-node-sdk version:
3.2.15

Milvus version:
3.2.9

Describe the bug:
我在用langchainjs集成milvus，希望构建知识库检索的功能，遇到这个问题：
Error: Insert fail: some field does not exist for this collection in line. 0

Steps to reproduce:

1.使用https://github.com/zilliztech/cloud-vectordb-examples/blob/master/node/HelloZillizCloud.js创建好了collection，
2.使用langchainjs官网集成milvus的例子https://js.langchain.com/docs/modules/indexes/vector_stores/integrations/milvus，Index and query docs标签下
`import { Milvus } from "langchain/vectorstores/milvus";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";

// text sample from Godel, Escher, Bach
const vectorStore = await Milvus.fromTexts(
[
"Tortoise: Labyrinth? Labyrinth? Could it Are we in the notorious Little
Harmonic Labyrinth of the dreaded Majotaur?",
"Achilles: Yiikes! What is that?",
"Tortoise: They say-although I person never believed it myself-that an I
Majotaur has created a tiny labyrinth sits in a pit in the middle of
it, waiting innocent victims to get lost in its fears complexity.
Then, when they wander and dazed into the center, he laughs and
laughs at them-so hard, that he laughs them to death!",
"Achilles: Oh, no!",
"Tortoise: But it's only a myth. Courage, Achilles.",
],
[{ id: 2 }, { id: 1 }, { id: 3 }, { id: 4 }, { id: 5 }],
new OpenAIEmbeddings(),
{
collectionName: "goldel_escher_bach",
}
);

// or alternatively from docs
const vectorStore = await Milvus.fromDocuments(docs, new OpenAIEmbeddings(), {
collectionName: "goldel_escher_bach",
});

const response = await vectorStore.similaritySearch("scared", 2);
3.我将ids改成[
{ id: 2, word_count: 24, book_intro: Math.floor(Math.random() * 1000) },
{ id: 1, word_count: 4, book_intro: Math.floor(Math.random() * 1000) },
{ id: 3, word_count: 6, book_intro: Math.floor(Math.random() * 1000) },
{ id: 4, word_count: 2, book_intro: Math.floor(Math.random() * 1000) },
{ id: 5, word_count: 9, book_intro: Math.floor(Math.random() * 1000) },
]`
4. 跑起来报上面的错，请帮忙看看怎么回事，谢谢

Milvus-node-sdk version:
2.2.10
Milvus version:
zilliz cloud

Describe the bug:
I created a collection with hyphens in the name (e.g. my-first-collection), and was never notified about it being invalid.

Until I tried inserting data I got a somewhat confusing error:

TypeError: Cannot read properties of null (reading 'fields')
  at formatDescribedCol (/Users/magnusdr/fink/node_modules/@zilliz/milvus2-sdk-node/milvus/utils/Format.ts:345:15)
  at MilvusClient.<anonymous> (/Users/magnusdr/fink/node_modules/@zilliz/milvus2-sdk-node/milvus/grpc/Collection.ts:303:39)
  at Generator.next (<anonymous>)
  at fulfilled (/Users/magnusdr/fink/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/grpc/Collection.js:5:58)
  at processTicksAndRejections (node:internal/process/task_queues:95:5)

formatDescribedCol fails because the data argument has schema: null, but if you look at it's status it says

status: {
  error_code: 'UnexpectedError',
  reason: 'Invalid collection name: public-fink-document-embeddings. Collection name can only contain numbers, letters, dollars and underscores.'
}

Steps to reproduce:

await client.createCollection({
  collection_name: 'my-first-collection',
  fields: [{
    name: 'id',
    data_type: DataType.Int64,
    description: 'Primary key',
    is_primary_key: true,
    autoID: true,
  }]
})
await client.insert({
   collection_name: MILVUS_PRIVATE_COLLECTION_NAME,
   fields_data: [],
});

Milvus-node-sdk version:
2.2.23

Milvus version:
2.0.2

Describe the bug:
I noticed I recently started receiving the below error when calling getVersion using our zilla-hosted Milvus instance. After talking to their support, they claim that this api is no longer supported on newer versions of Milvus (why??). I guess this means it should be removed from milvus-sdk-node?

Rejection - {
    "stack": "Error: 2 UNKNOWN: it's a deny api, fullMethod:/milvus.proto.milvus.MilvusService/GetVersion
    at callErrorFromStatus (/media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
    at Object.onReceiveStatus (/media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/client.js:192:76)
    at /media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/call-interface.js:78:35
    at Object.onReceiveStatus (/media/data0/dev/proj/node/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/utils/Grpc.js:180:29)
    at InterceptingListenerImpl.onReceiveStatus (/media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/call-interface.js:73:23)
    at Object.onReceiveStatus (/media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:360:141)
    at Object.onReceiveStatus (/media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
    at /media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/resolving-call.js:94:78
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
for call at
    at ServiceClientImpl.makeUnaryRequest (/media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/client.js:160:34)
    at ServiceClientImpl.<anonymous> (/media/data0/dev/proj/node/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
    at /media/data0/dev/proj/node/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/utils/Function.js:20:24
    at new Promise (<anonymous>)
    at promisify (/media/data0/dev/proj/node/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/utils/Function.js:17:17)
    at MilvusClient.<anonymous> (/media/data0/dev/proj/node/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/grpc/GrpcClient.js:183:44)
    at Generator.next (<anonymous>)
    at /media/data0/dev/proj/node/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/grpc/GrpcClient.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/media/data0/dev/proj/node/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/grpc/GrpcClient.js:4:12)",
    "message": "2 UNKNOWN: it's a deny api, fullMethod:/milvus.proto.milvus.MilvusService/GetVersion",
    "code": 2,
    "details": "it's a deny api, fullMethod:/milvus.proto.milvus.MilvusService/GetVersion",
    "metadata": {
        "content-type": [
            "application/grpc"
        ],
        "via": [
            "1.1 google"
        ],
        "date": [
            "Thu, 06 Jul 2023 17:38:36 GMT"
        ],
        "content-length": [
            "0"
        ],
        "alt-svc": [
            "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000"
        ]
    }
}

Describe the bug:
@zilliz/milvus2-sdk-node can not work on grpc/grpc-js: 1.8.19 default dependency

Steps to reproduce:

error demo: https://github.com/yuyicai/milvus2-sdk-node_grpc-error-demo

• milvus2-sdk-node version:

  "dependencies": {
    "@zilliz/milvus2-sdk-node": "=2.2.20"
  }

• node version:

v18.16.1

• install dependencies:
  run npm install, the grcp/grpc-js: 1.8.19 dependency will be installed automatically.

    "node_modules/@grpc/grpc-js": {
      "version": "1.8.19",
      "resolved": "https://registry.npmjs.org/@grpc/grpc-js/-/grpc-js-1.8.19.tgz",
      "integrity": "sha512-yCkvhtstJvUL3DEQAF5Uq5KoqQL27MTdfxVYvGsGduoGxiJcRCvJ29t5OmLfLhRuRpjAQ1OlhJD/IPOfX+8jNw==",
      "dependencies": {
        "@grpc/proto-loader": "^0.7.0",
        "@types/node": ">=12.12.47"
      },
      "engines": {
        "node": "^8.13.0 || >=10.10.0"
      }
    },

• error:
  index.ts

import { MilvusClient } from '@zilliz/milvus2-sdk-node';

(async () => {
  // build client
  const milvusClient = new MilvusClient({
    address: 'localhost:19530',
  });
  await milvusClient.checkHealth();
})();

run ts-node index.ts, the error is:

/milvus-foo/node_modules/@grpc/grpc-js/src/transport.ts:255
    logging.trace(LogVerbosity.DEBUG, 'keepalive', '(' + this.channelzRef.id + ') ' + this.subchannelAddressString + ' ' + text);
                                                                          ^
TypeError: Cannot read properties of undefined (reading 'id')
    at Http2Transport.keepaliveTrace (/milvus-foo/node_modules/@grpc/grpc-js/src/transport.ts:255:75)
    at Http2Transport.maybeStartKeepalivePingTimer (/milvus-foo/node_modules/@grpc/grpc-js/src/transport.ts:367:12)
    at new Http2Transport (/milvus-foo/node_modules/@grpc/grpc-js/src/transport.ts:151:12)
    at ClientHttp2Session.<anonymous> (/milvus-foo/node_modules/@grpc/grpc-js/src/transport.ts:622:17)
    at Object.onceWrapper (node:events:628:26)
    at ClientHttp2Session.emit (node:events:513:28)
    at ClientHttp2Session.emit (node:domain:489:12)
    at emit (node:internal/http2/core:330:3)
    at processTicksAndRejections (node:internal/process/task_queues:84:21)

• it work on grpc/grpc-js: 1.8.17 dependency
  when change the grpc/grpc-js dependency to 1.8.17, it works.

  "dependencies": {
    "@zilliz/milvus2-sdk-node": "=2.2.20",
    "@grpc/grpc-js": "=1.8.17"
  }

Milvus-node-sdk version:
2.2.20
Milvus version:
docker image milvusdb/milvus:v2.2.11

Describe the bug:
We are regularly seeing the following uncaught exception in our node app. All of our calls to the MilvusClient are awaited and wrapped in a try...catch block, so I guess something's being done internal to the lib that is allowing for the uncaught exception.

2023-06-19T16: 01: 33.637Z - uncaughtException - {
    "stack": "Error: write EPIPE
    at afterWriteDispatched (node:internal/stream_base_commons:160:15)
    at writeGeneric (node:internal/stream_base_commons:151:3)
    at Socket._writeGeneric (node:net:917:11)
    at Socket._write (node:net:929:8)
    at writeOrBuffer (node:internal/streams/writable:392:12)
    at _write (node:internal/streams/writable:333:10)
    at Writable.write (node:internal/streams/writable:337:10)
    at console.value (node:internal/console/constructor:300:16)
    at console.warn (node:internal/console/constructor:382:26)
    at Object.error (/home/user/dev/proj/node/node_modules/@grpc/grpc-js/build/src/logging.js:24:17)",
    "message": "write EPIPE",
    "errno": -32,
    "code": "EPIPE",
    "syscall": "write"
}

Milvus-node-sdk version:
2.2.17

Describe the feature:
Now we put all apis in one file, it will be more and more hard to maintain.

Plan to split code to different classes.

1. Milvus - connect | get milvus infos api
2. Collection - crud collection api
3. Partition - crud partitions api
4. Index - crud index api
5. Data - insert | search | query | flush api

Describe a specific use case for the feature:

Describe the bug:
I have a standalone installation of Milvus with Docker Compose. Using the Attu tool I can easily connect, create a collection and populate it with the sample data. Collection data also gets loaded and I can perform queries without any issues.
However, in my production environment with standalone installation using Kubernetes cluster (with Helm) the collection fails to load after creation and after some time I get a timeout notice.

The initial error message reads:
"failed to get channels, collection not loaded: collection=444710574464845978: collection not found"

This is somehow weird, as the collection is populated with data and is displayed in attu.
Not sure, whether this is related to some memory issues but loading a collection with only 100 entities shouldn't be of a big deal!

Here are the related logs from the pod:

[2023/10/06 12:29:00.212 +00:00] [INFO] [querycoordv2/services.go:787] ["get replicas request received"] [traceID=438b2f70d1034f2a08e6845efca866ae] [collectionID=444710574464845978] [with-shard-nodes=false]
2023-10-06T14:29:00.212777686+02:00 [2023/10/06 12:29:00.212 +00:00] [WARN] [querycoordv2/handlers.go:321] ["failed to get channels, collection not loaded"]
[2023/10/06 12:29:00.212 +00:00] [WARN] [querycoordv2/services.go:824] ["failed to get replica info"] [traceID=438b2f70d1034f2a08e6845efca866ae] [collectionID=444710574464845978] [replica=444710575524806673] [error="failed to get channels, collection not loaded: collection=444710574464845978: collection not found"] [errorVerbose="failed to get channels, collection not loaded: collection=444710574464845978: collection not found\n(1) attached stack trace\n -- stack trace:\n | github.com/milvus-io/milvus/pkg/util/merr.WrapErrCollectionNotFound\n | \t/go/src/github.com/milvus-io/milvus/pkg/util/merr/utils.go:244\n | [...repeated from below...]\nWraps: (2) failed to get channels, collection not loaded\nWraps: (3) attached stack trace\n -- stack trace:\n | github.com/milvus-io/milvus/pkg/util/merr.wrapWithField\n | \t/go/src/github.com/milvus-io/milvus/pkg/util/merr/utils.go:531\n | github.com/milvus-io/milvus/pkg/util/merr.WrapErrCollectionNotFound\n | \t/go/src/github.com/milvus-io/milvus/pkg/util/merr/utils.go:242\n | github.com/milvus-io/milvus/internal/querycoordv2.(*Server).fillReplicaInfo\n | \t/go/src/github.com/milvus-io/milvus/internal/querycoordv2/handlers.go:322\n | github.com/milvus-io/milvus/internal/querycoordv2.(*Server).GetReplicas\n | \t/go/src/github.com/milvus-io/milvus/internal/querycoordv2/services.go:822\n | github.com/milvus-io/milvus/internal/distributed/querycoord.(*Server).GetReplicas\n | \t/go/src/github.com/milvus-io/milvus/internal/distributed/querycoord/service.go:379\n | github.com/milvus-io/milvus/internal/proto/querypb._QueryCoord_GetReplicas_Handler.func1\n | \t/go/src/github.com/milvus-io/milvus/internal/proto/querypb/query_coord.pb.go:5537\n | github.com/milvus-io/milvus/pkg/util/interceptor.ServerIDValidationUnaryServerInterceptor.func1\n | \t/go/src/github.com/milvus-io/milvus/pkg/util/interceptor/server_id_interceptor.go:54\n | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n | \t/go/pkg/mod/github.
com/grpc-ecosystem/[email protected]/chain.go:25\n | github.com/milvus-io/milvus/pkg/util/interceptor.ClusterValidationUnaryServerInterceptor.func1\n | \t/go/src/github.com/milvus-io/milvus/pkg/util/interceptor/cluster_interceptor.go:48\n | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n | github.com/milvus-io/milvus/pkg/util/logutil.UnaryTraceLoggerInterceptor\n | \t/go/src/github.com/milvus-io/milvus/pkg/util/logutil/grpc_interceptor.go:23\n | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n | go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryServerInterceptor.func1\n | \t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/[email protected]/interceptor.go:342\n | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1\n | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25\n | github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1\n | \t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:34\n | github.com/milvus-io/milvus/internal/proto/querypb._QueryCoord_GetReplicas_Handler\n | \t/go/src/github.com/milvus-io/milvus/internal/proto/querypb/query_coord.pb.go:5539\n | google.golang.org/grpc.(*Server).processUnaryRPC\n | \t/go/pkg/mod/google.golang.org/[email protected]/server.go:1345\n | google.golang.org/grpc.(*Server).handleStream\n | \t/go/pkg/mod/google.golang.org/[email protected]/server.go:1722\n | google.golang.org/grpc.(*Server).serveStreams.func1.2\n | \t/go/pkg/mod/google.golang.org/[email protected]/server.go:966\n | runtime.goexit\n | \t/usr/local/go/src/runtime/asm_amd64.s:1598\nWraps: (4) collection=444710574464845978\nWraps: (5) collection not found\nError types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4)
*errutil.withPrefix (5) merr.milvusError"]

Milvus-node-sdk version:
2.3.1

Milvus version:
v2.3.1

changed from [{key: 'key1', value: 'xx}, {key: 'key2', value: 'yy} ] to {key1: xx, key2: yy}

Describe the feature:
PyMilvus has a nice timeout setting for requests. Could we have the same for node?

Describe a specific use case for the feature:
I don't want my whole service to hang indefinitely if the milvus server doesn't respond for some reason.

Describe the bug:
search results are wrong
Steps to reproduce:

search results are wrong

const { MilvusClient, DataType } = require('@zilliz/milvus2-sdk-node')
const collectionName = 'search_test'
const vectors = [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]

const params = {
collection_name: collectionName,
description: 'search test',
fields: [
{
name: 'id',
data_type: DataType.Int64,
is_primary_key: true,
description: ''
},
{
name: 'content',
data_type: DataType.VarChar,
max_length: 4096,
description: ''
}, {
name: 'vector',
description: '',
data_type: DataType.FloatVector,
dim: 2
}
]
}

async function testSearch() {
const config = $configure.getConfig()
const ssl = false

const milvusClient = new MilvusClient(
    config.milvus.host,
    ssl,
    config.milvus.user,
    config.milvus.pwd
)
const hasCollection = await milvusClient.hasCollection({ collection_name: collectionName })
if (hasCollection.value) {
    //await milvusClient.dropCollection({ collection_name: collectionName })
} else {
    const resStatus = await milvusClient.createCollection(params)
    console.log(resStatus.error_code)
    console.log(resStatus.reason)

    await createIndex(milvusClient)

    const data = []
    for (var i = 0; i < vectors.length; i++) {
        data.push({
            'id': i,
            'content': i,
            'vector': vectors[i]
        })
    }
    const insertStatus = await milvusClient.insert({
        collection_name: collectionName,
        fields_data: data,
    })
    console.log(insertStatus)
    await milvusClient.flush({ collection_names: [collectionName] })
}

const loadStatus = await milvusClient.loadCollection({
    collection_name: collectionName
})


console.log(loadStatus)

const searchParams = {
    anns_field: 'vector',
    topk: 1,
    metric_type: 'L2',
    params: JSON.stringify({ nprobe: 1 })
}

const tableHeadersResults = await milvusClient.search({
    collection_name: collectionName,
    vectors: [[1, 2], [3, 4], [5, 6], [1, 2]],
    search_params: searchParams,
    vector_type: DataType.FloatVector
})

//  [5, 6], [1, 2] get wrong results

milvusClient.closeConnection()
return true

}

async function createIndex(milvusClient) {
const index_params = {
metric_type: 'L2',
index_type: 'IVF_FLAT',
params: JSON.stringify({ nlist: 1 })
}

await milvusClient.createIndex({
    collection_name: collectionName,
    field_name: 'vector',
    extra_params: index_params,
})

}

Milvus-node-sdk version:
2.2.12
Milvus version:
latest

Describe the feature:
After load collection / partition, we can know when it's loaded.

1. show collections / partitions pass collection / partition name you want to check
2. response includes inMemory_percentages, 100% mean loaded

it seems useDatabase is not available in node api the docs mentions it . but its not there getting error

TypeError: client.useDatabase is not a function

Milvus-node-sdk version:
2.2.18
Milvus version:
2.2.10

Describe the bug:
Milvus support change max_receive_message_length in yaml.
So SDK should set max_receive_message_length as unlimited.
Give all validations to Milvus

Steps to reproduce:

1. Load Collection
2. Import data
3. Query data > 4MB
4. Will throw error by sdk: Received message larger than max (5508333 vs. 4194304)

Milvus-node-sdk version:
v2.0.1
Milvus version:
V2.0.1

Describe the feature:

Now response :

{
  collection_names: [ 'collection_whuqwi8c' ],
  collection_ids: [ '427148280541741057' ],
  created_timestamps: [ '427148280541741058' ], // i don't think this is useful, so we only return utc_timestamp to user
  created_utc_timestamps: [ '1629441377799' ],
  inMemory_percentages: [ '0' ],
 }

transform to:

[
{ name:"", id:"", timestamp:"", loadedPercentage:""}
]

Describe the feature:

• Nuxt3
• NuxtPlugins

Describe a specific use case for the feature:

/plugins/milvus.ts

import { MilvusClient } from '@zilliz/milvus2-sdk-node';

export default defineNuxtPlugin((nuxtApp) => {
  const config = useRuntimeConfig();

  const milvusClient = new MilvusClient({
    address: config.public.milvusHost,
    ssl: false,
  });

  nuxtApp.provide('milvus', milvusClient);
});

declare module '#app' {
  interface NuxtApp {
    $milvus: MilvusClient;
  }
}

declare module '@vue/runtime-core' {
  interface ComponentCustomProperties {
    $milvus: MilvusClient;
  }
}

Describe the feature:
Node doesn't support int64, it will parse int64(defined in the protobuf file) to string. The SDK should handle this situation since most of the fields will not exceed the limit of int32.

Provide a count function that sends only count of the items found of a query result, currently query sends ids, i don't want ids i just want a count

Describe a specific use case for the feature:

1. it might help in many cases where a queue/process is feeding the data into a collection, to track count of items present
2. it will be good for analytics of a specific meta data field
3. items added in last n minutes / hours

Describe the bug:
/milvus/grpc/Data.ts

const output_fields = [
        'id',
        ...(promise.results.output_fields ||
          fields_data.map(f => f.field_name)),
      ];

promise.results.output_fields is empty while data.output_fields is not. I changed promise.results.output_fields to data.output_fields and it works.
Steps to reproduce:

1.call search api

this.milvus.search({
    collection_name: 'LangChainCollection_001',
    output_fields: ['source', 'text'],
    search_params: {
        params: '{"ef":250}',
        anns_field: 'vector',
        topk: '10',
        metric_type: 'L2',
        round_decimal: -1
    },
    vector_type: 101,
    vectors: [
        [...]
    ]
})

2. Not return these output_fields in request

3.change promise.results.output_fields to data.output_fields, it works.

Milvus-node-sdk version:
@zilliz/milvus2-sdk-node: "^2.2.23"
Milvus version:
v2.2.6

Describe the feature:
search api is kind complex.
Describe a specific use case for the feature:

Describe the feature:

the client should support search for multiple vectors
See milvus-io/milvus#25639

Describe a specific use case for the feature:
Some collection need multiple vectors to represent them. The should be indexable and searchable.

I did not find any fields related to dynamic schema in the coresponding example file.
Is it really a dynamicSchema example?
In the python example, at least an option: enable_dynamic_field=True.
But in the ts file, no such thing at all.

I'm trying to build a pool of connections to make sure Milvus uptime is always or almost always.
In my logic I have something like this:

return {
        getClient: async () => {
          // Simple round-robin client selection
          let client = pool[currentIndex];
          currentIndex = (currentIndex + 1) % poolSize;

          // Check if client is connected
          if (!client.isConnected()) {
            try {
              // Replace the disconnected client in the pool
              client = await createClient();
              pool[currentIndex] = client;
            } catch (error) {
              console.error('Error reconnecting to Milvus:', error);
              // Handle reconnection failure if needed
            }
          }

          return client;
        },```
        

But I don't know how to translate `isConnected()` 
Any guidance is highly appreciated. 

Describe the bug:
When i try to store some vectors to Milvus the created collection vector field dimension is very low and insert failed with this error :
(I use llama/Llama_Cpp for embeddings)

Error: Error inserting data: {"succ_index":[],"err_index":[0,1,2,3,4],"status":{"error_code":"UnexpectedError","reason":"the length(165) of float data should divide the dim(36)","code":65535,"retriable":false,"detail":""},"IDs":null,"acknowledged":false,"insert_cnt":"0","delete_cnt":"0","upsert_cnt":"0","timestamp":"0"}

The created collection looks like this:

+---------------+----------------------------------------------+
| Name          | test                                         |
+---------------+----------------------------------------------+
| Description   |                                              |
+---------------+----------------------------------------------+
| Is Empty      | True                                         |
+---------------+----------------------------------------------+
| Entities      | 0                                            |
+---------------+----------------------------------------------+
| Primary Field | langchain_primaryid                          |
+---------------+----------------------------------------------+
| Schema        | Description:                                 |
|               |                                              |
|               | Auto ID: True                                |
|               |                                              |
|               | Fields(* is the primary field):              |
|               |  - id FLOAT  Metadata Number field           |
|               |  - *langchain_primaryid INT64  Primary key   |
|               |  - langchain_text VARCHAR  Text field        |
|               |  - vectors FLOAT_VECTOR dim: 36 Vector field |
+---------------+----------------------------------------------+
| Partitions    | - _default                                   |
+---------------+----------------------------------------------+
| Indexes       | - vectors                                    |
+---------------+----------------------------------------------+

This is the code i've ran

import { LlamaCppEmbeddings } from 'langchain/embeddings/llama_cpp';
import { Milvus } from 'langchain/vectorstores/milvus';

const address = `localhost:19530`;
const llamaPath =
  "/Users/cyrille/.cache/lm-studio/models/TheBloke/Llama-2-7B-GGUF/llama-2-7b.Q4_K_M.gguf";


const embeddings = new LlamaCppEmbeddings({
  modelPath: llamaPath
});
// text sample from Godel, Escher, Bach
const vectorStore = await Milvus.fromTexts(
  [
    'Tortoise: Labyrinth? Labyrinth? Could it Are we in the notorious Little\
            Harmonic Labyrinth of the dreaded Majotaur?',
    'Achilles: Yiikes! What is that?',
    'Tortoise: They say-although I person never believed it myself-that an I\
            Majotaur has created a tiny labyrinth sits in a pit in the middle of\
            it, waiting innocent victims to get lost in its fears complexity.\
            Then, when they wander and dazed into the center, he laughs and\
            laughs at them-so hard, that he laughs them to death!',
    'Achilles: Oh, no!',
    "Tortoise: But it's only a myth. Courage, Achilles.",
  ],
  [{ id: 2 }, { id: 1 }, { id: 3 }, { id: 4 }, { id: 5 }],
  embeddings,
  {
    collectionName : 'test',
    vectorField: 'vectors',
    url: address
  }
);

const response = await vectorStore.similaritySearch('scared', 2);

console.log('response', response);

Steps to reproduce:

1. Download LLM
2. Run the file

Milvus-node-sdk version:

  "dependencies": {
    "@zilliz/milvus2-sdk-node": "^2.3.5",
    "langchain": "^0.0.208",
    "node-llama-cpp": "^2.8.2",
    "pdf-parse": "^1.1.1"
  }

Milvus version:

    image: milvusdb/milvus:v2.3.3
    command: ["milvus", "run", "standalone"]

If you have any advice to understand how things are done here.
Thanks in advance.
Cyrille.

Describe the bug:
{
"error_code": "UnexpectedError",
"reason": "type param(max_length) should be specified for varChar field of collection AxVPo4A4QVqxx"
}
Steps to reproduce:
我的数据如下但是报错了
{
name: "noteId",
is_primary_key: true,
description: "",
data_type: DataType.VarChar,
type_params: {
max_length: "32"
}
}

Milvus-node-sdk version:
2.2.4
Milvus version:
我用的是zilliz云

Describe the feature:

1. create collection
2. search
3. create index

Now we need to pass param like:

{
    key: "index_type",
    value: "IVF_FLAT",
  },
  {
    key: "metric_type",
    value: "L2",
  },
  {
    key: "params",
    value: JSON.stringify({ nlist: 1024 }),
  },

Need to support like:

{index_type: "IVF_FLAT","metric_type":"L2", "params": JSON.stringify({ nlist: 1024 })}

It will be a breaking change.

Describe the bug:
l am trying to use milvus in nextjs:14.0.4,when i use pnpm dev,it's ok,

but when i try to use pnpm build,then pnpm start,it give me a error message,
Error: ENOENT: no such file or directory, open '/admin/.next/server/app/api/proto/proto/schema.proto.
my project directory structure is src/app/api/[methos]/route.ts

Steps to reproduce:

1.pnpm install @zilliz/milvus2-sdk-node
pnpm install copy-webpack-plugin
2.modify next.config.js

const path = require('path');
  const CopyWebpackPlugin = require('copy-webpack-plugin');
  
  /** @type {import('next').NextConfig} */
  const nextConfig = {
    webpack: (config, { isServer }) => {
      if (isServer) {
        // Copy the proto files to the server build directory
        config.plugins.push(
          new CopyWebpackPlugin({
            patterns: [
              {
                from: path.join(
                  __dirname,
                  'node_modules/@zilliz/milvus2-sdk-node/dist'
                ),
                to: path.join(__dirname, '.next'),
              },
            ],
          })
        );
      }
      // Important: return the modified config
      return config;
    },
  };
module.exports = nextConfig;

3.test script is

import { MilvusClient } from "@zilliz/milvus2-sdk-node";

import { NextRequest } from "next/server";

async function getData() {
  const milvusClient = new MilvusClient({
    // protoFilePath: {
    //   milvus:
    //     "/proto/proto/milvus.proto",
    //   schema:
    //     "/proto/proto/schema.proto",
    // },
    address: "192.168.1.5:19530",
    username: "root",
    password: "Milvus",
  });

  let res: any = await milvusClient.getMetric({
    request: { metric_type: "system_info" },
  });

  const result = res.response.nodes_info.map((v: any) => {
    return v.infos;
  });

  return result;
}

export async function POST(
  req: NextRequest,
  {
    params,
  }: {
    params: {
      action: "test";
    };
  }
) {
  if (params.action === "test") {
    const res = await getData();
    return Response.json(res);
  }
  return Response.json({ error: "Not Found" });
}

4.pnpm dev,it's correct

[
    {
        "has_error": false,
        "error_reason": "",
        "name": "querynode3",
        "hardware_infos": {
            "ip": "172.17.0.2:21123",
            "cpu_core_count": 4,
            "cpu_core_usage": 100,
            "memory": 8231878656,
            "memory_usage": 225267712,
            "disk": 104857600,
            "disk_usage": 2097152
        },
        "system_info": {
....

4.pnpm build& pnpm start,a error has occurred

⨯ Error: ENOENT: no such file or directory, open '/Users/normanyang/Documents/workspace/nextjs/dam/admin/.next/server/app/api/proto/proto/schema.proto' at Object.openSync (node:fs:581:18) at Object.readFileSync (node:fs:457:35)

As you can see,i try to specify the address of proto.It's not useful either

Milvus-node-sdk version:
2.3.5
Milvus version:
2.3.5

Describe the feature:
Milvus now supports specify Consistency Level while creating collection and performing search and query. Please add this feature to Node.js SDk
Describe a specific use case for the feature:

pass data like [{[field_name]: vectorData}]

Hello!

While using milvus-sdk-node, I'm frequently running into the grpc issue mentioned here.

It looks like there's a workaround to the problem by overriding the default grpc channelOptions. Are these somehow exposed by the Milvus SDK?

Thank you!

Describe the bug:
milvusClient.delete() can not delete entities when the primary key is string.

Steps to reproduce:

index.ts

import { MilvusClient, InsertReq, DataType } from '@zilliz/milvus2-sdk-node';

const COLLECTION_NAME = 'foobar';

(async () => {
  // build client
  const milvusClient = new MilvusClient({
    address: 'localhost:19530',
    logLevel: 'debug',
  });

  console.log('Node client is initialized.');
  // create collection
  const create = await milvusClient.createCollection({
    collection_name: COLLECTION_NAME,
    fields: [
      {
        name: 'id',
        data_type: DataType.VarChar,
        is_primary_key: true,
        autoID: false,
        type_params: {
          max_length: 1,
        },
      },
      {
        name: 'vector',
        data_type: DataType.FloatVector,
        type_params: {
          dim: 8,
        },
      },
    ],
  });
  // create index
  const createIndex = await milvusClient.createIndex({
    collection_name: COLLECTION_NAME,
    field_name: 'vector',
    metric_type: 'L2',
  });
  console.log('Create collection is finished.', create);

  // build example data
  const vectorsData = [
    {
      vector: [
        0.11878310581111173, 0.9694947902934701, 0.16443679307243175,
        0.5484226189097237, 0.9839246709011924, 0.5178387104937776,
        0.8716926129208069, 0.5616972243831446,
      ],
      id: 'a',
    },
    {
      vector: [
        0.11878310581111173, 0.9694947902934701, 0.16443679307243175,
        0.5484226189097237, 0.9839246709011924, 0.5178387104937776,
        0.8716926129208069, 0.5616972243831446,
      ],
      id: 'b',
    },
  ];
  const params: InsertReq = {
    collection_name: COLLECTION_NAME,
    fields_data: vectorsData,
  };
  // insert data into collection
  await milvusClient.insert(params);
  await milvusClient.flush({ collection_names: [COLLECTION_NAME] });
  console.log('Data is inserted.');

  // delete data from collection
  milvusClient.delete({
    collection_name: 'foobar',
    ids: ['a', 'b'],
  });
})();

run ts-node index.ts
get error:

[2023-08-01T11:26:59.793Z] Milvus-sdk-node debug: [DB:default:Delete] executed in 5ms, returns {"succ_index":[],"err_index":[],"status":{"error_code":"UnexpectedError","reason":"failed to create expr plan, expr = id in [a,b]"},"IDs":null,"acknowledged":false,"insert_cnt":"0","delete_cnt":"0","upsert_cnt":"0","timestamp":"0"}

Milvus-node-sdk version:
2.2.21

Milvus version:
v2.2.12

Describe the bug:
Following the README code, the await client.createIndex({... fails with an invalid index params error.

Steps to reproduce:

1. follow the whole README
2. run the code

the createIndex returns this:

      {
        error_code: 'UnexpectedError',
        reason: 'invalid index params: map[dim:128 index_type:HNSW metric_type:L2]',
        code: 0
      }

And then the following command, client.loadCollectionSync({collection_name}) fails with:

    ErrorCode: UnexpectedError. Reason: index doesn't exist, collectionID 441604979289427265

Milvus-node-sdk version:
2.2.10

Milvus version:
v2.2.8

1. How to capture when new MilvusClient(addr) fails?
2. The document lists some functions that have async, some are not, but basically all the return values of the functions in the API manual are promises, how to understand ?
3. Currently, I call all async functions (such createCollection/hasCollection/insert etc.), they will be blocked. That is, neither the then nor the catch responds. No matter how long you wait, what could be the reason?

Thanks ^_^

1. inMemory changed to loaded
2. MilvusNode changed to MilvusClient

Describe the feature:

1. Installation / support milvus version
2. List all the api we have with some description
3. List classic operations to example files
4. License
   Describe a specific use case for the feature:

Describe the bug:
Langchainjs nextjs application Milvus.fromExistingCollition request is failing due to below error. compiled milvus-sdk-node is failing

Steps to reproduce:

1. Install latest version on LangChainjs 0.0.83
2. Dockerize as Langchainjs nextjs application
3. Error thrown in following line

const vectorStoreMilvus = await Milvus.fromExistingCollection(
      new OpenAIEmbeddings({}),
      {
        collectionName:  <>
      }
    );

Error :

error Error: unresolvable extensions: 'extend google.protobuf.FileOptions' in .milvus.proto.milvus
    at Root.resolveAll (/app/node_modules/protobufjs/src/root.js:256:15)
    at loadProtosWithOptionsSync (/app/node_modules/@grpc/proto-loader/build/src/util.js:68:16)
    at loadSync (/app/node_modules/@grpc/proto-loader/build/src/index.js:197:61)
    at getGRPCService (/app/node_modules/@zilliz/milvus2-sdk-node/dist/utils/Grpc.js:26:57)
    at MilvusClient.BaseClient (/app/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/BaseClient.js:58:56)
    at MilvusClient.Collection [as constructor] (/app/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/Collection.js:87:42)
    at MilvusClient.Data [as constructor] (/app/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/Data.js:81:47)
    at MilvusClient.Index [as constructor] (/app/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/MilvusIndex.js:72:42)
    at MilvusClient.Partition [as constructor] (/app/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/Partition.js:61:42)
    at MilvusClient.Resource [as constructor] (/app/node_modules/@zilliz/milvus2-sdk-node/dist/milvus/Resource.js:61:42)

Milvus-node-sdk version:

Milvus version:

e.g. [{id: '1', score: '2', age: '3'}]

Describe the feature:

1. Before merge into main branch, need run test.
2. Use build files to test

Describe the bug:

Fail to run this sample with error messages: E Connection to dns:in03-02861d92d812d64.api.gcp-us-west1.zillizcloud.com at 34.160.220.160:443 rejected by server because of excess pings.

Steps to reproduce:

Milvus-node-sdk version: 2.2.23

Milvus version: Ziliz Cloud Serverless (Free)

htps://milvus.io/docs/modify_collection.md

Describe the feature:
We may need to know when milvus flush collection success.

1. Flush collection
2. Use GetPersistentSegmentInfo to get all segments flush state. All flushed means this collection flushed.

The node sdk is missing the ignore_growing flag on searches that allows the search to bypass unindexed data. This is vital for many use cases.

Describe the bug:
I am using "@zilliz/milvus2-sdk-node": "^2.3.2" in a NestJS app. The problem is that when milvus-standalone is not running, instantiation of MilvusClient throws an error and the error is not catchable using try-catch block.

The following is my code:

try {
      this.client = new MilvusClient('localhost:19530');
} catch (error) {
      console.error('Failed to connect to Milvus:', error);
}

Steps to reproduce:

1. Stop milvus-standalone docker container.
2. Instantiate MilvusClient.

Milvus-node-sdk version:
^2.3.2

Milvus version:
milvusdb/milvus:v2.2.9

Delete is not working, Delete entities not working, on flip side all these functions works in python api, i am using a custom primary int id , it seems this can be the issue, my current total records are around 65K in the collection

Steps to reproduce:

1. add a custom primary key
2. add records try deleting from node response is success but no records are deleted

Milvus-node-sdk version:
2.2.20
Milvus version:
2.2.11

https://github.com/milvus-io/milvus-sdk-node/blob/df556b9aee270c7ccb57db3dfcbd8258d4cb999e/milvus/grpc/Data.ts#L506C13-L506C42

Describe the feature:

Describe a specific use case for the feature:
Then we can code like:
await milvusClient.createIndex({
collection_name: COLLECTION_NAME,
field_name: VECTOR_FIELD_NAME,
extra_params: {
index_type: IndexType.BIN_IVF_FLAT,
metric_type: 'TANIMOTO',
params: JSON.stringify({ nlist: 1024 }),
},
});