inlinedio / ikv-store Goto Github PK

2024-04-12 22:59:58 UTC ERROR rdkafka::client librdkafka: Global error: PartitionEOF (Broker: No more messages): Fetch from broker 2 reached end of partition at offset 1476 (HighwaterMark 1476)

https://docs.inlined.io/clients/

A basic python usage example is present in the github readme file. Docs website needs more detailed docs

Hi!

I collect materials about applying Profile-Guided Optimization (PGO) to different applications in different software domains - all my results are available at https://github.com/zamazan4ik/awesome-pgo . According to my tests, PGO helps with achieving better performance in different areas, including many databases.

I think it will be interesting to see PGO results for IKV in benchmarks since according to my tests, even Redis performance can be improved with PGO (the results are available in the repo above).

Please do not treat the issue as a bug - it's an improvement idea.

Ikv go client only supports single get in IKVReader - ie a user can fetch one field from one document.
We should add the following batch/multi versions for each field type -

• Multiple primary keys, one field-name
• One primary key, multiple field-names

Tip: Minimize object allocation in the returned objects (ex. avoid maps)

Reference (in java client)
https://github.com/inlinedio/ikv-java-client/blob/32c10fc40e9397c7e6ced771a5768e6436244ef5/src/main/java/io/inlined/clients/DefaultInlineKVReader.java#L135

👋, I found this project from the github trending. It's seems a cool project.

I noticed that README said ikv support all public storage services but the code only has native s3 support. Do you have interest to integrate with opendal for more storage services support?

OpenDAL is a unified data access layer, empowering users to seamlessly and efficiently retrieve data from diverse storage services.

I'm the PMC Chair of this project. Please let me know if I can provide help.

Go client only supports reading/writing string and bytes fields.
https://docs.inlined.io/data-modeling#fields

We should add support for numeric types (both in writer and reader clients)
https://github.com/inlinedio/ikv-store/blob/master/ikv-go-client/client.go#L10

Compaction needs to be done to implement "hard" deletes (ex. trimming of memory mapped files) during IKV index builds

https://github.com/brianfrankcooper/YCSB/wiki/Adding-a-Database

YCSB is a widely used standard benchmarking tool. I see a RocksDB binding for it, we should be able to test IKV with it (and compare with client-server style DBs like Redis)