哪里可以找到新引擎floyd设计中提到的TTL信息存储在BlobIndex中的改动 about pika

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Title: Where can I find the changes in how TTL information is stored in BlobIndex mentioned in the new engine floyd design?

I saw that the TTL information mentioned in the new engine floyd design is stored in BlobIndex, but I looked through the code of the main branch and didn't seem to find it.
This should maintain a private rocksdb? Where can I find the relevant implementation, thank you!

[mzygQAQ]

@AlexStocks hi, alex, could u help me reslove this, thanks !

[AlexStocks]

你能把原始文档贴下吗？

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Can you post the original document?

[mzygQAQ]

你能把原始文档贴下吗？

#2052
我是在这里面看到的：

blobdb使用优化
RocksDB支持了key-value分离的实现，即通过将大value存储到blob文件中，在sst文件中存储大value在blob文件的索引信息，从而减少写写放大，有效提升大value场景下的写入性能。pika依赖自定义的compactionFilter实现过期数据的处理，ttl存储在value中，因此在compaction过程中不可避免导致额外的blob文件IO。一种方法是修改sst文件中存储的blobindex，在blobindex的相同offset位置存储value的ttl值，这样compaction过程中对过期数据的清理的逻辑，就不需要查询blob文件，减少额外的磁盘IO。对于非string类型的数据，通过实现FilterByBlobKey的方式，省掉额外的BLOB文件IO开销。

[dingxiaoshuai123]

你能把原始文档贴下吗？

#2052 我是在这里面看到的：

blobdb使用优化 RocksDB支持了key-value分离的实现，即通过将大value存储到blob文件中，在sst文件中存储大value在blob文件的索引信息，从而减少写写放大，有效提升大value场景下的写入性能。pika依赖自定义的compactionFilter实现过期数据的处理，ttl存储在value中，因此在compaction过程中不可避免导致额外的blob文件IO。一种方法是修改sst文件中存储的blobindex，在blobindex的相同offset位置存储value的ttl值，这样compaction过程中对过期数据的清理的逻辑，就不需要查询blob文件，减少额外的磁盘IO。对于非string类型的数据，通过实现FilterByBlobKey的方式，省掉额外的BLOB文件IO开销。

您好，考虑到 rocksdb 之后可能升级的问题， 所以这个 pr 当时提交了之后并没有 merge 进去。以下是 pr 地址https://github.com/pikiwidb/rocksdb/pull/2/files

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Can you post the original document?

#2052 I saw it here:

blobdb usage optimization RocksDB supports the implementation of key-value separation, that is, by storing large values ​​​​in blob files and storing the index information of large values ​​​​in blob files in sst files, thereby reducing write amplification and effectively improving large value scenarios. write performance below. Pika relies on a custom compaction filter to process expired data, and ttl is stored in value, so additional blob file IO is inevitably caused during the compaction process. One method is to modify the blobindex stored in the sst file and store the ttl value of the value at the same offset position of the blobindex. In this way, the logic of cleaning up expired data during the compaction process does not need to query the blob file and reduces additional disk IO. For non-string type data, additional BLOB file IO overhead is eliminated by implementing FilterByBlobKey.

Hello, considering the possible upgrade of rocksdb later, this PR was not merged after it was submitted. The following is the PR address https://github.com/pikiwidb/rocksdb/pull/2/files

[mzygQAQ]

你能把原始文档贴下吗？

#2052 我是在这里面看到的：
blobdb使用优化 RocksDB支持了key-value分离的实现，即通过将大value存储到blob文件中，在sst文件中存储大value在blob文件的索引信息，从而减少写写放大，有效提升大value场景下的写入性能。pika依赖自定义的compactionFilter实现过期数据的处理，ttl存储在value中，因此在compaction过程中不可避免导致额外的blob文件IO。一种方法是修改sst文件中存储的blobindex，在blobindex的相同offset位置存储value的ttl值，这样compaction过程中对过期数据的清理的逻辑，就不需要查询blob文件，减少额外的磁盘IO。对于非string类型的数据，通过实现FilterByBlobKey的方式，省掉额外的BLOB文件IO开销。

您好，考虑到 rocksdb 之后可能升级的问题， 所以这个 pr 当时提交了之后并没有 merge 进去。以下是 pr 地址https://github.com/pikiwidb/rocksdb/pull/2/files

好的，多谢老哥！

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Can you post the original document?

#2052 I saw it here:
blobdb usage optimization RocksDB supports the implementation of key-value separation, that is, by storing large values ​​in blob files and storing the index information of large values ​​in blob files in sst files, thereby reducing write amplification and effectively improving large values. Write performance under the scenario. Pika relies on a custom compaction filter to process expired data, and ttl is stored in value, so additional blob file IO is inevitably caused during the compaction process. One method is to modify the blobindex stored in the sst file and store the ttl value of value at the same offset position of the blobindex. In this way, the logic of cleaning up expired data during the compaction process does not need to query the blob file and reduces additional disk IO. For non-string type data, additional BLOB file IO overhead is eliminated by implementing FilterByBlobKey.

Hello, considering the possible upgrade of rocksdb later, this PR was not merged after it was submitted. The following is the PR address https://github.com/pikiwidb/rocksdb/pull/2/files

Okay, thank you bro!

[AlexStocks]

@guoxiangCN 你加入 pikiwidb 微信群了吗

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

@guoxiangCN Have you joined the pikiwidb WeChat group?

[mzygQAQ]

@guoxiangCN 你加入 pikiwidb 微信群了吗

已申请

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

@guoxiangCN Have you joined the pikiwidb WeChat group?

Already applied