in memory issue and does block cache data has lifetime ? about rocksdb

[ajkr]

A block stays in block cache until evicted by LRU.

From the LOG:

"raw_key_size": 901728 ... "raw_value_size": 1561798

So the uncompressed data size is only 1561798+901728 =~ 2.5MB. Block cache should hold all the blocks no matter its capacity is 8MB or 100MB.

What is the read performance issue? How much QPS are you trying to achieve?

[Athi2019rit]

A block stays in block cache until evicted by LRU.

So even the expired entries from sst files are removed during compaction. The cache still holds the value until it evicts by the LRU

What is the read performance issue? How much QPS are you trying to achieve?

In my use case, I didn't mean QPS but collection rate before and after using rocksdb.
Collection rate before rocksdb - 4000 per second (average).
Collection rate after rocksdb - 3800 per second (average).

Yes there might be difference in before and after collection rate because of doing some additional operation in after collection.
But reading from a unordered_map instead of rocksdb's LRU cache seems improves the after collection rate by +100.

So the uncompressed data size is only 1561798+901728 =~ 2.5MB. Block cache should hold all the blocks no matter its capacity is 8MB or 100MB.

Even the uncompressed data size is ~ 2.5 mb. Is there any operation or blocks use additional in-memory, because while using rocksdb I've noted that memory increases by ~ 15-20 mb.

Why turning on the cache_index_and_filter_blocks slows down the performance ?

[ajkr]

So even the expired entries from sst files are removed during compaction. The cache still holds the value until it evicts by the LRU

Right, blocks from deleted files remain in block cache until LRU evicts them

But reading from a unordered_map instead of rocksdb's LRU cache seems improves the after collection rate by +100.

SST file lookups are not particularly fast. You could try row cache to bypass SST file lookups. It'll use more memory though.

Even the uncompressed data size is ~ 2.5 mb. Is there any operation or blocks use additional in-memory, because while using rocksdb I've noted that memory increases by ~ 15-20 mb.

Related to your earlier question, the 2.5MB could be amplified if the data was compacted and the block cache still contains blocks from the deleted files. There's other uses, like memtable. I would hope that memtable does not consume much memory when it is empty, but can't promise, especially considering the version is 4.11. A profile could tell us more definitively.

Why turning on the cache_index_and_filter_blocks slows down the performance ?

Yes it's counterintuitive. It's because the index and filter blocks are held in memory either way. In case cache_index_and_filter_blocks=false they are held in table reader memory. When cache_index_and_filter_blocks=true they are held in block cache memory. Accessing them on table reader is cheaper because it doesn't require locking to manage an LRU list.

[Athi2019rit]

SST file lookups are not particularly fast. You could try row cache to bypass SST file lookups. It'll use more memory though.

Yes, but read happens from block cache right ? why there is a slowness in that .

Related to your earlier question, the 2.5MB could be amplified if the data was compacted and the block cache still contains blocks from the deleted files. There's other uses, like memtable. I would hope that memtable does not consume much memory when it is empty, but can't promise, especially considering the version is 4.11. A profile could tell us more definitively.

Does this means that the raw key and raw value size of sst files differs from the block cache key value size ? if it is, why it is happening ? Does the index happens in-memory ? In my case i do compaction for every 30 mins the above said memory increment is noted less than 30 mins.

[ajkr]

Yes, but read happens from block cache right ? why there is a slowness in that .

The layout of a sorted string table isn't the best for random lookups. Even if the sorted string table is entirely in memory, which it is in your case, an exact-match lookup usually won't be as fast as an std::unordered_map lookup. You can run a CPU profile on the lookup process to compare different approaches.

Another feature you could consider to try closing the perf gap with std::unordered_map is data block hash index:

rocksdb/include/rocksdb/table.h

Line 242 in a036525

kDataBlockBinaryAndHash = 1, // additional hash index

Does this means that the raw key and raw value size of sst files differs from the block cache key value size ? if it is, why it is happening ? Does the index happens in-memory ? In my case i do compaction for every 30 mins the above said memory increment is noted less than 30 mins.

Are you able to provide a heap profile? It sounds like we ruled out my last guess (data blocks surviving in block cache after their files are deleted). A profile might help us get to an answer faster than guess and check