Many empty memtables are created during MyRocks create index about rocksdb

[rockeet]

@rockeet would like to get started in contributing to rocksdb. Shall I assign this to myself?

Sure, thanks for your contribution!

[pranavshenoy]

@rockeet, I was going through the code. One thing I noticed was, isEmpty() check is absent before calling SwitchMemtable() in FlushMemTable() while rest all invocations of SwitchMemtable() has the check .(

rocksdb/db/db_impl/db_impl_compaction_flush.cc

Line 2344 in 1de6940

s = SwitchMemtable(cfd_stats, &context);

)

Let me know if I am going in the right direction. Thanks!

[pranavshenoy]

@rockeet would like to get started in contributing to rocksdb. Shall I assign this to myself?

[pranavshenoy]

@rockeet Can you tell me how can I reproduce this issue?

[rockeet]

@rockeet Can you tell me how can I reproduce this issue?

In MyRocks, create many indexes in parallel on existing large table while heavily writing to another table.

[rockeet]

PR #12230 did not fix the issue, we applied the PR to our branch, the issue still exists:

The gray text is empty MemTable.

[cbi42]

PR #12230 did not fix the issue, we applied the PR to our branch, the issue still exists:

The gray text is empty MemTable.

Are they still created in SwitchMemtable?

[pranavshenoy]

@rockeet I am trying to repro within RocksDB by writing a simple program. My intention is to check if Ingestion + write load is calling IngestExternalFiles -> FlushMemTable -> SwitchMemtable and also verify if at any time the memtable size is 0. I am having a hard time trying to reproing the issue. Am I missing something from trying to repro in RocksDB?

I am not familiar with the MyRocks setup and functionalities. It would be great if you could share me your setup or the steps that you followed.
Thanks!

[rockeet]

@rockeet I am trying to repro within RocksDB by writing a simple program. My intention is to check if Ingestion + write load is calling IngestExternalFiles -> FlushMemTable -> SwitchMemtable and also verify if at any time the memtable size is 0. I am having a hard time trying to reproing the issue. Am I missing something from trying to repro in RocksDB?

I am not familiar with the MyRocks setup and functionalities. It would be great if you could share me your setup or the steps that you followed. Thanks!

Your setup missed long lived iterators scanning the db while writting db.

[pranavshenoy]

Got it. missed this part. let me try adding it.