DML statements over reverse-ordered CFs are very slow after #86. about mysql-5.6

[hermanlee]

Comment by spetrunia
Friday Sep 04, 2015 at 20:50 GMT

---

Possible solutions:

1. Fix facebook/rocksdb#616. This might be complex.
2. We only care about the scenario where SQL layer does Deletes/Updates on the fly. This means, SQL layer will request to delete the row that was just read. Rdb_transaction could catch the Delete operation and before the iterator is invalidated, advance it so it is no longer pointing at the row that's about to be deleted.
3. A storage engine may implement Bulk Update/Delete API. This is not a full solution because there are cases where bulk update/delete is not used.

[hermanlee]

Comment by yoshinorim
Saturday Sep 05, 2015 at 02:09 GMT

---

@spetrunia : According to Siying, it is safe to mutable WriteBatchWithIndex while iterating it if using in a right way (the most important thing is to operate on a copy of the key when changing the state). Here is a unit test to verify it -- https://github.com/facebook/rocksdb/blob/3.13.fb/utilities/write_batch_with_index/write_batch_with_index_test.cc#L1247-L1366

[hermanlee]

Comment by siying
Tuesday Sep 08, 2015 at 17:40 GMT

---

@igorcanadi added a test case for mutating WriteBatchWithIndex: https://reviews.facebook.net/D39501

So you should be able to add/remove keys from it. You need to be careful about the way to do it. For example, the way will not work:

rocksdb::Slice rowkey= iter->key();
trx->Delete(rowkey);  // (*)

iter->Prev();  // (**)

rowkey is a reference to memory location. In our implementation issuing Delete() with it will cause problem as it changes in the middle of deletion and cause wrong results. But if you do:

std::string rowkey= iter->key(); 
trx->Delete(rowkey);  // (*)

iter->Prev();  // (**)

It will work.

[hermanlee]

Comment by spetrunia
Tuesday Sep 08, 2015 at 17:58 GMT

---

@yoshinorim @siying thanks for clarification. My example was overly simplified, in the actual code MyRocks copies away the key/value it has got from the iterator.

[hermanlee]

Comment by spetrunia
Tuesday Sep 08, 2015 at 17:59 GMT

---

https://reviews.facebook.net/D45873 (Issue #86 patch) is now updated to make use of this

[yoshinorim]

@spetrunia I think this is no longer an issue and #70 is tracking the rest performance issue with large data sets. Could you close this issue?