ZSTD_TrainDictionary runs even when the compression is set to kNoCompression for a given level about rocksdb

[kwadhwa18]

I have attempted a fix #12420. PTAL!

[ajkr]

Another case is max_dict_bytes > 0 will build a dictionary even when compression type is kNoCompression. These sound like good sanitizations to add. Would you be interested in adding some of them?

[kwadhwa18]

are you referring to

rocksdb/table/block_based/block_based_table_builder.cc

Lines 1874 to 1887 in 210c8df

	for (size_t i = 0;
	i < kNumBlocksBuffered && compression_dict_samples.size() < kSampleBytes;
	++i) {
	size_t copy_len = std::min(kSampleBytes - compression_dict_samples.size(),
	r->data_block_buffers[buffer_idx].size());
	compression_dict_samples.append(r->data_block_buffers[buffer_idx], 0,
	copy_len);
	compression_dict_sample_lens.emplace_back(copy_len);
	buffer_idx += kPrimeGeneratorRemainder;
	if (buffer_idx >= kNumBlocksBuffered) {
	buffer_idx -= kNumBlocksBuffered;
	}
	}

?

I can help with the sanitizations - is the level information available inside BlockBasedTableBuilder?

[ajkr]

are you referring to

rocksdb/table/block_based/block_based_table_builder.cc

Lines 1874 to 1887 in 210c8df

	for (size_t i = 0;
	i < kNumBlocksBuffered && compression_dict_samples.size() < kSampleBytes;
	++i) {
	size_t copy_len = std::min(kSampleBytes - compression_dict_samples.size(),
	r->data_block_buffers[buffer_idx].size());
	compression_dict_samples.append(r->data_block_buffers[buffer_idx], 0,
	copy_len);
	compression_dict_sample_lens.emplace_back(copy_len);
	buffer_idx += kPrimeGeneratorRemainder;
	if (buffer_idx >= kNumBlocksBuffered) {
	buffer_idx -= kNumBlocksBuffered;
	}
	}

?

Yes.

I can help with the sanitizations - is the level information available inside BlockBasedTableBuilder?

Yes , BlockBasedTableBuilder::Rep has compression_type and compression_opts, which are the settings specific to the level for which the table is being built

edit: Technically the answer to your question is no, but my point is BlockBasedTableBuilder::Rep has everything you need without it