JNI ColumnFamilyHandle#getDescriptor method leaks memory about rocksdb

[adamretter]

I am just looking at the code, and unless I have missed it, I can't see any heap allocations taking place:

The JNI code from java/rocksjni/columnfamilyhandle.cc:

jobject Java_org_rocksdb_ColumnFamilyHandle_getDescriptor(JNIEnv* env,
                                                          jobject /*jobj*/,
                                                          jlong jhandle) {
  auto* cfh = reinterpret_cast<ROCKSDB_NAMESPACE::ColumnFamilyHandle*>(jhandle);
  ROCKSDB_NAMESPACE::ColumnFamilyDescriptor desc;
  ROCKSDB_NAMESPACE::Status s = cfh->GetDescriptor(&desc);
  if (s.ok()) {
    return ROCKSDB_NAMESPACE::ColumnFamilyDescriptorJni::construct(env, &desc);
  } else {
    ROCKSDB_NAMESPACE::RocksDBExceptionJni::ThrowNew(env, s);
    return nullptr;
  }
}

The underlying C++ API code from db/column_family.cc:

Status ColumnFamilyHandleImpl::GetDescriptor(ColumnFamilyDescriptor* desc) {
  // accessing mutable cf-options requires db mutex.
  InstrumentedMutexLock l(mutex_);
  *desc = ColumnFamilyDescriptor(cfd()->GetName(), cfd()->GetLatestCFOptions());
  return Status::OK();
}

If there are no heap allocations in C++ then I don't think there is anything that we need to free explicitly. What do you think?

[nicktelford]

Unless I'm mistaken, this invokes the copy-constructor of ColumnFamilyDescriptor:

*desc = ColumnFamilyDescriptor(cfd()->GetName(), cfd()->GetLatestCFOptions());

[adamretter]

this invokes the copy-constructor

Yes, as I understand it that is the case. However, isn't that copy on the Stack (and not the Heap), and so it will be deleted implicitly when it goes out of scope?

[nicktelford]

Good point, although it looks like the ColumnFamilyOptions within that descriptor is copied into the Java ColumnFamilyDescriptor, here:

  static jobject construct(JNIEnv* env, ColumnFamilyDescriptor* cfd) {
    jbyteArray jcf_name = JniUtil::copyBytes(env, cfd->name);
    jobject cfopts = ColumnFamilyOptionsJni::construct(env, &(cfd->options));

My guess is that the "leak" I'm seeing is really this ColumnFamilyOptions, in which case, whenever we call ColumnFamilyHandle#getDescriptor() to generate a ColumnFamilyDescriptor, we always need to call columnFamilyDescriptor.getOptions().close() to free the options after use?

[adamretter]

@nicktelford Good spot!
I can't see a great solution to this apart from calling columnFamilyDescriptor.getOptions().close() at the moment.

I can see a few possible options:

1. We could perhaps have ColumnFamilyDescriptor extend AutoCloseable and implement ColumnFamilyDescriptor#close() as:

@Override
public void close() {
  columnFamilyOptions_.close();
}

2. We refactor ColumnFamilyDescriptor in RocksJava so that it is also a RocksObject that wraps a C++ ColumnFamilyDescriptor. Previously we tried to keep it simple as in the past it only had a columnFamilyName.

[nicktelford]

I can see a few possible options:

1. We could perhaps have ColumnFamilyDescriptor extend AutoCloseable and implement ColumnFamilyDescriptor#close() as:

@Override
public void close() {
  columnFamilyOptions_.close();
}

This seems reasonable. It doesn't seem likely that anyone would explicitly want the lifetime of the options to exceed the descriptor.

At the very least a comment in the JavaDoc would be better than the status quo. 😄

[adamretter]

This seems reasonable

Yes, but it may be more tricky than that, I will need to find some time to try it out...

At the very least a comment in the JavaDoc would be better than the status quo.

Please feel free to send a PR for that

[adamretter]

2. We refactor ColumnFamilyDescriptor in RocksJava so that it is also a RocksObject that wraps a C++ ColumnFamilyDescriptor. Previously we tried to keep it simple as in the past it only had a columnFamilyName.

I think we should attempt this approach initially!