[Bug]: btree_multiset memory leak with ASAN enabled about abseil-cpp

[derekmauro]

I was not able to reproduce this issue, although I don't doubt there could be an issue. #388 is a similar issue. In general our code is not battle-tested with exceptions enabled.

[dfarley1]

In general our code is not battle-tested with exceptions enabled.

We don't use a ton of exceptions either, so originally btree_allocator was returning nullptr instead of throwing, but none of the btree code checks for nullptr either. Do the abseil containers generally not handle memory allocation failures?

In any case, I discovered the root cause. In btree::internal_emplace, ASAN enables a second replace_leaf_root_node() call. If we're inserting the first element into the btree and that allocation fails, we get into the broken state where size_ == 0 but root_ != EmptyNode(). When this happens, btree::clear() won't bother to free the dangling root_ and it leaks.

So there's a couple different solutions I see:

1. btree::clear() should check if (!empty() || root() != EmptyNode()) instead. This might help prevent some other similar issues, but it does add an additional branch. Gating the second check behind ASAN could work if the extra check is unbearable.
2. btree::internal_emplace() should catch std::bad_alloc and clean up the old root_ before re-throwing. This would have the benefit of being a much more targeted fix, but that means it might not catch any other similar issues.

We're probably going to go with the clear() fix in the hope that it potentially saves us from other issues.

I was not able to reproduce this issue, although I don't doubt there could be an issue.

That's a bummer, although I don't see how it could be related to our specific build. I think the Godbolt link I posted will repro it, but I can't get the leak sanitizer to turn on...

[derekmauro]

Do the abseil containers generally not handle memory allocation failures?

Like I said, this isn't well tested. Our production memory allocator is TCMalloc, which actually just aborts the program on failure.

That's a bummer, although I don't see how it could be related to our specific build. I think the Godbolt link I posted will repro it, but I can't get the leak sanitizer to turn on...

I played with this a bit because from the information you provided, something was obviously wrong. I finally got it to reproduce. A fix should be coming.

[derekmauro]

Following up here. I sent a proposed fix and unit test to the maintainers of our btree and hash containers. The feedback was that they don't want to guarantee that these containers are exception safe because this is not thoroughly tested and because maintaining exception safety may complicate code that is performance critical. We decided to explicitly document that this is not supported in e181410.

I know this isn't the outcome you hoped for and I'm a bit disappointed as well, but I can sympathize with the logic here. Thanks for your interest in Abseil.

[dfarley1]

Bummer to hear, but i'm not entirely surprised. Luckily everything seems to work fine in non-ASAN builds, allocation failures there don't appear to have any side effects as far as I can tell. Could you post the proposed fix here? We might steal it for ourselves.

I did encounter another instance of this bug in the rebalance_or_split path during insertion. I wasn't able to dig as deep or create a minimal repro, but I expect the root cause is mostly the same.

[derekmauro]

It is largely based on what you posted. The change to clear() that you mentioned: if (!empty() || root() != EmptyNode()).

Here is the test case:

#ifdef ABSL_HAVE_EXCEPTIONS

// An allocator that repeately throws `std::bad_alloc()` after the counter
// variable pointed to by `good_allocs_remaining` reaches 0.  The state lives
// outside the allocator so that copies don't interfere with the state.
template <typename T>
class ThrowAfterAllocator {
 public:
  using Allocator = std::allocator<T>;
  using AllocatorTraits = std::allocator_traits<Allocator>;
  using value_type = typename AllocatorTraits::value_type;
  using pointer = typename AllocatorTraits::pointer;
  using const_pointer = typename AllocatorTraits::const_pointer;
  using size_type = typename AllocatorTraits::size_type;
  using difference_type = typename AllocatorTraits::difference_type;

  explicit ThrowAfterAllocator(int64_t *good_allocs_remaining) noexcept
      : good_allocs_remaining_(good_allocs_remaining) {}

  template <typename U>
  ThrowAfterAllocator(const ThrowAfterAllocator<U> &x) noexcept
      : good_allocs_remaining_(x.good_allocs_remaining_) {}

  pointer allocate(
      size_type n,
      typename AllocatorTraits::const_void_pointer hint = nullptr) {
    if ((*good_allocs_remaining_)-- <= 0) {
      throw std::bad_alloc();
    }
    Allocator allocator;
    return AllocatorTraits::allocate(allocator, n, hint);
  }

  void deallocate(pointer p, size_type n) {
    Allocator allocator;
    AllocatorTraits::deallocate(allocator, p, n);
  }

  int64_t* good_allocs_remaining_;
};

TEST(Btree, ClearAfterAllocatorThrow) {
  using Alloc = ThrowAfterAllocator<int>;
  using Set = absl::btree_set<int, std::less<int>, Alloc>;
  int64_t good_allocs_remaining = 1;
  Alloc alloc(&good_allocs_remaining);
  Set set(alloc);
  try {
    set.emplace_hint(set.cend(), 42);
  } catch (const std::exception &) {
  }
  set.clear();
}

#endif  // ABSL_HAVE_EXCEPTIONS