boostorg / lockfree Goto Github PK

View Code? Open in Web Editor NEW

115.0 12.0 87.0 374 KB

Boost.Lockfree

C++ 96.62% HTML 0.29% CMake 3.09%

lockfree's People

Contributors

Stargazers

Watchers

Forkers

sanderbog erenon danieljames jianmink marcelraad chunyang-wen rdaysky ephraimo crzftx tempbottle limbolily cdglove joachim-faulhaber wolfie2x crossbuild pleuba chenbk85 htfy96 cgaebel hincoin autodesk-forks tnovotny danielae xltools rzinsly orgads brycelelbach njlr ninlar hotgloupi linux-on-ibm-z bazelboost malterlib cor3ntin myestro mike-devel liangzhou9527 cromwellenage linhanyu yuvalif buckaroo-pm jinhang2008 austin-beer danielmevi brinkqiang2cpp theworldwego zhaohui0923 sumeetb25 ashleyllew glenfe f3rm4rf3r mishatal samd2 sdarwin nirvananimbusa supercline vardhanyadav1714 opensourceeda pshoben felixpetriconi fieryswampshire longgang 5741978 msuvajac ylhstore jingliang2005 mrityunjay-tripathi eikeatot isabella232 gtton shards-lang xiovlad rlipovsky slowriot aalmos starbeat zhaoyaogit ka-de seanpm2001 sibman sam-caldwell grafikrobot infinityofficialnetwork

lockfree's Issues

boost::lockfree queues support r-values and move in push/pop/consume etc?

I can get around it right now by using std::shared_ptr, but would be nice if the push function could accept r-values and move appropriately.
Similarly the pop & consume functions could move the objects out of the queue?

Modular Boost C++ Libraries Request

We are in the process of making B2 build changes to all of the B2 build files
to support "modular" consumption of the Boost Libraries by users. See this list
post for some details: https://lists.boost.org/Archives/boost/2024/01/255704.php

The process requires making a variety of changes to make each Boost library
independent of the super-project structure. But the changes do not remove the
super-project structure or the comprehensive Boost release. The changes make
solely make it possible, optionally, for users, like package manages, to easily
consume libraries individually.

Generally the changes include:

Adding a libroot/build.jam.
Porting any functionality from libroot/jamfile to libroot/build.jam.
Moving boost-install declaration from libroot/build/jamfile is applicable.
Adjusting other B2 build files in the library, like test/jamfile, as needed.
Possible changes to C++ source files to remove includes relative to the
super-project boostroot location.

Some examples of such changes:

We are asking how you would like us to handle the changes. We would prefer if
you allow the owners of the Boost.org GitHub project to make changes to B2
build files, as needed, to accomplish the changes. But understand
that you may want to manage the proposed changes yourself.

We previously sent emails to all known maintainers to fill out a form with their
preference. We are contacting you in this issue as we have not gotten a response
to that email. You can see the ongoing responses for that form and the responses
to these issues here https://github.com/users/grafikrobot/projects/1/views/6

We are now asking if you can reply directly to this issue to indicate your
preference of handling the changes. Please supply a response to this question
and close the issue (so that we can verify you are a maintainer).

How would you like the build changes to be processed?

Pull request, reviewed and merged by a BOOSTORG OWNER.
Pull request, reviewed and merged by YOU.
Other. (please specify details in the reply)

Also please indicate any special instructions you want us to consider. Or other
information you want us to be aware of.

Thanks you, René

a new lock-free queue

In theory, it is a queue without waiting, which supports multi-threaded reading and writing. Of course, the disadvantage is that the order of entering the queue may not be strongly ordered.

template<typename T, size_t N>
class queue
{
private:
    ring_buffer<T, N> value_;

    ring_buffer<atomic<size_t>, N> readable_flag_{0};
    ring_buffer<atomic<size_t>, N> writable_flag_{0};

    atomic<size_t> writable_limit_;
    atomic<size_t> readable_limit_;
public:
    queue() = default;
    ~queue() = default;

    void push(const T & val)
    {
        size_t index = writable_limit_.fetch_add(1);

        while (writable_flag_[index] != index / max_size())
            ;
        
        value_[index] = val;
        readable_flag_[index] = (index / max_size()) + 1;
    }

    void pop(T&val)
    {
        size_t index = readable_limit_.fetch_add(1);

        while(readable_flag_[index] != (index / max_size()) + 1)
            ;

        val = value_[index];
        writable_flag_[index] = (index / max_size()) + 1;
    }

    size_t max_size() const
    {
        return N;
    }

    size_t size() const
    {
        size_t writable_limit = writable_limit_;
        size_t readable_limit = readable_limit_;

        return writable_limit > readable_limit ? writable_limit - readable_limit : 0;
    }
};

Boost CMake testing procedure fails for Lockfree

The documented testing procedure

mkdir __build && cd __build
cmake -DBUILD_TESTING=ON -DBOOST_INCLUDE_LIBRARIES=lockfree ..
cmake --build . --target tests
ctest --output-on-failure --no-tests=error

fails for Boost.Lockfree, because the test executables aren't built by the target tests. BoostTest handles this automatically, but for tests declared manually, one needs to first declare the tests target if not present

if(NOT TARGET tests)
    add_custom_target(tests)
endif()

and then for each test executable, use add_dependencies(test my_test_executable).

Or, since there is already a target boost_lockfree_all_tests, it should be enough to make it a dependency of tests: add_dependencies(tests boost_lockfree_all_tests).

(For this to keep working after the eventual fix, it might be a good idea to add it to CI.)

It's also good practice to make the test executables EXCLUDE_FOR_ALL, so that building the tests target builds them, but building the default target doesn't, but this step is not in principle required.

spsc_queue with compile_time_sized_ringbuffer does not call destructors of remaining queue items when destroyed

Unlike runtime_sized_ringbuffer, compile_time_sized_ringbuffer does not have a custom destructor, and so it does not clean up any outstanding queue items when destroyed.

I am not a lockfree guru, so I do not know if there is some hidden limitation that makes this impossible, or if it is simply a bug.
I could not find any documentation for this behavior, so I suspect the latter.

I can make a PR for the proposed fix below, if it seems correct.

Here is a small test that fails unexpectedly for me:

int g_numObjs = 0;

struct Obj {
	Obj() {
		++g_numObjs;
	}
	Obj(const Obj&) {
		++g_numObjs;
	}
	~Obj() {
		--g_numObjs;
	}
};

typedef boost::lockfree::spsc_queue<
		Obj
		// Note: commenting out this line makes it into a
		// runtime_sized_ringbuffer, and the test passes.
		,boost::lockfree::capacity<10>
	> MyQueue;

TEST_CASE("boost spsc bug") {
	{
		REQUIRE(g_numObjs == 0);
		MyQueue queue;
		queue.push(Obj());
		REQUIRE(g_numObjs == 1);
	} // queue gets destroyed

	// FAILS
	REQUIRE(g_numObjs == 0);
}

If I add a destructor to compile_time_sized_ringbuffer, such as below (copied from runtime_sized_ringbuffer), then the test passes and everything seems fine:

~compile_time_sized_ringbuffer(void)
{
    // destroy all remaining items
    T out;
    while (pop(&out, 1)) {}
}

Note: I suppose another implemenation might be to call reset() from inside ~spsc_queue() itself, which would benefit from the optimization of not calling trivial destructors. Or maybe a similar optimization would be warranted in the destructors for compile_time_sized... and runtime_sized_... as well?

Need way to shrink the freelist

Hi,everybody~

I found nodes in freelist can never be shrinked, it can only grow larger(like "reserve_unsafe" and "reserve" do).
The only way to reduce the capacity of freelist is destruct queue,ringbuffer or stack. But that is not convenient. Sorry I must make the memory occupy not so high affter some request burst.
Is it possible to improve that?
If cann't, then "capacity_unsafe" function to tell current capacity is needed.
I understand data structure in freelist implementation should lock free as much as possible. But it can have a flexible way like limit the max freelist capacity. Such as when return node to freelist, examine the current capacity. It doesn't need to be accurate,so "capacity" doesn't need to be a atmoic variable.
Just some my silly ideas~

Thanks~

change constructor (queue()) has_capacity assert to static assert

lockfree/include/boost/lockfree/queue.hpp

Line 188 in ffd91db

BOOST_ASSERT(has_capacity);

This check for construction of the queue, if the capacity has been specified as the template parameter should be moved to static assert rather than having the run time assert.

alignment change in 1.77 beta breaks VS 2017 builds

The underlying issue with the VS2017 compiler is shown here on godbolt.

The error was triggered in our code base (which works fine with boost-1.76) where we have boost::lockfree::queue in classes managed by shared_ptr. It is probably due to this commit.

Note that boost::lockfree::queue<int, boost::lockfree::capacity<N>>, which is already pretty large, increases by 60 bytes.

Of course, there is an easy fix by defining _ENABLE_EXTENDED_ALIGNED_STORAGE, but there is a reason they did not enable this by default.

This issue is obviously not a deal breaker but it will make upgrading boost from 1.76 to 1.77 troublesome for many users.

wrong predef flag usage

Working on a new boost release I found the lockfree queue stopped working proper and tracked it down: some x86_64 codes are included improperly in a couple of files, namely in boost/lockfree/detail/prefix.hpp and tagged_ptr_ptrcompression.hpp:

#ifdef BOOST_ARCH_X86_64

Should be

#if BOOST_ARCH_X86_64

Otherwise codes on all the other ARCHs will include the wrong piece. I tested the change on powerpc and it works fine so far.

[Enhancement] Allow bounded queue size to exceed 65,535 for 64bit systems

in https://github.com/boostorg/lockfree/blob/develop/include/boost/lockfree/detail/freelist.hpp lines 335 and 369, replace the numeric const with compile time const derived from 32bit vs. 64bit compilation
(e.g. #if __x86_64__ || __ppc64__ )

lockfree on armv7, bus error (core dumped)

# root at alarmpi in ~ [21:36:15]
$ cat test.cpp
#include <atomic>
#include <boost/lockfree/queue.hpp>
#include <iostream>

std::atomic_int intcount;
boost::lockfree::queue<int> intpool(100);

int main()
{
        return 0;
}





# root at alarmpi in ~ [21:36:53]
$ g++ test.cpp -std=c++11





# root at alarmpi in ~ [21:37:15]
$ ./a.out
[1]    31406 bus error (core dumped)  ./a.out





# root at alarmpi in ~ [21:37:27]
$ coredumpctl gdb 31406
           PID: 31406 (a.out)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 7 (BUS)
     Timestamp: 一 2015-04-27 21:37:27 CST (12s ago)
  Command Line: ./a.out
    Executable: /root/a.out
 Control Group: /user.slice/user-0.slice/session-c6.scope
          Unit: session-c6.scope
         Slice: user-0.slice
       Session: c6
     Owner UID: 0 (root)
       Boot ID: f2ff2834fe86475d8008560472158aa9
    Machine ID: 20d4f52b53de4dd9aa7ac72e93285b0a
      Hostname: alarmpi
      Coredump: /var/lib/systemd/coredump/core.a\x2eout.0.f2ff2834fe86475d8008560472158aa9.31406.1430141847000000.lz4
       Message: Process 31406 (a.out) of user 0 dumped core.

GNU gdb (GDB) 7.9
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "armv7l-unknown-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /root/a.out...(no debugging symbols found)...done.
[New LWP 31406]
Core was generated by `./a.out'.
Program terminated with signal SIGBUS, Bus error.
#0  0x00010ec8 in std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::freelist_node> >::load(std::memory_order) const ()
(gdb) bt
#0  0x00010ec8 in std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::freelist_node> >::load(std::memory_order) const ()
#1  0x0001110c in boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::deallocate_impl_unsafe(boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node*) ()
#2  0x00010e94 in void boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::deallocate<false>(boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node*) ()
#3  0x00010bd0 in boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::freelist_stack<std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >(std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> const&, unsigned int) ()
#4  0x00010a34 in boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::queue(unsigned int) ()
#5  0x000107e8 in __static_initialization_and_destruction_0(int, int) ()
#6  0x00010820 in _GLOBAL__sub_I_intcount ()
#7  0x000119f4 in __libc_csu_init ()
#8  0x76c22ccc in __libc_start_main () from /usr/lib/libc.so.6
#9  0x0001065c in _start ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) q






# root at alarmpi in ~ [21:37:47]
$ uname -a
Linux alarmpi 3.18.11-4-ARCH #1 SMP PREEMPT Tue Apr 21 21:52:58 MDT 2015 armv7l GNU/Linux





# root at alarmpi in ~ [21:37:50]
$ g++ -v
使用内建 specs。
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/armv7l-unknown-linux-gnueabihf/4.9.2/lto-wrapper
目标：armv7l-unknown-linux-gnueabihf
配置为：/build/gcc/src/gcc-4.9-20150304/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://github.com/archlinuxarm/PKGBUILDs/issues --enable-languages=c,c++,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-cloog-backend=isl --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --disable-multilib --disable-werror --enable-checking=release --host=armv7l-unknown-linux-gnueabihf --build=armv7l-unknown-linux-gnueabihf --with-arch=armv7-a --with-float=hard --with-fpu=vfpv3-d16
线程模型：posix
gcc 版本 4.9.2 20150304 (prerelease) (GCC)

# root at alarmpi in ~ [21:37:58]
$ /lib/libc-2.21.so
GNU C Library (GNU libc) stable release version 2.21, by Roland McGrath et al.
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.9.2 20150304 (prerelease).
Available extensions:
        crypt add-on version 2.1 by Michael Glad and others
        GNU Libidn by Simon Josefsson
        Native POSIX Threads Library by Ulrich Drepper et al
        BIND-8.2.3-T5B
libc ABIs: UNIQUE
For bug reporting instructions, please see:
<https://github.com/archlinuxarm/PKGBUILDs/issues>.

Test stack_test is LittleEndian dependant

Hi,
Running the test stack_test on AIX, which is BigEndian, I see that it fails while running the test fixed_size_stack_test_exhausted .

../libs/lockfree/test/stack_test.cpp : 
     85 BOOST_AUTO_TEST_CASE( fixed_size_stack_test_exhausted )
     86 {
     87     boost::lockfree::stack<long, boost::lockfree::capacity<2> > stk;
     88
     89     stk.push(1);
     90     stk.push(2);
     ....

The first stk.push(1) seems OK, but not really, since:

../boost/lockfree/detail/freelist.hpp:552

552                 tagged_index new_pool(next_index->get_index(), old_pool.get_next_tag());
(gdb) whatis next_index
 type = boost::lockfree::detail::tagged_index *

AIX:
  (gdb) p * next_index
  $12 = {index = 0, tag = 1}

Linux:
  (gdb) p * next_index
  $14 = {index = 0, tag = 65535}

Then, stk.push(2) does not put correct values in stk.

Main differences between Fedora/x86_64 and AIX after line stk.push(2) are:

padding is 60 on Fedora and 124 on AIX (compiled in 64bit too)
stk.pool.pool_._M_i = {index = 2, tag = 2} On Fedora but : {index = 0, tag = 0} on AIX.

On Fedora/x86_64:

stk = {static has_capacity = true, static capacity = 2, static fixed_sized = false, static node_based = false, static compile_time_sized = true,
     tos = { static _S_min_alignment = 4, static _S_alignment = 4, _M_i = {index = 0, tag = 0}},
     static padding_size = 60,
 padding = ...........,
 pool = {<boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<long, boost::lockfree::capacity<2> >::node, 2>>
      = {data = {
       elems = .......}},
         pool_ = {static _S_min_alignment = 4, static _S_alignment = 4, _M_i = {index = 2, tag = 2}}}}

On AIX:

stk = {static has_capacity = true, static capacity = 2, static fixed_sized = false, static node_based = false, static compile_time_sized = true,
      tos = { static _S_min_alignment = 4, static _S_alignment = 4, _M_i = {index = 0, tag = 0}},
      static padding_size = 124,
 padding = ........,
 pool = {<boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<long, boost::lockfree::capacity<2> >::node, 2>>
      = {data = { elems = ..........}},
         pool_ = {static_S_min_alignment = 4, static _S_alignment = 4, _M_i = {index = 0, tag = 0}}}}

However, looking at details with gdb:
Fedora/x86_64:

(gdb) p & stk.pool.pool_._M_i
  $54 = (boost::lockfree::detail::tagged_index *) 0x7fffffffc1c0
  (gdb) x/2x 0x7fffffffc1c0
  0x7fffffffc1c0: 0x00020002      0x00000000

AIX:

(gdb)  p & stk.pool.pool_._M_i
  $12 = (boost::lockfree::detail::tagged_index *) 0xfffffffffffdb08
  (gdb) x/4x 0xfffffffffffdb08
  0xfffffffffffdb08:      0x00000000      0x00000002      0x00020001      0x001baca8

So, it seems that the code does not write the 0x00020002 value at the right place on AIX.

This is probably due to the following code:

boost/lockfree/detail/freelist.hpp : 
class
BOOST_ALIGNMENT( 4 ) // workaround for bugs in MSVC
tagged_index
{
public:
   typedef boost::uint16_t tag_t;
   typedef boost::uint16_t index_t;
...
protected:
   index_t index;
   tag_t   tag;
};

since using 16bit has already shown within another library of Boost that it does not work fine in BigEndian.

Note: On Fedora/PPC64LE (PowerPC LittleEndian), this works fine.

Lockfree queue delivers data out of order

This change, 3d45c00, introduced in boost 1.71.0 to fix valgrind issues, actually introduced a very subtle ordering issue in the queue implementation which also exists in 1.72 and 1.73.

Specifically, the code previously did NOT initialize the tagged_node_handle next on construction. which works beautifully when a node is allocated/freed/allocated

-            data(v)//, next(tagged_node_handle(0, 0))
+            data(v), next(tagged_node_handle(null_handle, 0))

By initializing the next pointer, the change introduces a classic ABA problem (from Wikipedia, "when a location is read twice, has the same value for both reads, and 'value is the same' is used to indicate 'nothing has changed'. However, another thread can execute between the two reads and change the value, do other work, then change the value back, thus fooling the first thread into thinking "nothing has changed" even though the second thread did work that violates that assumption.")

Reverting that single line fixes the issue in both 1.71 and 1.73

The following program demonstrates the problem where data can be returned from the queue out of order. Note that recreating this depends on system load and system architecture, but I have reliably recreated the problem on both server and desktop class systems by repeating this test 10,000 times.

// Boost queue stress test....all credits to Wes Darvin
#include <boost/lockfree/queue.hpp>
#include <iostream>
#include <thread>
#include <vector>

static constexpr int BLOCK_SIZE = 1000;
static constexpr int NUM_THREADS = 20;

// Enqueue BLOCK_SIZE values in order
void enqueueThreadMethod(boost::lockfree::queue<int32_t> *queue, int threadID) {
    int start = BLOCK_SIZE * threadID;
    for (int i = 0; i < BLOCK_SIZE; i++) {
        while (!queue->push(start + i))
            ;
    }
}

// Dequeue BLOCK_SIZE values.  Note that the values returned may come from
// different enqueuers, but for each enqueuer they should be in increasing
// order.
void dequeueThreadMethod(
    boost::lockfree::queue<int32_t> *queue,
    std::array<int32_t, NUM_THREADS * BLOCK_SIZE> *retValues, int threadID) {
    for (int i = 0; i < BLOCK_SIZE; i++) {
        while (!queue->pop((*retValues)[threadID * BLOCK_SIZE + i]))
            ;
    }
}

int main(int argc, char **argv) {
    std::array<int32_t, NUM_THREADS * BLOCK_SIZE> retValues;

    std::vector<std::thread> threadVector;

    boost::lockfree::queue<int32_t> queue(100);

    // Enqueue and dequeue relatively simulataniously. And yes, we could make
    // this more robust by adding a barrier, but it works as-is
    for (int i = 0; i < NUM_THREADS; i++) {
        threadVector.emplace_back(&enqueueThreadMethod, &queue, i);
        threadVector.emplace_back(&dequeueThreadMethod, &queue, &retValues, i);
    }

    for (std::thread &t : threadVector) {
        t.join();
    }

    // Now validate the values we dequeued.   All we are checking is
    // that for each enqueuer, its values only increase...i.e. we
    // are checking for out-of-order conditions
    for (int i = 0; i < NUM_THREADS; i++) {
        // vector of last-seen values by enqueuer. Each
        // entry should increase
        std::vector<int> localMax(NUM_THREADS, -1);

        // check the values dequeued by a single dequeuer
        for (int j = 0; j < BLOCK_SIZE; j++) {
            int localValue = retValues[i * BLOCK_SIZE + j];
            int index = localValue / BLOCK_SIZE;

            if (localValue < localMax[index]) {
                std::cout << "Out of order values: " << localValue << " < "
                          << localMax[index] << " i: " << i << ", j: " << j
                          << ", index: " << index << std::endl;
            }
            localMax[index] = localValue;
        }
    }

    // Now sort all the dequeued values and make sure there are none missing
    // or duplicated
    std::sort(retValues.begin(), retValues.end());
    for (int i = 0; i < (BLOCK_SIZE * NUM_THREADS); i++) {
        if (retValues[i] != i) {
            std::cout << "Error checking sorted values at index " << i
                      << std::endl;
        }
    }

    return 0;
}

variable-sized queue with a custom allocator is not constructible

due to template param deduction failure in constructor declaration (line 249), see godbolt

as a quick and dirty workaround I replaced "typename boost::allocator_rebind<node_allocator, U>::type const& alloc" with plain "U const &alloc" there in C:\Project\Libs\vcpkg\installed\x64-windows-static\include\boost\lockfree\queue.hpp and that made it compile and work but I hope it's going to be fixed in the trunk

What is the use of the btiwise and in tagged_ptr_dcas::get_next_tag?

Here
https://github.com/boostorg/lockfree/blob/develop/include/boost/lockfree/detail/tagged_ptr_dcas.hpp#L107

What is the use of such bitwise and? This seems to be a no-op for me. In first place, because overflow is well-defined for size_t (wrapping). Doing x & ~(size_t)0 is always x. In second place, because even if the overflow were not defined, the bitwise and would happen after the overflow, and UB would be invoked anyway.

MacOS: Workaround for extremely slow performance creating lockfree::stack instances (99.999999% for sure a MacOS problem only)

Motivation for adding this ticket: Just for the record, in case anyone else searches for a solution to this issues.

Problem: At least on MacOS 14, creating 16 lockfree:stack instances sequentially takes more than 10 seconds(!) on an M3 macbook.

A solution:

#define NUM_POOLS 16
#define POOL boost::lockfree::stack<RT_mempool_data*>
+  std::thread t[NUM_POOLS];
+  
+  for(int i=0;i<NUM_POOLS;i++)
+  {
+         // Allocate each pool in it's own thread. Workaround for really slow memory allocation performance on Macos.
+         t[i] = std::thread([i](){        
+                 g_pools[i] = new POOL(MAX_POOL_SIZE);
+         });
+  }
+
   for(int i=0;i<NUM_POOLS;i++)
-    g_pools[i] = new POOL(MAX_POOL_SIZE);
+         t[i].join();

version 1.73 boost::lockfree::queue crash under android 12 when destruct?

version 1.73 boost::lockfree::queue crash under android 12 when destruct?
eg.
{
boost::lockfree::queue<int, boost::fixed_sized > test_que(10);
}
it does nothing else, it crash backtrace like this:
boost::lockfree::queue<int, boost::fixed_sized >::~queue()+272

the same apk in not android 12 phone, it works ok, but android 12

what happened?

lockfree queue: memory orderings are questionable, 1.68.0

The possible problems are in queue::do_push and queue::pop.
I'm not versed in the lingo of atomics and memory orderings, so my phrasing might be a bit off - but I'm quite sure my questions are correct.

(1) double loads of tail_ and head_ in do_push and pop

First let me mention that in the original algorithm by Scott and Michael these functions have a double load of tail and head (respectively), and then the values of the first and second load are compared.
The purpose of these are to make sure the node has not been freed (std::free()) already, and then the &(node->next) might become a non dereferenceable address, or else its value could be randomly changed to false-positively pass the tag check.
However in any case that the nodes are not freed but rather put in a free-list, we don’t have this problem at all.

The reason they work in the algorithm, is because the algorithm is sequentially consistent memory:

push: the node can be freed only after tail_ has been exchanged to a value further ahead in the list than the node itself. If the value is exchanged so in any thread, it would either be visible by the second load of tail_ or it would happen after it, due to sequential consistency.

pop: basically same explanation for head_

So, our queue is using weak memory ordering, therefore the above explanations do not apply.
In case the node is actually freed, rather than put in a free-list, we need some other way to make sure &(node->next) is actually still dereferenceable.
For example, make the second loads (of tail and head) a read-modify-write operation (not for the purpose of changing the value, but for the purpose of having a strong order of visibility with the actual value exchanges).

I know that after c++11, for 14/17/20 the relationships between seq_cst and weak order operations have been strengthened. Is this what is assumed here? That the seq_cst exchanges of tail_ and head_ are strongly ordered with these second loads?

(2) questionable memory ordering in do_push()

tagged_node_handle tail = tail_.load(memory_order_acquire);
node * tail_node = pool.get_pointer(tail);
tagged_node_handle next = tail_node->next.load(memory_order_acquire);
node * next_ptr = pool.get_pointer(next);
tagged_node_handle tail2 = tail_.load(memory_order_acquire);

(2.1)
So, why would the second load of tail_ be mo_acquire?!
What is it acquiring for?! There are no further loads after that.
If at all it is needed, it could just as well be mo_relaxed.

(2.2)
Why would tail_node->next.load() be mo_acquire?!
If it is acquiring for the second tail_.load(), it could only be if some other thread had a matching:
tail_.store()
tail_node->store( …, mo_release );
But this doesn’t exist anywhere.
The only node->next mo_release in the code is later in do_push() and there are no tail_.store() before it, only an tail_.exhange() after.

(3) questionable memory ordering in pop()

tagged_node_handle head = head_.load(memory_order_acquire);
node * head_ptr = pool.get_pointer(head);
tagged_node_handle tail = tail_.load(memory_order_acquire);
tagged_node_handle next = head_ptr->next.load(memory_order_acquire);
node * next_ptr = pool.get_pointer(next);
tagged_node_handle head2 = head_.load(memory_order_acquire);

(3.1)
Why would the second head_.load be mo_aquire?!
What is it acquiring for?
If at all needed, it could be mo_relaxed

(3.2)
Why does head_ptr->next.load() has mo_acquire?!
As if it’s acquiring for the second head_.load()
But there’s no matching release for this memory.
edit: However, it needs to be at least a mo_consume for the following reading of the payload.

(4) Seems to me, in order to make sense of 2.2 and 3.2 above, maybe add
head_ptr->next.store(..., mo_release )
inside pop() after the head.compare_exchange succeeded.
Then at least it would be releasing for the head exchange and also for tail exchange (because the tail exchange happens before the head exchange, in the same pop() or earlier).

(5) Why does pop() copy the payload before the head.compare_exchange every loop, instead of only once after it succeeds? This is a performance difference.

detail::copy_payload(next_ptr->data, ret);
tagged_node_handle new_head(pool.get_handle(next), head.get_next_tag());
if (head_.compare_exchange_weak(head, new_head)) {
  ...

Build failure MSVC 14 (x86) on Boost 1.68.0

Hi,

After upgrading to Boost 1.68.0, builds using lockfree fail on x86; x64 works.

Compiler version:

Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

On trying to compile the test cases, I get this error (complete log attached):

stack_bounded_stress_test.cpp
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(508): error C2719: '_Val': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(669): note: see reference to class template instantiation 'std::_Atomic_base<_Ty,8>' being compiled
        with
        [
            _Ty=boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::stack<long>::node,std::allocator<boost::lockfree::stack<long>::node>>::freelist_node>
        ]
T:\boost_1_68_0\boost/lockfree/detail/freelist.hpp(255): note: see reference to class template instantiation 'std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<T,Alloc>::freelist_node>>' being compiled
        with
        [
            T=boost::lockfree::stack<long>::node,
            Alloc=std::allocator<boost::lockfree::stack<long>::node>
        ]
T:\boost_1_68_0\boost/lockfree/stack.hpp(100): note: see reference to class template instantiation 'boost::lockfree::detail::freelist_stack<T,Alloc>' being compiled
        with
        [
            T=boost::lockfree::stack<long>::node,
            Alloc=std::allocator<boost::lockfree::stack<long>::node>
        ]
stack_bounded_stress_test.cpp(25): note: see reference to class template instantiation 'boost::lockfree::stack<long>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(54): note: see reference to class template instantiation 'boost::arg<9>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(53): note: see reference to class template instantiation 'boost::arg<8>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(52): note: see reference to class template instantiation 'boost::arg<7>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(51): note: see reference to class template instantiation 'boost::arg<6>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(50): note: see reference to class template instantiation 'boost::arg<5>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(49): note: see reference to class template instantiation 'boost::arg<4>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(48): note: see reference to class template instantiation 'boost::arg<3>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(47): note: see reference to class template instantiation 'boost::arg<2>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(46): note: see reference to class template instantiation 'boost::arg<1>' being compiled
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(519): error C2719: '_Right': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(525): error C2719: '_Right': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(542): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(548): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(580): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(587): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(594): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(602): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(610): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(617): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(624): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(632): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(640): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(647): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(680): error C2719: '_Val': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(686): error C2719: '_Right': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(691): error C2719: '_Right': formal parameter with requested alignment of 8 won't be aligned

Could you please have a look on how to fix this issue?

Complete log

boost.lockfree tagged_ptr conflict with Hardware-assisted AddressSanitizer(HWASan) from LLVM project

As we know, boost using tagged_ptr avoid CAS ABA issue. But in aarch64, tagged_ptr_ptrcompression using top bits of pointer, but HWASan also using top bits of pointer, so conflict.
HWASan help detect memory bugs in LLVM project, so should boost lockfree compat HWASan ?

x86-64 pointer tagging is unsound on most recent Intel CPUs (Ice Lake)

According to the boost::lockfree documentation and from what I can see from boost/lockfree/detail/tagged_ptr_ptrcompression.hpp, the implementation of the free-list stores a 16-bit tag value in the upper bits of a 64-bit pointer.

Most recent Intel x86_48 microarchs (Ice Lake) use 5-level page tables and 57-bit virtual addresses, meaning this technique of pointer tagging can fail unexpectedly when used on such a processor.

compilation error using tagged_ptr_dcas.hpp

this code...

#include <boost/lockfree/detail/tagged_ptr_dcas.hpp>

int main(int argc, char **argv)
{
int i = argc;
return i;
}

generate this error...

[gustavo@casa c++]$ g++ boost.cc -o boost
En el fichero incluido desde boost.cc:1:
/usr/include/boost/lockfree/detail/tagged_ptr_dcas.hpp:30:17: error: expected unqualified-id before numeric constant
30 | BOOST_ALIGNMENT(2 * sizeof(void*))
| ^
/usr/include/boost/lockfree/detail/tagged_ptr_dcas.hpp:30:17: error: expected ‘)’ before numeric constant
30 | BOOST_ALIGNMENT(2 * sizeof(void*))
| ~^
| )

Can the Queue be movable?

Hi everyone,

Is there any obligation to make the Queue movable class? If there is not, I would like to contribute.

Thanks in advance

Data race in lockfree::stack

Running my program (uses boost 1.60) through Thread Sanitizer reports the following data race:

WARNING: ThreadSanitizer: data race (pid=23760)
Write of size 2 at 0x7dc400000110 by thread T6:
#0 boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::link_nodes_atomic(boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node_, boost::lockfree::stack<MYDATA_, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node_) /.../include/boost/lockfree/stack.hpp:226 (bin+0x00000079812c)
#1 bool boost::lockfree::stack<MYDATA_, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::do_push(MYDATA* const&) /.../include/boost/lockfree/stack.hpp:314 (bin+0x000000798034)
#2 boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::bounded_push(MYDATA* const&) /.../include/boost/lockfree/stack.hpp:302 (bin+0x000000797a20)

Previous read of size 2 at 0x7dc400000110 by thread T5:
#0 boost::lockfree::detail::tagged_index::get_index() const /.../include/boost/lockfree/detail/freelist.hpp:281 (bin+0x000000797e59)
#1 boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::allocate_impl() /.../include/boost/lockfree/detail/freelist.hpp:545 (bin+0x0000007982e0)
#2 unsigned short boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::allocate() /.../include/boost/lockfree/detail/freelist.hpp:527 (bin+0x0000007981f9)
#3 boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node* boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::construct<true, true, MYDATA*>(MYDATA* const&) /.../include/boost/lockfree/detail/freelist.hpp:443 (bin+0x000000798071)
#4 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::do_push(MYDATA* const&) /.../include/boost/lockfree/stack.hpp:310 (bin+0x000000798021)
#5 boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::bounded_push(MYDATA* const&) /.../include/boost/lockfree/stack.hpp:302 (bin+0x000000797a20)

And another similar one:

WARNING: ThreadSanitizer: data race (pid=23760)
Write of size 2 at 0x7dc400001190 by thread T5:
#0 boost::lockfree::detail::tagged_index::set_index(unsigned short) /.../include/boost/lockfree/detail/freelist.hpp:286 (bin+0x000000797cdd)
#1 boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::deallocate_impl(unsigned short) /.../include/boost/lockfree/detail/freelist.hpp:585 (bin+0x000000805953)
#2 void boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::deallocate(unsigned short) /.../include/boost/lockfree/detail/freelist.hpp:573 (bin+0x0000008058d0)
#3 void boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::destruct(boost::lockfree::detail::tagged_index) /.../include/boost/lockfree/detail/freelist.hpp:471 (bin+0x00000080589f)
#4 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::consume_oneboost::lockfree::detail::consume_via_copy<MYDATA* >(boost::lockfree::detail::consume_via_copy<MYDATA*>&) /.../include/boost/lockfree/stack.hpp:502 (bin+0x00000080580c)
#5 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::pop<MYDATA*>(MYDATA_&) /.../include/boost/lockfree/stack.hpp:435 (bin+0x0000008056e2)
#6 boost::lockfree::stack<MYDATA_, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::pop(MYDATA*&) /.../include/boost/lockfree/stack.hpp:417 (bin+0x0000008056a0)

Previous read of size 2 at 0x7dc400001190 by thread T6:
#0 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::consume_oneboost::lockfree::detail::consume_via_copy<MYDATA* >(boost::lockfree::detail::consume_via_copy<MYDATA*>&) /.../include/boost/lockfree/stack.hpp:498 (bin+0x000000805797)
#1 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::pop<MYDATA*>(MYDATA_&) /.../include/boost/lockfree/stack.hpp:435 (bin+0x0000008056e2)
#2 boost::lockfree::stack<MYDATA_, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::pop(MYDATA*&) /.../include/boost/lockfree/stack.hpp:417 (bin+0x0000008056a0)

Upon closer examine the offending line, I can see there is a compare_exchange_weak check immediately following it which throws away the result if the target is modified by other threads. However, the unprotected nonatomic r/w technically is still a data race in C++ language, which causes the behaviour of the program become undefined.

Switch to BOOST_NO_CXX11_HDR_ATOMIC ?

The existence of <atomic> not only depends on the version of compiler, but also the version of standard library.

When detail/atomic.hpp was created in this project, there was no BOOST_NO_CXX11_HDR_ATOMIC (which was introduced around Sep, 2014?) in boost::config. Now BOOST_NO_CXX11_HDR_ATOMIC has been well tested and covers far more compilers than BOOST_LOCKFREE_NO_HDR_ATOMIC. So I think it's time to switch to BOOST_NO_CXX11_HDR_ATOMIC .

Lockfree Queue Triggers Thread Sanitizer Data Race Warning

If there are multiple consumers, there will be tsan data race warnings. Here is the code to reproduce:

>  cat lockfree_race.cc; echo ==================; clang++-12 lockfree_race.cc -std=c++20 -fsanitize="thread" -o lockfree_race; ./lockfree_race                                                                                                                                                                                       
#include <boost/version.hpp>                                                                                                                                                                                                                                                                                                                         
#include <boost/lockfree/queue.hpp>                                                                                                                                                                                                                                                                                                                  
                                                                                                                                                                          
#include <iostream>                                                                  
#include <thread>                                                                                                                                                         
                                                                                                                                                                                                                                                                                                                                                     
int main() {                                                                                                                                                                                                                                                                                                                                         
    std::cout << BOOST_VERSION << std::endl;                                                                                                                                                                                                                                                                                                         
    const int count = 1000000;                                                                                                                                                                                                                                                                                                                       
    boost::lockfree::queue<int> queue(10);                                                                                                                                                                                                                                                                                                           
    auto producer = [&queue](){                                                                                                                                                                                                                                                                                                                      
        for (int i = 0; i < count; ++i) {                                                                                                                                 
            queue.push(i);                           
        }                                                                                                                                                                 
    };                                                                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                                                                     
    auto consumer = [&queue](){                                                                                                                                                                                                                                                                                                                      
        for (int i = 0; i < count; ++i) {                                                                                                                                 
            int val;                             
            queue.pop(val);                                                          
        }                                                                                                                                                                                                                                                                                                                                            
    };                                                                                                                                                                    
                                                                                                                                                                                                                                                                                                                                                     
    std::vector<std::thread> t;                                                                                                                                                                                                                                                                                                                      
    t.emplace_back(producer);                                                                                                                                                                                                                                                                                                                        
    t.emplace_back(consumer);                                                                                                                                                                                                                                                                                                                        
    t.emplace_back(consumer);                                                                                                                                             
                                                                                                                                                                          
    for (auto& tt : t) {                                                                                                                                                  
        tt.join();                                                                                                                                                                                                                                                                                                                                   
    }                                                                                                                                                                     
    return 0;                                 
}                                                                                    
==================                                              
107100                                       
==================                                                                                                                                                                                                                                                                                                                         [127/1026]
WARNING: ThreadSanitizer: data race (pid=682920)                                                                                                                                                                                                                                                                                                     
  Write of size 8 at 0x7b10000087c0 by thread T2:                                                                                                                         
    #0 boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::freelist_node>::set_ptr(boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::freelist_node*) <n
ull> (lockfree_race+0x4bc1b3)                                                                                                                                             
    #1 boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::deallocate_impl(boost::lockfree::queue<int>::node*) <null> (lockfree_race+0x4c0cc6)
    #2 void boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::deallocate<true>(boost::lockfree::queue<int>::node*) <null> (lockfree_race+0x4c0bd5)
    #3 void boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::destruct<true>(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> const&) <null> (lockfree_race+0x4c0b7a)
    #4 bool boost::lockfree::queue<int>::pop<int>(int&) <null> (lockfree_race+0x4c0aba)                                  
    #5 boost::lockfree::queue<int>::pop(int&) <null> (lockfree_race+0x4c07c5)                                                                                             
    #6 main::$_1::operator()() const <null> (lockfree_race+0x4bb021)
    #7 void std::__invoke_impl<void, main::$_1>(std::__invoke_other, main::$_1&&) <null> (lockfree_race+0x4baf9d)
    #8 std::__invoke_result<main::$_1>::type std::__invoke<main::$_1>(main::$_1&&) <null> (lockfree_race+0x4baeed)
    #9 void std::thread::_Invoker<std::tuple<main::$_1> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4bae95)       
    #10 std::thread::_Invoker<std::tuple<main::$_1> >::operator()() <null> (lockfree_race+0x4bae35)                                                                                                                                                                                                                                                  
    #11 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_1> > >::_M_run() <null> (lockfree_race+0x4bad29)                                                                                                                                                                                                                           
    #12 <null> <null> (libstdc++.so.6+0xda6b3)                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                          
  Previous atomic read of size 8 at 0x7b10000087c0 by thread T3:                                                                                                          
    #0 __tsan_atomic64_load <null> (lockfree_race+0x4717ce)                                                                                                               
    #1 std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::load(std::memory_order) const <null> (lockfree_race+0x4bd37d)
    #2 bool boost::lockfree::queue<int>::pop<int>(int&) <null> (lockfree_race+0x4c089b)                           
    #3 boost::lockfree::queue<int>::pop(int&) <null> (lockfree_race+0x4c07c5)                                                                                             
    #4 main::$_1::operator()() const <null> (lockfree_race+0x4bb021)                                                                                                      
    #5 void std::__invoke_impl<void, main::$_1>(std::__invoke_other, main::$_1&&) <null> (lockfree_race+0x4baf9d)                                                         
    #6 std::__invoke_result<main::$_1>::type std::__invoke<main::$_1>(main::$_1&&) <null> (lockfree_race+0x4baeed)
    #7 void std::thread::_Invoker<std::tuple<main::$_1> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4bae95)
    #8 std::thread::_Invoker<std::tuple<main::$_1> >::operator()() <null> (lockfree_race+0x4bae35)                                      
    #9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_1> > >::_M_run() <null> (lockfree_race+0x4bad29)                                                                                                                                                                                                                            
    #10 <null> <null> (libstdc++.so.6+0xda6b3)                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                                     
  Location is heap block of size 64 at 0x7b10000087c0 allocated by thread T1:                                                                                                                                                                                                                                                                        
    #0 operator new(unsigned long, std::align_val_t) <null> (lockfree_race+0x4b72fa)                                                                                                                                                                                                                                                                 
    #1 __gnu_cxx::new_allocator<boost::lockfree::queue<int>::node>::allocate(unsigned long, void const*) <null> (lockfree_race+0x4bbe18)                                  
    #2 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate_impl<false>() <null> (lockfree_race+0x4bc6f9)
    #3 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate<true, false>() <null> (lockfree_race+0x4bc5d5)
    #4 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be550)
    #5 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)
    #6 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)                                                                                      
    #7 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)                                                                                                                                                                                                                                                                                 
    #8 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)                                                                                                                                                                                                                                    
    #9 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)                                                                                                                                                                                                                                   
    #10 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)                                        
    #11 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)                                                             
    #12 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)                                                                                                                                                                                                                           
    #13 <null> <null> (libstdc++.so.6+0xda6b3)                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                                     
  Thread T2 (tid=682923, running) created by main thread at:                                                                                                              
    #0 pthread_create <null> (lockfree_race+0x425beb)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
    #2 decltype(new ((void*)(0))std::thread(std::declval<main::$_1&>())) std::construct_at<std::thread, main::$_1&>(std::thread*, main::$_1&) <null> (lockfree_race+0x4ba9c5)
    #3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_1&>(std::allocator<std::thread>&, std::thread*, main::$_1&) <null> (lockfree_race+0x4ba584)
    #4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_1&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_1&) <null> (lockfree_race+0x4ba6c9)
    #5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_1&>(main::$_1&) <null> (lockfree_race+0x4b98ba)
    #6 main <null> (lockfree_race+0x4b952a)

  Thread T3 (tid=682924, running) created by main thread at:
    #0 pthread_create <null> (lockfree_race+0x425beb)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
    #2 decltype(new ((void*)(0))std::thread(std::declval<main::$_1&>())) std::construct_at<std::thread, main::$_1&>(std::thread*, main::$_1&) <null> (lockfree_race+0x4ba9c5)
    #3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_1&>(std::allocator<std::thread>&, std::thread*, main::$_1&) <null> (lockfree_race+0x4ba584)
    #4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_1&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_1&) <null> (lockfree_race+0x4ba6c9)
    #5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_1&>(main::$_1&) <null> (lockfree_race+0x4b98ba)
    #6 main <null> (lockfree_race+0x4b9542)

  Thread T1 (tid=682922, running) created by main thread at:
    #0 pthread_create <null> (lockfree_race+0x425beb)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
    #2 decltype(new ((void*)(0))std::thread(std::declval<main::$_0&>())) std::construct_at<std::thread, main::$_0&>(std::thread*, main::$_0&) <null> (lockfree_race+0x4b9d95)
    #3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0&>(std::allocator<std::thread>&, std::thread*, main::$_0&) <null> (lockfree_race+0x4b9954)
    #4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_0&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_0&) <null> (lockfree_race+0x4b9a99)
    #5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0&>(main::$_0&) <null> (lockfree_race+0x4b978a)
    #6 main <null> (lockfree_race+0x4b9512)

SUMMARY: ThreadSanitizer: data race (/home/chenhao/lockfree_race+0x4bc1b3) in boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::freelist_node>::set_ptr(boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, 
std::allocator<boost::lockfree::queue<int>::node> >::freelist_node*)
==================
==================                                                                                                                                                                                                                                                                                                                                   
WARNING: ThreadSanitizer: data race (pid=682920)                                                                                                                                                                                                                                                                                                     
  Write of size 4 at 0x7b100000ce48 by thread T1:                                                                                                                                                                                                                                                                                                    
    #0 boost::lockfree::queue<int>::node::node(int const&, boost::lockfree::queue<int>::node*) <null> (lockfree_race+0x4be835)                                                                                                                                                                                                                       
    #1 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be5a0)                     
    #2 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)                                                                                                                                                                                                                                                  
    #3 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)                                                                                                                                                                                                                                                                 
    #4 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)                                                                                                                                                                                                                                                                                 
    #5 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)                                                                                                                                                                                                                                    
    #6 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)                                                                                                                                                                                                                                   
    #7 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)                                                                                                                                                                                                                    
    #8 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)                                                                                                                                                                                                                                                   
    #9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)                                                                                                                                                                                                                            
    #10 <null> <null> (libstdc++.so.6+0xda6b3)                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                                     
  Previous read of size 4 at 0x7b100000ce48 by thread T3:                                                                                                                                                                                                                                                                                            
    #0 void boost::lockfree::detail::copy_convertible::copy<int, int>(int&, int&) <null> (lockfree_race+0x4bd9c5)                                                                                                                                                                                                                                    
    #1 void boost::lockfree::detail::copy_payload<int, int>(int&, int&) <null> (lockfree_race+0x4bd845)                                                                                                                                                                                                                                              
    #2 bool boost::lockfree::queue<int>::pop<int>(int&) <null> (lockfree_race+0x4c0a18)                                                                                                                                                                                                                                                              
    #3 boost::lockfree::queue<int>::pop(int&) <null> (lockfree_race+0x4c07c5)                                                                                                                                                                                                                                                                        
    #4 main::$_1::operator()() const <null> (lockfree_race+0x4bb021)                                                                                                                                                                                                                                                                                 
    #5 void std::__invoke_impl<void, main::$_1>(std::__invoke_other, main::$_1&&) <null> (lockfree_race+0x4baf9d)                                                                                                                                                                                                                                    
    #6 std::__invoke_result<main::$_1>::type std::__invoke<main::$_1>(main::$_1&&) <null> (lockfree_race+0x4baeed)                                                                                                                                                                                                                                   
    #7 void std::thread::_Invoker<std::tuple<main::$_1> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4bae95)                                                                                                                                                                                                                    
    #8 std::thread::_Invoker<std::tuple<main::$_1> >::operator()() <null> (lockfree_race+0x4bae35)                                                                                                                                                                                                                                                   
    #9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_1> > >::_M_run() <null> (lockfree_race+0x4bad29)                                                                                                                                                                                                                            
    #10 <null> <null> (libstdc++.so.6+0xda6b3)                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                                     
  Location is heap block of size 64 at 0x7b100000ce40 allocated by thread T1:                                                                                                                                                                                                                                                                        
    #0 operator new(unsigned long, std::align_val_t) <null> (lockfree_race+0x4b72fa)                                                                                                                                                                                                                                                                 
    #1 __gnu_cxx::new_allocator<boost::lockfree::queue<int>::node>::allocate(unsigned long, void const*) <null> (lockfree_race+0x4bbe18)                                                                                                                                                                                                             
    #2 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate_impl<false>() <null> (lockfree_race+0x4bc6f9)                                                                                                                     
    #3 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate<true, false>() <null> (lockfree_race+0x4bc5d5)                                                                                                                    
    #4 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be550)                     
    #5 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)                                                                                                                                                                                                                                                  
    #6 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)                                                                                                                                                                                                                                                                 
    #7 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)                                                                                                                                                                                                                                                                                 
    #8 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)                                                                                                                                                                                                                                    
    #9 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)                                                                                                                                                                                                                                   
    #10 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)                                                                                                                                                                                                                   
    #11 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)                                                                                                                                                                                                                                                  
    #12 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)                                                                                                                                                                                                                           
    #13 <null> <null> (libstdc++.so.6+0xda6b3)                                                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                                                                                     
  Thread T1 (tid=682922, running) created by main thread at:                                                                                                                                                                                                                                                                                         
    #0 pthread_create <null> (lockfree_race+0x425beb)                                                                                                                                                                                                                                                                                                
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)                                                                                                                                                                                     
    #2 decltype(new ((void*)(0))std::thread(std::declval<main::$_0&>())) std::construct_at<std::thread, main::$_0&>(std::thread*, main::$_0&) <null> (lockfree_race+0x4b9d95)                                                                                                                                                                        
    #3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0&>(std::allocator<std::thread>&, std::thread*, main::$_0&) <null> (lockfree_race+0x4b9954)                                                                                                                                                          
    #4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_0&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_0&) <null> (lockfree_race+0x4b9a99)                                                                                                   
    #5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0&>(main::$_0&) <null> (lockfree_race+0x4b978a)                                                                                                                                                                                                     
    #6 main <null> (lockfree_race+0x4b9512)

  Thread T3 (tid=682924, running) created by main thread at:
    #0 pthread_create <null> (lockfree_race+0x425beb)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
    #2 decltype(new ((void*)(0))std::thread(std::declval<main::$_1&>())) std::construct_at<std::thread, main::$_1&>(std::thread*, main::$_1&) <null> (lockfree_race+0x4ba9c5)
    #3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_1&>(std::allocator<std::thread>&, std::thread*, main::$_1&) <null> (lockfree_race+0x4ba584)
    #4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_1&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_1&) <null> (lockfree_race+0x4ba6c9)
    #5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_1&>(main::$_1&) <null> (lockfree_race+0x4b98ba)
    #6 main <null> (lockfree_race+0x4b9542)

SUMMARY: ThreadSanitizer: data race (/home/chenhao/lockfree_race+0x4be835) in boost::lockfree::queue<int>::node::node(int const&, boost::lockfree::queue<int>::node*)
==================
==================
WARNING: ThreadSanitizer: data race (pid=682920)
  Write of size 8 at 0x7b100000d940 by thread T1:
    #0 memcpy <null> (lockfree_race+0x42fcae)
    #1 std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::atomic(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>) <null> (lockfree_race+0x4bb81a)
    #2 boost::lockfree::queue<int>::node::node(int const&, boost::lockfree::queue<int>::node*) <null> (lockfree_race+0x4be804)
    #3 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be5a0)
    #4 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)
    #5 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)
    #6 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)
    #7 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)
    #8 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)
    #9 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)
    #10 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)
    #11 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)
    #12 <null> <null> (libstdc++.so.6+0xda6b3)

  Previous atomic read of size 8 at 0x7b100000d940 by thread T3:
    #0 __tsan_atomic64_load <null> (lockfree_race+0x4717ce)
    #1 std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::load(std::memory_order) const <null> (lockfree_race+0x4bd37d)
    #2 bool boost::lockfree::queue<int>::pop<int>(int&) <null> (lockfree_race+0x4c089b)
    #3 boost::lockfree::queue<int>::pop(int&) <null> (lockfree_race+0x4c07c5)
    #4 main::$_1::operator()() const <null> (lockfree_race+0x4bb021)
    #5 void std::__invoke_impl<void, main::$_1>(std::__invoke_other, main::$_1&&) <null> (lockfree_race+0x4baf9d)
    #6 std::__invoke_result<main::$_1>::type std::__invoke<main::$_1>(main::$_1&&) <null> (lockfree_race+0x4baeed)
    #7 void std::thread::_Invoker<std::tuple<main::$_1> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4bae95)
    #8 std::thread::_Invoker<std::tuple<main::$_1> >::operator()() <null> (lockfree_race+0x4bae35)
    #9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_1> > >::_M_run() <null> (lockfree_race+0x4bad29)
    #10 <null> <null> (libstdc++.so.6+0xda6b3)

  Location is heap block of size 64 at 0x7b100000d940 allocated by thread T1:
    #0 operator new(unsigned long, std::align_val_t) <null> (lockfree_race+0x4b72fa)
    #1 __gnu_cxx::new_allocator<boost::lockfree::queue<int>::node>::allocate(unsigned long, void const*) <null> (lockfree_race+0x4bbe18)
    #2 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate_impl<false>() <null> (lockfree_race+0x4bc6f9)
    #3 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate<true, false>() <null> (lockfree_race+0x4bc5d5)
    #4 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be550)
    #5 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)
    #6 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)
    #7 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)
    #8 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)
    #9 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)
    #10 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)
    #11 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)
    #12 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)
    #13 <null> <null> (libstdc++.so.6+0xda6b3)

  Thread T1 (tid=682922, running) created by main thread at:
    #0 pthread_create <null> (lockfree_race+0x425beb)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
    #2 decltype(new ((void*)(0))std::thread(std::declval<main::$_0&>())) std::construct_at<std::thread, main::$_0&>(std::thread*, main::$_0&) <null> (lockfree_race+0x4b9d95)
    #3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0&>(std::allocator<std::thread>&, std::thread*, main::$_0&) <null> (lockfree_race+0x4b9954)
    #4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_0&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_0&) <null> (lockfree_race+0x4b9a99)
    #5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0&>(main::$_0&) <null> (lockfree_race+0x4b978a)
    #6 main <null> (lockfree_race+0x4b9512)

  Thread T3 (tid=682924, running) created by main thread at:
    #0 pthread_create <null> (lockfree_race+0x425beb)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
    #2 decltype(new ((void*)(0))std::thread(std::declval<main::$_1&>())) std::construct_at<std::thread, main::$_1&>(std::thread*, main::$_1&) <null> (lockfree_race+0x4ba9c5)
    #3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_1&>(std::allocator<std::thread>&, std::thread*, main::$_1&) <null> (lockfree_race+0x4ba584)
    #4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_1&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_1&) <null> (lockfree_race+0x4ba6c9)
    #5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_1&>(main::$_1&) <null> (lockfree_race+0x4b98ba)
    #6 main <null> (lockfree_race+0x4b9542)

SUMMARY: ThreadSanitizer: data race (/home/chenhao/lockfree_race+0x42fcae) in memcpy
==================

Environment:

> clang++-12 --version
Ubuntu clang version 12.0.0-3ubuntu1~20.04.5
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
> uname -a
Linux chenhao 5.4.0-109-generic #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

queue misaligned access fails UBSAN

Simple usage of the queue triggers gcc's UBSAN checker flagging misaligned 64b accesses. Test case:

#include <iostream>
#include <boost/lockfree/queue.hpp>

int main()
{
    boost::lockfree::queue<std::size_t> q(5);

    q.push(1);
    q.push(2);
    q.push(3);
    q.push(4);
    q.push(5);

    std::size_t top;
    q.pop(top);
    std::cout << "top of queue was " << top << "\n";

    if (q.empty())
    {
        std::cout << "the queue is now empty (!?)\n";
    }

    q.consume_all(
        [](std::size_t v)
        {
            std::cout << "next value " << v << "\n";
        });
}

Reconfiguring with BUILD_TESTING=OFF doesn't disable tests

When Lockfree is configured with cmake -DBUILD_TESTING=ON .., and then reconfigured with cmake -DBUILD_TESTING=OFF ., tests remain enabled. That's because BUILD_TESTING is only used once as the default value of BOOST_LOCKFREE_BUILD_TESTS and is ignored afterwards.

That's not how all other Boost libraries work; they check BUILD_TESTING directly and do not add_subdirectory(test) if it's off.

differering destructor behavior with compile-time size

I noticed something I found surprising - a boost::lockfree:spsc_queue behaves differently on destruction if you give it a compile-time size vs giving it a runtime size. When you have a runtime-sized queue, then any elements still in the queue are destructed when the queue destructs (which is how I usually expect containers to behave). That is handled here. But compile_time_sized_ringbuffer doesn't have a destructor declared, and it's storage is just a boost::aligned_storage, so it won't destruct any stored objects either.

My assumption is that this is not intentional, and that we expect the remaining elements of the queue to be destructed when destroying a compile-time-sized spsc_queue?

queue::has_capacity Requirements Not Documented

The constructors for queue do not document their requirements with respect to the capacity tag.

Moreover in certain constructors a runtime assert (only present in debug mode) are used to verify the requirement. It shouldn't be necessary to build code in debug mode to discover what is logically a compile time constraint.

unsynchronized_push does not compile while stack capacity is specified

The code below does not compile( boost 1.72, visual c++ 2019 community), the error message is:

boost\boost\lockfree\stack.hpp(241,42): error C2440: '=': cannot convert from 'T *' to 'unsigned short'

#include <boost/lockfree/stack.hpp>

int main()
{
    boost::lockfree::stack<int, boost::lockfree::capacity<256>> int_stack;

    int_stack.unsynchronized_push(100);

    return 0;
}

error in tagged_ptr_dcas.hpp about BOOST_ALIGNMENT

#include <boost/lockfree/detail/tagged_ptr_dcas.hpp>

int main() {}

/*
[gustavo@casa bug]$ g++ stack.cc -o stack
En el fichero incluido desde stack.cc:1:
/usr/include/boost/lockfree/detail/tagged_ptr_dcas.hpp:30:17: error: expected unqualified-id before numeric constant
30 | BOOST_ALIGNMENT(2 * sizeof(void*))
| ^
/usr/include/boost/lockfree/detail/tagged_ptr_dcas.hpp:30:17: error: expected ‘)’ before numeric constant
30 | BOOST_ALIGNMENT(2 * sizeof(void*))
| ~^
| )
*/

How to contribute code

I want to contribute some lockfree code that I wrote myself. What should I do? What are the requirements for the code?

fix valgrind issue of uninitialized next value in node

The following two errors appear in all queue related tests when running under valgrind (valgrind-3.14.0).
valgrind output when running the queue example binary:

==29902== Conditional jump or move depends on uninitialised value(s)
==29902==    at 0x415273: std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::compare_exchange_weak(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>&, boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>, std::memory_order, std::memory_order) (atomic:288)
==29902==    by 0x414292: std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::compare_exchange_weak(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>&, boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>, std::memory_order) (atomic:308)
==29902==    by 0x412C88: bool boost::lockfree::queue<int>::do_push<false>(int const&) (queue.hpp:318)
==29902==    by 0x41135A: boost::lockfree::queue<int>::push(int const&) (queue.hpp:280)
==29902==    by 0x40C711: producer() (queue.cpp:27)
==29902==    by 0x416302: boost::detail::thread_data<void (*)()>::run() (thread.hpp:117)
==29902==    by 0x4E59475: thread_proxy (thread.cpp:177)
==29902==    by 0x61E9593: start_thread (in /usr/lib64/libpthread-2.27.so)
==29902==    by 0x64FAF4E: clone (in /usr/lib64/libc-2.27.so)
==29902== 
==29902== Conditional jump or move depends on uninitialised value(s)
==29902==    at 0x412C8B: bool boost::lockfree::queue<int>::do_push<false>(int const&) (queue.hpp:318)
==29902==    by 0x41135A: boost::lockfree::queue<int>::push(int const&) (queue.hpp:280)
==29902==    by 0x40C711: producer() (queue.cpp:27)
==29902==    by 0x416302: boost::detail::thread_data<void (*)()>::run() (thread.hpp:117)
==29902==    by 0x4E59475: thread_proxy (thread.cpp:177)
==29902==    by 0x61E9593: start_thread (in /usr/lib64/libpthread-2.27.so)
==29902==    by 0x64FAF4E: clone (in /usr/lib64/libc-2.27.so)

As I understand, this is not a real issue, since it does not matter which tag we use at the beginning, as long as we increment it, however, it does create noise when running with valgrind, and does not seem to have any cost in perf.
Also, the fix is simple, as it could be done by uncommenting existing code in the ctor of the node class in queue.hpp

"No constructor could take the source type, or constructor overload resolution was ambiguous" with boost::lockfree::queue::unsynchronized_push

When I used the code below in vs 2019:

#include<boost/lockfree/queue.hpp>

int main() {
  boost::lockfree::queue<unsigned, boost::lockfree::capacity<32>> q;
  q.unsynchronized_push(1);
  return 0;
}

The building failed and reported:

1>D:\boost_1_77_0\boost\lockfree\queue.hpp(377,45): error C2039: 'next': is not a member of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\detail\freelist.hpp(269): message : see declaration of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(378,36): error C2039: 'get_ptr': is not a member of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\detail\freelist.hpp(269): message : see declaration of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(381,23): error C2039: 'next': is not a member of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\detail\freelist.hpp(269): message : see declaration of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(381,52): error C2440: '': cannot convert from 'initializer list' to 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(369,1): message : No constructor could take the source type, or constructor overload resolution was ambiguous
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(382,47): error C2440: '': cannot convert from 'initializer list' to 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(369,1): message : No constructor could take the source type, or constructor overload resolution was ambiguous
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(386,47): error C2440: '': cannot convert from 'initializer list' to 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(369,1): message : No constructor could take the source type, or constructor overload resolution was ambiguous

queue_interprocess_test and stack_interprocess_test failed in clang on Windows for conflicting types

These two tests queue_interprocess_test and stack_interprocess_test failed in clang on Windows for conflicting types.
Compilation command:

clang queue_interprocess_test.cpp -o queue_interprocess_test.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_ATOMIC_STATIC_LINK=1 -DBOOST_CHRONO_STATIC_LINK=1 -DBOOST_TEST_NO_AUTO_LINK=1 -DBOOST_THREAD_BUILD_LIB=1 -DBOOST_THREAD_USE_LIB=1 -DBOOST_THREAD_WIN32 -I"..\..\.."

Error message:

In file included from queue_interprocess_test.cpp:12:
In file included from ../../..\boost/thread/thread.hpp:12:
In file included from ../../..\boost/thread/thread_only.hpp:15:
In file included from ../../..\boost/thread/win32/thread_data.hpp:11:
In file included from ../../..\boost/thread/win32/thread_primitives.hpp:22:
../../..\boost/winapi/semaphore.hpp:25:1: error: conflicting types for
      'CreateSemaphoreA'
CreateSemaphoreA(
^
../../..\boost/interprocess/detail/win32_api.hpp:926:51: note: previous
      declaration is here
extern "C" __declspec(dllimport) void * __stdcall CreateSemaphoreA(inter...
                                                  ^

cl from msvc will successfully compile this test.

cl queue_interprocess_test.cpp -TP /Z7 /Od /Ob0 /W3 /GR /MDd /Zc:forScope /Zc:wchar_t /favor:blend /wd4675 /EHs /std:c++17   -D_CRT_USE_BUILTIN_OFFSETOF -c  -DBOOST_ALL_NO_LIB=1  -DBOOST_ATOMIC_STATIC_LINK=1  -DBOOST_CHRONO_STATIC_LINK=1  -DBOOST_TEST_NO_AUTO_LINK=1  -DBOOST_THREAD_BUILD_LIB=1  -DBOOST_THREAD_USE_LIB=1  -DBOOST_THREAD_WIN32  "-I..\..\.."

I am still investigating the reason. If you know why, please let me know. Thanks!

move only types are not supported

It is currently not possible to get a move only type into the lockfree datastructures. It leades to compile errors.

queue template constructors uncallable

Using queue with an allocator, it is impossible to write code that invokes this template constructor:

template <typename U>
    queue(size_type n, typename node_allocator::template rebind<U>::other const & alloc):

(or the other one with the rebind).

The template argument U cannot be deduced (at least with gcc 5.3.0).

I had to add the following form in order to use this with a custom allocator:

explicit queue(size_type n, allocator const & alloc)

I notice that the constructor that takes only an alloc already has this overload.

Spsc queue with no bounded size

Since I saw boost lockfree I wasn't sure why there isn't any implemention which increase is queue is by factor of 2 like std::vector.
I have a simple implemention using my own custom queue https://github.com/danikv/DanielThreadSafeLibrary/blob/master/include/growing_spsc_queue.h , does that interest anyone or is this use case only relevant to me?

1.6.7 lockfree queue, out of order

single thread push/pop outoforder

boostorg / lockfree Goto Github PK

lockfree's People

Contributors

Stargazers

Watchers

Forkers

lockfree's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs