boostorg / lockfree Goto Github PK
View Code? Open in Web Editor NEWBoost.Lockfree
Boost.Lockfree
Working on a new boost release I found the lockfree queue stopped working proper and tracked it down: some x86_64 codes are included improperly in a couple of files, namely in boost/lockfree/detail/prefix.hpp and tagged_ptr_ptrcompression.hpp:
#ifdef BOOST_ARCH_X86_64
Should be
#if BOOST_ARCH_X86_64
Otherwise codes on all the other ARCHs will include the wrong piece. I tested the change on powerpc and it works fine so far.
I want to contribute some lockfree code that I wrote myself. What should I do? What are the requirements for the code?
As we know, boost using tagged_ptr avoid CAS ABA issue. But in aarch64, tagged_ptr_ptrcompression using top bits of pointer, but HWASan also using top bits of pointer, so conflict.
HWASan help detect memory bugs in LLVM project, so should boost lockfree compat HWASan ?
I can get around it right now by using std::shared_ptr, but would be nice if the push function could accept r-values and move appropriately.
Similarly the pop & consume functions could move the objects out of the queue?
Hi,everybody~
I found nodes in freelist can never be shrinked, it can only grow larger(like "reserve_unsafe" and "reserve" do).
The only way to reduce the capacity of freelist is destruct queue,ringbuffer or stack. But that is not convenient. Sorry I must make the memory occupy not so high affter some request burst.
Is it possible to improve that?
If cann't, then "capacity_unsafe" function to tell current capacity is needed.
I understand data structure in freelist implementation should lock free as much as possible. But it can have a flexible way like limit the max freelist capacity. Such as when return node to freelist, examine the current capacity. It doesn't need to be accurate,so "capacity" doesn't need to be a atmoic variable.
Just some my silly ideas~
Thanks~
# root at alarmpi in ~ [21:36:15]
$ cat test.cpp
#include <atomic>
#include <boost/lockfree/queue.hpp>
#include <iostream>
std::atomic_int intcount;
boost::lockfree::queue<int> intpool(100);
int main()
{
return 0;
}
# root at alarmpi in ~ [21:36:53]
$ g++ test.cpp -std=c++11
# root at alarmpi in ~ [21:37:15]
$ ./a.out
[1] 31406 bus error (core dumped) ./a.out
# root at alarmpi in ~ [21:37:27]
$ coredumpctl gdb 31406
PID: 31406 (a.out)
UID: 0 (root)
GID: 0 (root)
Signal: 7 (BUS)
Timestamp: 一 2015-04-27 21:37:27 CST (12s ago)
Command Line: ./a.out
Executable: /root/a.out
Control Group: /user.slice/user-0.slice/session-c6.scope
Unit: session-c6.scope
Slice: user-0.slice
Session: c6
Owner UID: 0 (root)
Boot ID: f2ff2834fe86475d8008560472158aa9
Machine ID: 20d4f52b53de4dd9aa7ac72e93285b0a
Hostname: alarmpi
Coredump: /var/lib/systemd/coredump/core.a\x2eout.0.f2ff2834fe86475d8008560472158aa9.31406.1430141847000000.lz4
Message: Process 31406 (a.out) of user 0 dumped core.
GNU gdb (GDB) 7.9
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "armv7l-unknown-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /root/a.out...(no debugging symbols found)...done.
[New LWP 31406]
Core was generated by `./a.out'.
Program terminated with signal SIGBUS, Bus error.
#0 0x00010ec8 in std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::freelist_node> >::load(std::memory_order) const ()
(gdb) bt
#0 0x00010ec8 in std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::freelist_node> >::load(std::memory_order) const ()
#1 0x0001110c in boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::deallocate_impl_unsafe(boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node*) ()
#2 0x00010e94 in void boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::deallocate<false>(boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node*) ()
#3 0x00010bd0 in boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node, std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >::freelist_stack<std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> >(std::allocator<boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::node> const&, unsigned int) ()
#4 0x00010a34 in boost::lockfree::queue<int, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::queue(unsigned int) ()
#5 0x000107e8 in __static_initialization_and_destruction_0(int, int) ()
#6 0x00010820 in _GLOBAL__sub_I_intcount ()
#7 0x000119f4 in __libc_csu_init ()
#8 0x76c22ccc in __libc_start_main () from /usr/lib/libc.so.6
#9 0x0001065c in _start ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) q
# root at alarmpi in ~ [21:37:47]
$ uname -a
Linux alarmpi 3.18.11-4-ARCH #1 SMP PREEMPT Tue Apr 21 21:52:58 MDT 2015 armv7l GNU/Linux
# root at alarmpi in ~ [21:37:50]
$ g++ -v
使用内建 specs。
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/armv7l-unknown-linux-gnueabihf/4.9.2/lto-wrapper
目标:armv7l-unknown-linux-gnueabihf
配置为:/build/gcc/src/gcc-4.9-20150304/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://github.com/archlinuxarm/PKGBUILDs/issues --enable-languages=c,c++,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-cloog-backend=isl --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --disable-multilib --disable-werror --enable-checking=release --host=armv7l-unknown-linux-gnueabihf --build=armv7l-unknown-linux-gnueabihf --with-arch=armv7-a --with-float=hard --with-fpu=vfpv3-d16
线程模型:posix
gcc 版本 4.9.2 20150304 (prerelease) (GCC)
# root at alarmpi in ~ [21:37:58]
$ /lib/libc-2.21.so
GNU C Library (GNU libc) stable release version 2.21, by Roland McGrath et al.
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.9.2 20150304 (prerelease).
Available extensions:
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
libc ABIs: UNIQUE
For bug reporting instructions, please see:
<https://github.com/archlinuxarm/PKGBUILDs/issues>.
I noticed something I found surprising - a boost::lockfree:spsc_queue behaves differently on destruction if you give it a compile-time size vs giving it a runtime size. When you have a runtime-sized queue, then any elements still in the queue are destructed when the queue destructs (which is how I usually expect containers to behave). That is handled here. But compile_time_sized_ringbuffer
doesn't have a destructor declared, and it's storage is just a boost::aligned_storage
, so it won't destruct any stored objects either.
My assumption is that this is not intentional, and that we expect the remaining elements of the queue to be destructed when destroying a compile-time-sized spsc_queue?
in https://github.com/boostorg/lockfree/blob/develop/include/boost/lockfree/detail/freelist.hpp lines 335 and 369, replace the numeric const with compile time const derived from 32bit vs. 64bit compilation
(e.g. #if __x86_64__ || __ppc64__
)
Unlike runtime_sized_ringbuffer
, compile_time_sized_ringbuffer
does not have a custom destructor, and so it does not clean up any outstanding queue items when destroyed.
I am not a lockfree guru, so I do not know if there is some hidden limitation that makes this impossible, or if it is simply a bug.
I could not find any documentation for this behavior, so I suspect the latter.
I can make a PR for the proposed fix below, if it seems correct.
Here is a small test that fails unexpectedly for me:
int g_numObjs = 0;
struct Obj {
Obj() {
++g_numObjs;
}
Obj(const Obj&) {
++g_numObjs;
}
~Obj() {
--g_numObjs;
}
};
typedef boost::lockfree::spsc_queue<
Obj
// Note: commenting out this line makes it into a
// runtime_sized_ringbuffer, and the test passes.
,boost::lockfree::capacity<10>
> MyQueue;
TEST_CASE("boost spsc bug") {
{
REQUIRE(g_numObjs == 0);
MyQueue queue;
queue.push(Obj());
REQUIRE(g_numObjs == 1);
} // queue gets destroyed
// FAILS
REQUIRE(g_numObjs == 0);
}
If I add a destructor to compile_time_sized_ringbuffer
, such as below (copied from runtime_sized_ringbuffer
), then the test passes and everything seems fine:
~compile_time_sized_ringbuffer(void)
{
// destroy all remaining items
T out;
while (pop(&out, 1)) {}
}
Note: I suppose another implemenation might be to call reset()
from inside ~spsc_queue()
itself, which would benefit from the optimization of not calling trivial destructors. Or maybe a similar optimization would be warranted in the destructors for compile_time_sized...
and runtime_sized_...
as well?
this code...
#include <boost/lockfree/detail/tagged_ptr_dcas.hpp>
int main(int argc, char **argv)
{
int i = argc;
return i;
}
generate this error...
[gustavo@casa c++]$ g++ boost.cc -o boost
En el fichero incluido desde boost.cc:1:
/usr/include/boost/lockfree/detail/tagged_ptr_dcas.hpp:30:17: error: expected unqualified-id before numeric constant
30 | BOOST_ALIGNMENT(2 * sizeof(void*))
| ^
/usr/include/boost/lockfree/detail/tagged_ptr_dcas.hpp:30:17: error: expected ‘)’ before numeric constant
30 | BOOST_ALIGNMENT(2 * sizeof(void*))
| ~^
| )
The code below does not compile( boost 1.72, visual c++ 2019 community), the error message is:
boost\boost\lockfree\stack.hpp(241,42): error C2440: '=': cannot convert from 'T *' to 'unsigned short'
#include <boost/lockfree/stack.hpp>
int main()
{
boost::lockfree::stack<int, boost::lockfree::capacity<256>> int_stack;
int_stack.unsynchronized_push(100);
return 0;
}
Since I saw boost lockfree I wasn't sure why there isn't any implemention which increase is queue is by factor of 2 like std::vector.
I have a simple implemention using my own custom queue https://github.com/danikv/DanielThreadSafeLibrary/blob/master/include/growing_spsc_queue.h , does that interest anyone or is this use case only relevant to me?
single thread push/pop outoforder
The documented testing procedure
mkdir __build && cd __build
cmake -DBUILD_TESTING=ON -DBOOST_INCLUDE_LIBRARIES=lockfree ..
cmake --build . --target tests
ctest --output-on-failure --no-tests=error
fails for Boost.Lockfree, because the test executables aren't built by the target tests
. BoostTest handles this automatically, but for tests declared manually, one needs to first declare the tests
target if not present
if(NOT TARGET tests)
add_custom_target(tests)
endif()
and then for each test executable, use add_dependencies(test my_test_executable)
.
Or, since there is already a target boost_lockfree_all_tests
, it should be enough to make it a dependency of tests
: add_dependencies(tests boost_lockfree_all_tests)
.
(For this to keep working after the eventual fix, it might be a good idea to add it to CI.)
It's also good practice to make the test executables EXCLUDE_FOR_ALL, so that building the tests
target builds them, but building the default target doesn't, but this step is not in principle required.
due to template param deduction failure in constructor declaration (line 249), see godbolt
as a quick and dirty workaround I replaced "typename boost::allocator_rebind<node_allocator, U>::type const& alloc" with plain "U const &alloc" there in C:\Project\Libs\vcpkg\installed\x64-windows-static\include\boost\lockfree\queue.hpp and that made it compile and work but I hope it's going to be fixed in the trunk
When I used the code below in vs 2019:
#include<boost/lockfree/queue.hpp>
int main() {
boost::lockfree::queue<unsigned, boost::lockfree::capacity<32>> q;
q.unsynchronized_push(1);
return 0;
}
The building failed and reported:
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(377,45): error C2039: 'next': is not a member of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\detail\freelist.hpp(269): message : see declaration of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(378,36): error C2039: 'get_ptr': is not a member of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\detail\freelist.hpp(269): message : see declaration of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(381,23): error C2039: 'next': is not a member of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\detail\freelist.hpp(269): message : see declaration of 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(381,52): error C2440: '': cannot convert from 'initializer list' to 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(369,1): message : No constructor could take the source type, or constructor overload resolution was ambiguous
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(382,47): error C2440: '': cannot convert from 'initializer list' to 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(369,1): message : No constructor could take the source type, or constructor overload resolution was ambiguous
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(386,47): error C2440: '': cannot convert from 'initializer list' to 'boost::lockfree::detail::tagged_index'
1>D:\boost_1_77_0\boost\lockfree\queue.hpp(369,1): message : No constructor could take the source type, or constructor overload resolution was ambiguous
The following two errors appear in all queue related tests when running under valgrind (valgrind-3.14.0).
valgrind output when running the queue example binary:
==29902== Conditional jump or move depends on uninitialised value(s)
==29902== at 0x415273: std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::compare_exchange_weak(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>&, boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>, std::memory_order, std::memory_order) (atomic:288)
==29902== by 0x414292: std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::compare_exchange_weak(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>&, boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>, std::memory_order) (atomic:308)
==29902== by 0x412C88: bool boost::lockfree::queue<int>::do_push<false>(int const&) (queue.hpp:318)
==29902== by 0x41135A: boost::lockfree::queue<int>::push(int const&) (queue.hpp:280)
==29902== by 0x40C711: producer() (queue.cpp:27)
==29902== by 0x416302: boost::detail::thread_data<void (*)()>::run() (thread.hpp:117)
==29902== by 0x4E59475: thread_proxy (thread.cpp:177)
==29902== by 0x61E9593: start_thread (in /usr/lib64/libpthread-2.27.so)
==29902== by 0x64FAF4E: clone (in /usr/lib64/libc-2.27.so)
==29902==
==29902== Conditional jump or move depends on uninitialised value(s)
==29902== at 0x412C8B: bool boost::lockfree::queue<int>::do_push<false>(int const&) (queue.hpp:318)
==29902== by 0x41135A: boost::lockfree::queue<int>::push(int const&) (queue.hpp:280)
==29902== by 0x40C711: producer() (queue.cpp:27)
==29902== by 0x416302: boost::detail::thread_data<void (*)()>::run() (thread.hpp:117)
==29902== by 0x4E59475: thread_proxy (thread.cpp:177)
==29902== by 0x61E9593: start_thread (in /usr/lib64/libpthread-2.27.so)
==29902== by 0x64FAF4E: clone (in /usr/lib64/libc-2.27.so)
As I understand, this is not a real issue, since it does not matter which tag we use at the beginning, as long as we increment it, however, it does create noise when running with valgrind, and does not seem to have any cost in perf.
Also, the fix is simple, as it could be done by uncommenting existing code in the ctor of the node class in queue.hpp
It is currently not possible to get a move only type into the lockfree datastructures. It leades to compile errors.
lockfree/include/boost/lockfree/queue.hpp
Line 188 in ffd91db
This check for construction of the queue, if the capacity has been specified as the template parameter should be moved to static assert rather than having the run time assert.
Running my program (uses boost 1.60) through Thread Sanitizer reports the following data race:
WARNING: ThreadSanitizer: data race (pid=23760)
Write of size 2 at 0x7dc400000110 by thread T6:
#0 boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::link_nodes_atomic(boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node_, boost::lockfree::stack<MYDATA_, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node_) /.../include/boost/lockfree/stack.hpp:226 (bin+0x00000079812c)
#1 bool boost::lockfree::stack<MYDATA_, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::do_push(MYDATA* const&) /.../include/boost/lockfree/stack.hpp:314 (bin+0x000000798034)
#2 boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::bounded_push(MYDATA* const&) /.../include/boost/lockfree/stack.hpp:302 (bin+0x000000797a20)Previous read of size 2 at 0x7dc400000110 by thread T5:
#0 boost::lockfree::detail::tagged_index::get_index() const /.../include/boost/lockfree/detail/freelist.hpp:281 (bin+0x000000797e59)
#1 boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::allocate_impl() /.../include/boost/lockfree/detail/freelist.hpp:545 (bin+0x0000007982e0)
#2 unsigned short boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::allocate() /.../include/boost/lockfree/detail/freelist.hpp:527 (bin+0x0000007981f9)
#3 boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node* boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::construct<true, true, MYDATA*>(MYDATA* const&) /.../include/boost/lockfree/detail/freelist.hpp:443 (bin+0x000000798071)
#4 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::do_push(MYDATA* const&) /.../include/boost/lockfree/stack.hpp:310 (bin+0x000000798021)
#5 boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::bounded_push(MYDATA* const&) /.../include/boost/lockfree/stack.hpp:302 (bin+0x000000797a20)
And another similar one:
WARNING: ThreadSanitizer: data race (pid=23760)
Write of size 2 at 0x7dc400001190 by thread T5:
#0 boost::lockfree::detail::tagged_index::set_index(unsigned short) /.../include/boost/lockfree/detail/freelist.hpp:286 (bin+0x000000797cdd)
#1 boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::deallocate_impl(unsigned short) /.../include/boost/lockfree/detail/freelist.hpp:585 (bin+0x000000805953)
#2 void boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::deallocate(unsigned short) /.../include/boost/lockfree/detail/freelist.hpp:573 (bin+0x0000008058d0)
#3 void boost::lockfree::detail::fixed_size_freelist<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::node, 4096ul> >::destruct(boost::lockfree::detail::tagged_index) /.../include/boost/lockfree/detail/freelist.hpp:471 (bin+0x00000080589f)
#4 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::consume_oneboost::lockfree::detail::consume_via_copy<MYDATA* >(boost::lockfree::detail::consume_via_copy<MYDATA*>&) /.../include/boost/lockfree/stack.hpp:502 (bin+0x00000080580c)
#5 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::pop<MYDATA*>(MYDATA_&) /.../include/boost/lockfree/stack.hpp:435 (bin+0x0000008056e2)
#6 boost::lockfree::stack<MYDATA_, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::pop(MYDATA*&) /.../include/boost/lockfree/stack.hpp:417 (bin+0x0000008056a0)Previous read of size 2 at 0x7dc400001190 by thread T6:
#0 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::consume_oneboost::lockfree::detail::consume_via_copy<MYDATA* >(boost::lockfree::detail::consume_via_copy<MYDATA*>&) /.../include/boost/lockfree/stack.hpp:498 (bin+0x000000805797)
#1 bool boost::lockfree::stack<MYDATA*, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::pop<MYDATA*>(MYDATA_&) /.../include/boost/lockfree/stack.hpp:435 (bin+0x0000008056e2)
#2 boost::lockfree::stack<MYDATA_, boost::lockfree::capacity<4096ul>, boost::parameter::void_, boost::parameter::void_>::pop(MYDATA*&) /.../include/boost/lockfree/stack.hpp:417 (bin+0x0000008056a0)
Upon closer examine the offending line, I can see there is a compare_exchange_weak check immediately following it which throws away the result if the target is modified by other threads. However, the unprotected nonatomic r/w technically is still a data race in C++ language, which causes the behaviour of the program become undefined.
Using queue with an allocator, it is impossible to write code that invokes this template constructor:
template <typename U>
queue(size_type n, typename node_allocator::template rebind<U>::other const & alloc):
(or the other one with the rebind).
The template argument U cannot be deduced (at least with gcc 5.3.0).
I had to add the following form in order to use this with a custom allocator:
explicit queue(size_type n, allocator const & alloc)
I notice that the constructor that takes only an alloc
already has this overload.
Hi,
After upgrading to Boost 1.68.0, builds using lockfree fail on x86; x64 works.
Compiler version:
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x86
Copyright (C) Microsoft Corporation. All rights reserved.
On trying to compile the test cases, I get this error (complete log attached):
stack_bounded_stress_test.cpp
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(508): error C2719: '_Val': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(669): note: see reference to class template instantiation 'std::_Atomic_base<_Ty,8>' being compiled
with
[
_Ty=boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::stack<long>::node,std::allocator<boost::lockfree::stack<long>::node>>::freelist_node>
]
T:\boost_1_68_0\boost/lockfree/detail/freelist.hpp(255): note: see reference to class template instantiation 'std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<T,Alloc>::freelist_node>>' being compiled
with
[
T=boost::lockfree::stack<long>::node,
Alloc=std::allocator<boost::lockfree::stack<long>::node>
]
T:\boost_1_68_0\boost/lockfree/stack.hpp(100): note: see reference to class template instantiation 'boost::lockfree::detail::freelist_stack<T,Alloc>' being compiled
with
[
T=boost::lockfree::stack<long>::node,
Alloc=std::allocator<boost::lockfree::stack<long>::node>
]
stack_bounded_stress_test.cpp(25): note: see reference to class template instantiation 'boost::lockfree::stack<long>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(54): note: see reference to class template instantiation 'boost::arg<9>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(53): note: see reference to class template instantiation 'boost::arg<8>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(52): note: see reference to class template instantiation 'boost::arg<7>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(51): note: see reference to class template instantiation 'boost::arg<6>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(50): note: see reference to class template instantiation 'boost::arg<5>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(49): note: see reference to class template instantiation 'boost::arg<4>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(48): note: see reference to class template instantiation 'boost::arg<3>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(47): note: see reference to class template instantiation 'boost::arg<2>' being compiled
T:\boost_1_68_0\boost/bind/placeholders.hpp(46): note: see reference to class template instantiation 'boost::arg<1>' being compiled
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(519): error C2719: '_Right': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(525): error C2719: '_Right': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(542): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(548): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(580): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(587): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(594): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(602): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(610): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(617): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(624): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(632): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(640): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(647): error C2719: '_Value': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(680): error C2719: '_Val': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(686): error C2719: '_Right': formal parameter with requested alignment of 8 won't be aligned
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\atomic(691): error C2719: '_Right': formal parameter with requested alignment of 8 won't be aligned
Could you please have a look on how to fix this issue?
Simple usage of the queue triggers gcc's UBSAN checker flagging misaligned 64b accesses. Test case:
#include <iostream>
#include <boost/lockfree/queue.hpp>
int main()
{
boost::lockfree::queue<std::size_t> q(5);
q.push(1);
q.push(2);
q.push(3);
q.push(4);
q.push(5);
std::size_t top;
q.pop(top);
std::cout << "top of queue was " << top << "\n";
if (q.empty())
{
std::cout << "the queue is now empty (!?)\n";
}
q.consume_all(
[](std::size_t v)
{
std::cout << "next value " << v << "\n";
});
}
This change, 3d45c00, introduced in boost 1.71.0 to fix valgrind issues, actually introduced a very subtle ordering issue in the queue implementation which also exists in 1.72 and 1.73.
Specifically, the code previously did NOT initialize the tagged_node_handle
next
on construction. which works beautifully when a node is allocated/freed/allocated
- data(v)//, next(tagged_node_handle(0, 0))
+ data(v), next(tagged_node_handle(null_handle, 0))
By initializing the next pointer, the change introduces a classic ABA problem (from Wikipedia, "when a location is read twice, has the same value for both reads, and 'value is the same' is used to indicate 'nothing has changed'. However, another thread can execute between the two reads and change the value, do other work, then change the value back, thus fooling the first thread into thinking "nothing has changed" even though the second thread did work that violates that assumption.")
Reverting that single line fixes the issue in both 1.71 and 1.73
The following program demonstrates the problem where data can be returned from the queue out of order. Note that recreating this depends on system load and system architecture, but I have reliably recreated the problem on both server and desktop class systems by repeating this test 10,000 times.
// Boost queue stress test....all credits to Wes Darvin
#include <boost/lockfree/queue.hpp>
#include <iostream>
#include <thread>
#include <vector>
static constexpr int BLOCK_SIZE = 1000;
static constexpr int NUM_THREADS = 20;
// Enqueue BLOCK_SIZE values in order
void enqueueThreadMethod(boost::lockfree::queue<int32_t> *queue, int threadID) {
int start = BLOCK_SIZE * threadID;
for (int i = 0; i < BLOCK_SIZE; i++) {
while (!queue->push(start + i))
;
}
}
// Dequeue BLOCK_SIZE values. Note that the values returned may come from
// different enqueuers, but for each enqueuer they should be in increasing
// order.
void dequeueThreadMethod(
boost::lockfree::queue<int32_t> *queue,
std::array<int32_t, NUM_THREADS * BLOCK_SIZE> *retValues, int threadID) {
for (int i = 0; i < BLOCK_SIZE; i++) {
while (!queue->pop((*retValues)[threadID * BLOCK_SIZE + i]))
;
}
}
int main(int argc, char **argv) {
std::array<int32_t, NUM_THREADS * BLOCK_SIZE> retValues;
std::vector<std::thread> threadVector;
boost::lockfree::queue<int32_t> queue(100);
// Enqueue and dequeue relatively simulataniously. And yes, we could make
// this more robust by adding a barrier, but it works as-is
for (int i = 0; i < NUM_THREADS; i++) {
threadVector.emplace_back(&enqueueThreadMethod, &queue, i);
threadVector.emplace_back(&dequeueThreadMethod, &queue, &retValues, i);
}
for (std::thread &t : threadVector) {
t.join();
}
// Now validate the values we dequeued. All we are checking is
// that for each enqueuer, its values only increase...i.e. we
// are checking for out-of-order conditions
for (int i = 0; i < NUM_THREADS; i++) {
// vector of last-seen values by enqueuer. Each
// entry should increase
std::vector<int> localMax(NUM_THREADS, -1);
// check the values dequeued by a single dequeuer
for (int j = 0; j < BLOCK_SIZE; j++) {
int localValue = retValues[i * BLOCK_SIZE + j];
int index = localValue / BLOCK_SIZE;
if (localValue < localMax[index]) {
std::cout << "Out of order values: " << localValue << " < "
<< localMax[index] << " i: " << i << ", j: " << j
<< ", index: " << index << std::endl;
}
localMax[index] = localValue;
}
}
// Now sort all the dequeued values and make sure there are none missing
// or duplicated
std::sort(retValues.begin(), retValues.end());
for (int i = 0; i < (BLOCK_SIZE * NUM_THREADS); i++) {
if (retValues[i] != i) {
std::cout << "Error checking sorted values at index " << i
<< std::endl;
}
}
return 0;
}
The possible problems are in queue::do_push and queue::pop.
I'm not versed in the lingo of atomics and memory orderings, so my phrasing might be a bit off - but I'm quite sure my questions are correct.
(1) double loads of tail_ and head_ in do_push and pop
First let me mention that in the original algorithm by Scott and Michael these functions have a double load of tail and head (respectively), and then the values of the first and second load are compared.
The purpose of these are to make sure the node has not been freed (std::free()) already, and then the &(node->next) might become a non dereferenceable address, or else its value could be randomly changed to false-positively pass the tag check.
However in any case that the nodes are not freed but rather put in a free-list, we don’t have this problem at all.
The reason they work in the algorithm, is because the algorithm is sequentially consistent memory:
push: the node can be freed only after tail_ has been exchanged to a value further ahead in the list than the node itself. If the value is exchanged so in any thread, it would either be visible by the second load of tail_ or it would happen after it, due to sequential consistency.
pop: basically same explanation for head_
So, our queue is using weak memory ordering, therefore the above explanations do not apply.
In case the node is actually freed, rather than put in a free-list, we need some other way to make sure &(node->next) is actually still dereferenceable.
For example, make the second loads (of tail and head) a read-modify-write operation (not for the purpose of changing the value, but for the purpose of having a strong order of visibility with the actual value exchanges).
I know that after c++11, for 14/17/20 the relationships between seq_cst and weak order operations have been strengthened. Is this what is assumed here? That the seq_cst exchanges of tail_ and head_ are strongly ordered with these second loads?
(2) questionable memory ordering in do_push()
tagged_node_handle tail = tail_.load(memory_order_acquire);
node * tail_node = pool.get_pointer(tail);
tagged_node_handle next = tail_node->next.load(memory_order_acquire);
node * next_ptr = pool.get_pointer(next);
tagged_node_handle tail2 = tail_.load(memory_order_acquire);
(2.1)
So, why would the second load of tail_ be mo_acquire?!
What is it acquiring for?! There are no further loads after that.
If at all it is needed, it could just as well be mo_relaxed.
(2.2)
Why would tail_node->next.load() be mo_acquire?!
If it is acquiring for the second tail_.load(), it could only be if some other thread had a matching:
tail_.store()
tail_node->store( …, mo_release );
But this doesn’t exist anywhere.
The only node->next mo_release in the code is later in do_push() and there are no tail_.store() before it, only an tail_.exhange() after.
(3) questionable memory ordering in pop()
tagged_node_handle head = head_.load(memory_order_acquire);
node * head_ptr = pool.get_pointer(head);
tagged_node_handle tail = tail_.load(memory_order_acquire);
tagged_node_handle next = head_ptr->next.load(memory_order_acquire);
node * next_ptr = pool.get_pointer(next);
tagged_node_handle head2 = head_.load(memory_order_acquire);
(3.1)
Why would the second head_.load be mo_aquire?!
What is it acquiring for?
If at all needed, it could be mo_relaxed
(3.2)
Why does head_ptr->next.load() has mo_acquire?!
As if it’s acquiring for the second head_.load()
But there’s no matching release for this memory.
edit: However, it needs to be at least a mo_consume for the following reading of the payload.
(4) Seems to me, in order to make sense of 2.2 and 3.2 above, maybe add
head_ptr->next.store(..., mo_release )
inside pop() after the head.compare_exchange succeeded.
Then at least it would be releasing for the head exchange and also for tail exchange (because the tail exchange happens before the head exchange, in the same pop() or earlier).
(5) Why does pop() copy the payload before the head.compare_exchange every loop, instead of only once after it succeeds? This is a performance difference.
detail::copy_payload(next_ptr->data, ret);
tagged_node_handle new_head(pool.get_handle(next), head.get_next_tag());
if (head_.compare_exchange_weak(head, new_head)) {
...
We are in the process of making B2 build changes to all of the B2 build files
to support "modular" consumption of the Boost Libraries by users. See this list
post for some details: https://lists.boost.org/Archives/boost/2024/01/255704.php
The process requires making a variety of changes to make each Boost library
independent of the super-project structure. But the changes do not remove the
super-project structure or the comprehensive Boost release. The changes make
solely make it possible, optionally, for users, like package manages, to easily
consume libraries individually.
Generally the changes include:
Some examples of such changes:
We are asking how you would like us to handle the changes. We would prefer if
you allow the owners of the Boost.org GitHub project to make changes to B2
build files, as needed, to accomplish the changes. But understand
that you may want to manage the proposed changes yourself.
We previously sent emails to all known maintainers to fill out a form with their
preference. We are contacting you in this issue as we have not gotten a response
to that email. You can see the ongoing responses for that form and the responses
to these issues here https://github.com/users/grafikrobot/projects/1/views/6
We are now asking if you can reply directly to this issue to indicate your
preference of handling the changes. Please supply a response to this question
and close the issue (so that we can verify you are a maintainer).
How would you like the build changes to be processed?
Also please indicate any special instructions you want us to consider. Or other
information you want us to be aware of.
Thanks you, René
#include <boost/lockfree/detail/tagged_ptr_dcas.hpp>
int main() {}
/*
[gustavo@casa bug]$ g++ stack.cc -o stack
En el fichero incluido desde stack.cc:1:
/usr/include/boost/lockfree/detail/tagged_ptr_dcas.hpp:30:17: error: expected unqualified-id before numeric constant
30 | BOOST_ALIGNMENT(2 * sizeof(void*))
| ^
/usr/include/boost/lockfree/detail/tagged_ptr_dcas.hpp:30:17: error: expected ‘)’ before numeric constant
30 | BOOST_ALIGNMENT(2 * sizeof(void*))
| ~^
| )
*/
What is the use of such bitwise and? This seems to be a no-op for me. In first place, because overflow is well-defined for size_t
(wrapping). Doing x & ~(size_t)0
is always x
. In second place, because even if the overflow were not defined, the bitwise and would happen after the overflow, and UB would be invoked anyway.
When Lockfree is configured with cmake -DBUILD_TESTING=ON ..
, and then reconfigured with cmake -DBUILD_TESTING=OFF .
, tests remain enabled. That's because BUILD_TESTING
is only used once as the default value of BOOST_LOCKFREE_BUILD_TESTS
and is ignored afterwards.
That's not how all other Boost libraries work; they check BUILD_TESTING
directly and do not add_subdirectory(test)
if it's off.
Hi everyone,
Is there any obligation to make the Queue movable class? If there is not, I would like to contribute.
Thanks in advance
Hi,
Running the test stack_test on AIX, which is BigEndian, I see that it fails while running the test fixed_size_stack_test_exhausted .
../libs/lockfree/test/stack_test.cpp :
85 BOOST_AUTO_TEST_CASE( fixed_size_stack_test_exhausted )
86 {
87 boost::lockfree::stack<long, boost::lockfree::capacity<2> > stk;
88
89 stk.push(1);
90 stk.push(2);
....
The first stk.push(1) seems OK, but not really, since:
../boost/lockfree/detail/freelist.hpp:552
552 tagged_index new_pool(next_index->get_index(), old_pool.get_next_tag());
(gdb) whatis next_index
type = boost::lockfree::detail::tagged_index *
AIX:
(gdb) p * next_index
$12 = {index = 0, tag = 1}
Linux:
(gdb) p * next_index
$14 = {index = 0, tag = 65535}
Then, stk.push(2) does not put correct values in stk.
Main differences between Fedora/x86_64 and AIX after line stk.push(2) are:
On Fedora/x86_64:
stk = {static has_capacity = true, static capacity = 2, static fixed_sized = false, static node_based = false, static compile_time_sized = true,
tos = { static _S_min_alignment = 4, static _S_alignment = 4, _M_i = {index = 0, tag = 0}},
static padding_size = 60,
padding = ...........,
pool = {<boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<long, boost::lockfree::capacity<2> >::node, 2>>
= {data = {
elems = .......}},
pool_ = {static _S_min_alignment = 4, static _S_alignment = 4, _M_i = {index = 2, tag = 2}}}}
On AIX:
stk = {static has_capacity = true, static capacity = 2, static fixed_sized = false, static node_based = false, static compile_time_sized = true,
tos = { static _S_min_alignment = 4, static _S_alignment = 4, _M_i = {index = 0, tag = 0}},
static padding_size = 124,
padding = ........,
pool = {<boost::lockfree::detail::compiletime_sized_freelist_storage<boost::lockfree::stack<long, boost::lockfree::capacity<2> >::node, 2>>
= {data = { elems = ..........}},
pool_ = {static_S_min_alignment = 4, static _S_alignment = 4, _M_i = {index = 0, tag = 0}}}}
However, looking at details with gdb:
Fedora/x86_64:
(gdb) p & stk.pool.pool_._M_i
$54 = (boost::lockfree::detail::tagged_index *) 0x7fffffffc1c0
(gdb) x/2x 0x7fffffffc1c0
0x7fffffffc1c0: 0x00020002 0x00000000
AIX:
(gdb) p & stk.pool.pool_._M_i
$12 = (boost::lockfree::detail::tagged_index *) 0xfffffffffffdb08
(gdb) x/4x 0xfffffffffffdb08
0xfffffffffffdb08: 0x00000000 0x00000002 0x00020001 0x001baca8
So, it seems that the code does not write the 0x00020002 value at the right place on AIX.
This is probably due to the following code:
boost/lockfree/detail/freelist.hpp :
class
BOOST_ALIGNMENT( 4 ) // workaround for bugs in MSVC
tagged_index
{
public:
typedef boost::uint16_t tag_t;
typedef boost::uint16_t index_t;
...
protected:
index_t index;
tag_t tag;
};
since using 16bit has already shown within another library of Boost that it does not work fine in BigEndian.
Note: On Fedora/PPC64LE (PowerPC LittleEndian), this works fine.
These two tests queue_interprocess_test and stack_interprocess_test failed in clang on Windows for conflicting types.
Compilation command:
clang queue_interprocess_test.cpp -o queue_interprocess_test.obj -std=c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_ATOMIC_STATIC_LINK=1 -DBOOST_CHRONO_STATIC_LINK=1 -DBOOST_TEST_NO_AUTO_LINK=1 -DBOOST_THREAD_BUILD_LIB=1 -DBOOST_THREAD_USE_LIB=1 -DBOOST_THREAD_WIN32 -I"..\..\.."
Error message:
In file included from queue_interprocess_test.cpp:12:
In file included from ../../..\boost/thread/thread.hpp:12:
In file included from ../../..\boost/thread/thread_only.hpp:15:
In file included from ../../..\boost/thread/win32/thread_data.hpp:11:
In file included from ../../..\boost/thread/win32/thread_primitives.hpp:22:
../../..\boost/winapi/semaphore.hpp:25:1: error: conflicting types for
'CreateSemaphoreA'
CreateSemaphoreA(
^
../../..\boost/interprocess/detail/win32_api.hpp:926:51: note: previous
declaration is here
extern "C" __declspec(dllimport) void * __stdcall CreateSemaphoreA(inter...
^
cl from msvc will successfully compile this test.
cl queue_interprocess_test.cpp -TP /Z7 /Od /Ob0 /W3 /GR /MDd /Zc:forScope /Zc:wchar_t /favor:blend /wd4675 /EHs /std:c++17 -D_CRT_USE_BUILTIN_OFFSETOF -c -DBOOST_ALL_NO_LIB=1 -DBOOST_ATOMIC_STATIC_LINK=1 -DBOOST_CHRONO_STATIC_LINK=1 -DBOOST_TEST_NO_AUTO_LINK=1 -DBOOST_THREAD_BUILD_LIB=1 -DBOOST_THREAD_USE_LIB=1 -DBOOST_THREAD_WIN32 "-I..\..\.."
I am still investigating the reason. If you know why, please let me know. Thanks!
In theory, it is a queue without waiting, which supports multi-threaded reading and writing. Of course, the disadvantage is that the order of entering the queue may not be strongly ordered.
template<typename T, size_t N>
class queue
{
private:
ring_buffer<T, N> value_;
ring_buffer<atomic<size_t>, N> readable_flag_{0};
ring_buffer<atomic<size_t>, N> writable_flag_{0};
atomic<size_t> writable_limit_;
atomic<size_t> readable_limit_;
public:
queue() = default;
~queue() = default;
void push(const T & val)
{
size_t index = writable_limit_.fetch_add(1);
while (writable_flag_[index] != index / max_size())
;
value_[index] = val;
readable_flag_[index] = (index / max_size()) + 1;
}
void pop(T&val)
{
size_t index = readable_limit_.fetch_add(1);
while(readable_flag_[index] != (index / max_size()) + 1)
;
val = value_[index];
writable_flag_[index] = (index / max_size()) + 1;
}
size_t max_size() const
{
return N;
}
size_t size() const
{
size_t writable_limit = writable_limit_;
size_t readable_limit = readable_limit_;
return writable_limit > readable_limit ? writable_limit - readable_limit : 0;
}
};
The underlying issue with the VS2017 compiler is shown here on godbolt.
The error was triggered in our code base (which works fine with boost-1.76) where we have boost::lockfree::queue in classes managed by shared_ptr. It is probably due to this commit.
Note that boost::lockfree::queue<int, boost::lockfree::capacity<N>>
, which is already pretty large, increases by 60 bytes.
Of course, there is an easy fix by defining _ENABLE_EXTENDED_ALIGNED_STORAGE
, but there is a reason they did not enable this by default.
This issue is obviously not a deal breaker but it will make upgrading boost from 1.76 to 1.77 troublesome for many users.
Motivation for adding this ticket: Just for the record, in case anyone else searches for a solution to this issues.
Problem: At least on MacOS 14, creating 16 lockfree:stack instances sequentially takes more than 10 seconds(!) on an M3 macbook.
A solution:
#define NUM_POOLS 16
#define POOL boost::lockfree::stack<RT_mempool_data*>
+ std::thread t[NUM_POOLS];
+
+ for(int i=0;i<NUM_POOLS;i++)
+ {
+ // Allocate each pool in it's own thread. Workaround for really slow memory allocation performance on Macos.
+ t[i] = std::thread([i](){
+ g_pools[i] = new POOL(MAX_POOL_SIZE);
+ });
+ }
+
for(int i=0;i<NUM_POOLS;i++)
- g_pools[i] = new POOL(MAX_POOL_SIZE);
+ t[i].join();
According to the boost::lockfree documentation and from what I can see from boost/lockfree/detail/tagged_ptr_ptrcompression.hpp, the implementation of the free-list stores a 16-bit tag value in the upper bits of a 64-bit pointer.
Most recent Intel x86_48 microarchs (Ice Lake) use 5-level page tables and 57-bit virtual addresses, meaning this technique of pointer tagging can fail unexpectedly when used on such a processor.
The existence of <atomic>
not only depends on the version of compiler, but also the version of standard library.
When detail/atomic.hpp
was created in this project, there was no BOOST_NO_CXX11_HDR_ATOMIC
(which was introduced around Sep, 2014?) in boost::config
. Now BOOST_NO_CXX11_HDR_ATOMIC
has been well tested and covers far more compilers than BOOST_LOCKFREE_NO_HDR_ATOMIC
. So I think it's time to switch to BOOST_NO_CXX11_HDR_ATOMIC
.
The constructors for queue
do not document their requirements with respect to the capacity
tag.
Moreover in certain constructors a runtime assert (only present in debug mode) are used to verify the requirement. It shouldn't be necessary to build code in debug mode to discover what is logically a compile time constraint.
If there are multiple consumers, there will be tsan data race warnings. Here is the code to reproduce:
> cat lockfree_race.cc; echo ==================; clang++-12 lockfree_race.cc -std=c++20 -fsanitize="thread" -o lockfree_race; ./lockfree_race
#include <boost/version.hpp>
#include <boost/lockfree/queue.hpp>
#include <iostream>
#include <thread>
int main() {
std::cout << BOOST_VERSION << std::endl;
const int count = 1000000;
boost::lockfree::queue<int> queue(10);
auto producer = [&queue](){
for (int i = 0; i < count; ++i) {
queue.push(i);
}
};
auto consumer = [&queue](){
for (int i = 0; i < count; ++i) {
int val;
queue.pop(val);
}
};
std::vector<std::thread> t;
t.emplace_back(producer);
t.emplace_back(consumer);
t.emplace_back(consumer);
for (auto& tt : t) {
tt.join();
}
return 0;
}
==================
107100
================== [127/1026]
WARNING: ThreadSanitizer: data race (pid=682920)
Write of size 8 at 0x7b10000087c0 by thread T2:
#0 boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::freelist_node>::set_ptr(boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::freelist_node*) <n
ull> (lockfree_race+0x4bc1b3)
#1 boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::deallocate_impl(boost::lockfree::queue<int>::node*) <null> (lockfree_race+0x4c0cc6)
#2 void boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::deallocate<true>(boost::lockfree::queue<int>::node*) <null> (lockfree_race+0x4c0bd5)
#3 void boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::destruct<true>(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> const&) <null> (lockfree_race+0x4c0b7a)
#4 bool boost::lockfree::queue<int>::pop<int>(int&) <null> (lockfree_race+0x4c0aba)
#5 boost::lockfree::queue<int>::pop(int&) <null> (lockfree_race+0x4c07c5)
#6 main::$_1::operator()() const <null> (lockfree_race+0x4bb021)
#7 void std::__invoke_impl<void, main::$_1>(std::__invoke_other, main::$_1&&) <null> (lockfree_race+0x4baf9d)
#8 std::__invoke_result<main::$_1>::type std::__invoke<main::$_1>(main::$_1&&) <null> (lockfree_race+0x4baeed)
#9 void std::thread::_Invoker<std::tuple<main::$_1> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4bae95)
#10 std::thread::_Invoker<std::tuple<main::$_1> >::operator()() <null> (lockfree_race+0x4bae35)
#11 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_1> > >::_M_run() <null> (lockfree_race+0x4bad29)
#12 <null> <null> (libstdc++.so.6+0xda6b3)
Previous atomic read of size 8 at 0x7b10000087c0 by thread T3:
#0 __tsan_atomic64_load <null> (lockfree_race+0x4717ce)
#1 std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::load(std::memory_order) const <null> (lockfree_race+0x4bd37d)
#2 bool boost::lockfree::queue<int>::pop<int>(int&) <null> (lockfree_race+0x4c089b)
#3 boost::lockfree::queue<int>::pop(int&) <null> (lockfree_race+0x4c07c5)
#4 main::$_1::operator()() const <null> (lockfree_race+0x4bb021)
#5 void std::__invoke_impl<void, main::$_1>(std::__invoke_other, main::$_1&&) <null> (lockfree_race+0x4baf9d)
#6 std::__invoke_result<main::$_1>::type std::__invoke<main::$_1>(main::$_1&&) <null> (lockfree_race+0x4baeed)
#7 void std::thread::_Invoker<std::tuple<main::$_1> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4bae95)
#8 std::thread::_Invoker<std::tuple<main::$_1> >::operator()() <null> (lockfree_race+0x4bae35)
#9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_1> > >::_M_run() <null> (lockfree_race+0x4bad29)
#10 <null> <null> (libstdc++.so.6+0xda6b3)
Location is heap block of size 64 at 0x7b10000087c0 allocated by thread T1:
#0 operator new(unsigned long, std::align_val_t) <null> (lockfree_race+0x4b72fa)
#1 __gnu_cxx::new_allocator<boost::lockfree::queue<int>::node>::allocate(unsigned long, void const*) <null> (lockfree_race+0x4bbe18)
#2 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate_impl<false>() <null> (lockfree_race+0x4bc6f9)
#3 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate<true, false>() <null> (lockfree_race+0x4bc5d5)
#4 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be550)
#5 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)
#6 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)
#7 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)
#8 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)
#9 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)
#10 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)
#11 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)
#12 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)
#13 <null> <null> (libstdc++.so.6+0xda6b3)
Thread T2 (tid=682923, running) created by main thread at:
#0 pthread_create <null> (lockfree_race+0x425beb)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
#2 decltype(new ((void*)(0))std::thread(std::declval<main::$_1&>())) std::construct_at<std::thread, main::$_1&>(std::thread*, main::$_1&) <null> (lockfree_race+0x4ba9c5)
#3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_1&>(std::allocator<std::thread>&, std::thread*, main::$_1&) <null> (lockfree_race+0x4ba584)
#4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_1&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_1&) <null> (lockfree_race+0x4ba6c9)
#5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_1&>(main::$_1&) <null> (lockfree_race+0x4b98ba)
#6 main <null> (lockfree_race+0x4b952a)
Thread T3 (tid=682924, running) created by main thread at:
#0 pthread_create <null> (lockfree_race+0x425beb)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
#2 decltype(new ((void*)(0))std::thread(std::declval<main::$_1&>())) std::construct_at<std::thread, main::$_1&>(std::thread*, main::$_1&) <null> (lockfree_race+0x4ba9c5)
#3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_1&>(std::allocator<std::thread>&, std::thread*, main::$_1&) <null> (lockfree_race+0x4ba584)
#4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_1&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_1&) <null> (lockfree_race+0x4ba6c9)
#5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_1&>(main::$_1&) <null> (lockfree_race+0x4b98ba)
#6 main <null> (lockfree_race+0x4b9542)
Thread T1 (tid=682922, running) created by main thread at:
#0 pthread_create <null> (lockfree_race+0x425beb)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
#2 decltype(new ((void*)(0))std::thread(std::declval<main::$_0&>())) std::construct_at<std::thread, main::$_0&>(std::thread*, main::$_0&) <null> (lockfree_race+0x4b9d95)
#3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0&>(std::allocator<std::thread>&, std::thread*, main::$_0&) <null> (lockfree_race+0x4b9954)
#4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_0&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_0&) <null> (lockfree_race+0x4b9a99)
#5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0&>(main::$_0&) <null> (lockfree_race+0x4b978a)
#6 main <null> (lockfree_race+0x4b9512)
SUMMARY: ThreadSanitizer: data race (/home/chenhao/lockfree_race+0x4bc1b3) in boost::lockfree::detail::tagged_ptr<boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::freelist_node>::set_ptr(boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node,
std::allocator<boost::lockfree::queue<int>::node> >::freelist_node*)
==================
==================
WARNING: ThreadSanitizer: data race (pid=682920)
Write of size 4 at 0x7b100000ce48 by thread T1:
#0 boost::lockfree::queue<int>::node::node(int const&, boost::lockfree::queue<int>::node*) <null> (lockfree_race+0x4be835)
#1 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be5a0)
#2 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)
#3 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)
#4 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)
#5 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)
#6 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)
#7 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)
#8 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)
#9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)
#10 <null> <null> (libstdc++.so.6+0xda6b3)
Previous read of size 4 at 0x7b100000ce48 by thread T3:
#0 void boost::lockfree::detail::copy_convertible::copy<int, int>(int&, int&) <null> (lockfree_race+0x4bd9c5)
#1 void boost::lockfree::detail::copy_payload<int, int>(int&, int&) <null> (lockfree_race+0x4bd845)
#2 bool boost::lockfree::queue<int>::pop<int>(int&) <null> (lockfree_race+0x4c0a18)
#3 boost::lockfree::queue<int>::pop(int&) <null> (lockfree_race+0x4c07c5)
#4 main::$_1::operator()() const <null> (lockfree_race+0x4bb021)
#5 void std::__invoke_impl<void, main::$_1>(std::__invoke_other, main::$_1&&) <null> (lockfree_race+0x4baf9d)
#6 std::__invoke_result<main::$_1>::type std::__invoke<main::$_1>(main::$_1&&) <null> (lockfree_race+0x4baeed)
#7 void std::thread::_Invoker<std::tuple<main::$_1> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4bae95)
#8 std::thread::_Invoker<std::tuple<main::$_1> >::operator()() <null> (lockfree_race+0x4bae35)
#9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_1> > >::_M_run() <null> (lockfree_race+0x4bad29)
#10 <null> <null> (libstdc++.so.6+0xda6b3)
Location is heap block of size 64 at 0x7b100000ce40 allocated by thread T1:
#0 operator new(unsigned long, std::align_val_t) <null> (lockfree_race+0x4b72fa)
#1 __gnu_cxx::new_allocator<boost::lockfree::queue<int>::node>::allocate(unsigned long, void const*) <null> (lockfree_race+0x4bbe18)
#2 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate_impl<false>() <null> (lockfree_race+0x4bc6f9)
#3 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate<true, false>() <null> (lockfree_race+0x4bc5d5)
#4 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be550)
#5 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)
#6 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)
#7 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)
#8 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)
#9 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)
#10 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)
#11 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)
#12 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)
#13 <null> <null> (libstdc++.so.6+0xda6b3)
Thread T1 (tid=682922, running) created by main thread at:
#0 pthread_create <null> (lockfree_race+0x425beb)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
#2 decltype(new ((void*)(0))std::thread(std::declval<main::$_0&>())) std::construct_at<std::thread, main::$_0&>(std::thread*, main::$_0&) <null> (lockfree_race+0x4b9d95)
#3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0&>(std::allocator<std::thread>&, std::thread*, main::$_0&) <null> (lockfree_race+0x4b9954)
#4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_0&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_0&) <null> (lockfree_race+0x4b9a99)
#5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0&>(main::$_0&) <null> (lockfree_race+0x4b978a)
#6 main <null> (lockfree_race+0x4b9512)
Thread T3 (tid=682924, running) created by main thread at:
#0 pthread_create <null> (lockfree_race+0x425beb)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
#2 decltype(new ((void*)(0))std::thread(std::declval<main::$_1&>())) std::construct_at<std::thread, main::$_1&>(std::thread*, main::$_1&) <null> (lockfree_race+0x4ba9c5)
#3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_1&>(std::allocator<std::thread>&, std::thread*, main::$_1&) <null> (lockfree_race+0x4ba584)
#4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_1&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_1&) <null> (lockfree_race+0x4ba6c9)
#5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_1&>(main::$_1&) <null> (lockfree_race+0x4b98ba)
#6 main <null> (lockfree_race+0x4b9542)
SUMMARY: ThreadSanitizer: data race (/home/chenhao/lockfree_race+0x4be835) in boost::lockfree::queue<int>::node::node(int const&, boost::lockfree::queue<int>::node*)
==================
==================
WARNING: ThreadSanitizer: data race (pid=682920)
Write of size 8 at 0x7b100000d940 by thread T1:
#0 memcpy <null> (lockfree_race+0x42fcae)
#1 std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::atomic(boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node>) <null> (lockfree_race+0x4bb81a)
#2 boost::lockfree::queue<int>::node::node(int const&, boost::lockfree::queue<int>::node*) <null> (lockfree_race+0x4be804)
#3 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be5a0)
#4 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)
#5 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)
#6 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)
#7 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)
#8 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)
#9 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)
#10 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)
#11 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)
#12 <null> <null> (libstdc++.so.6+0xda6b3)
Previous atomic read of size 8 at 0x7b100000d940 by thread T3:
#0 __tsan_atomic64_load <null> (lockfree_race+0x4717ce)
#1 std::atomic<boost::lockfree::detail::tagged_ptr<boost::lockfree::queue<int>::node> >::load(std::memory_order) const <null> (lockfree_race+0x4bd37d)
#2 bool boost::lockfree::queue<int>::pop<int>(int&) <null> (lockfree_race+0x4c089b)
#3 boost::lockfree::queue<int>::pop(int&) <null> (lockfree_race+0x4c07c5)
#4 main::$_1::operator()() const <null> (lockfree_race+0x4bb021)
#5 void std::__invoke_impl<void, main::$_1>(std::__invoke_other, main::$_1&&) <null> (lockfree_race+0x4baf9d)
#6 std::__invoke_result<main::$_1>::type std::__invoke<main::$_1>(main::$_1&&) <null> (lockfree_race+0x4baeed)
#7 void std::thread::_Invoker<std::tuple<main::$_1> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4bae95)
#8 std::thread::_Invoker<std::tuple<main::$_1> >::operator()() <null> (lockfree_race+0x4bae35)
#9 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_1> > >::_M_run() <null> (lockfree_race+0x4bad29)
#10 <null> <null> (libstdc++.so.6+0xda6b3)
Location is heap block of size 64 at 0x7b100000d940 allocated by thread T1:
#0 operator new(unsigned long, std::align_val_t) <null> (lockfree_race+0x4b72fa)
#1 __gnu_cxx::new_allocator<boost::lockfree::queue<int>::node>::allocate(unsigned long, void const*) <null> (lockfree_race+0x4bbe18)
#2 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate_impl<false>() <null> (lockfree_race+0x4bc6f9)
#3 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::allocate<true, false>() <null> (lockfree_race+0x4bc5d5)
#4 boost::lockfree::queue<int>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<int>::node, std::allocator<boost::lockfree::queue<int>::node> >::construct<true, false, int, boost::lockfree::queue<int>::node*>(int const&, boost::lockfree::queue<int>::node* const&) <null> (lockfree_race+0x4be550)
#5 bool boost::lockfree::queue<int>::do_push<false>(int const&) <null> (lockfree_race+0x4be209)
#6 boost::lockfree::queue<int>::push(int const&) <null> (lockfree_race+0x4be135)
#7 main::$_0::operator()() const <null> (lockfree_race+0x4ba403)
#8 void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) <null> (lockfree_race+0x4ba36d)
#9 std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) <null> (lockfree_race+0x4ba2bd)
#10 void std::thread::_Invoker<std::tuple<main::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (lockfree_race+0x4ba265)
#11 std::thread::_Invoker<std::tuple<main::$_0> >::operator()() <null> (lockfree_race+0x4ba205)
#12 std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() <null> (lockfree_race+0x4ba0f9)
#13 <null> <null> (libstdc++.so.6+0xda6b3)
Thread T1 (tid=682922, running) created by main thread at:
#0 pthread_create <null> (lockfree_race+0x425beb)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
#2 decltype(new ((void*)(0))std::thread(std::declval<main::$_0&>())) std::construct_at<std::thread, main::$_0&>(std::thread*, main::$_0&) <null> (lockfree_race+0x4b9d95)
#3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0&>(std::allocator<std::thread>&, std::thread*, main::$_0&) <null> (lockfree_race+0x4b9954)
#4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_0&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_0&) <null> (lockfree_race+0x4b9a99)
#5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0&>(main::$_0&) <null> (lockfree_race+0x4b978a)
#6 main <null> (lockfree_race+0x4b9512)
Thread T3 (tid=682924, running) created by main thread at:
#0 pthread_create <null> (lockfree_race+0x425beb)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xda989)
#2 decltype(new ((void*)(0))std::thread(std::declval<main::$_1&>())) std::construct_at<std::thread, main::$_1&>(std::thread*, main::$_1&) <null> (lockfree_race+0x4ba9c5)
#3 void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_1&>(std::allocator<std::thread>&, std::thread*, main::$_1&) <null> (lockfree_race+0x4ba584)
#4 void std::vector<std::thread, std::allocator<std::thread> >::_M_realloc_insert<main::$_1&>(__gnu_cxx::__normal_iterator<std::thread*, std::vector<std::thread, std::allocator<std::thread> > >, main::$_1&) <null> (lockfree_race+0x4ba6c9)
#5 std::thread& std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_1&>(main::$_1&) <null> (lockfree_race+0x4b98ba)
#6 main <null> (lockfree_race+0x4b9542)
SUMMARY: ThreadSanitizer: data race (/home/chenhao/lockfree_race+0x42fcae) in memcpy
==================
Environment:
> clang++-12 --version
Ubuntu clang version 12.0.0-3ubuntu1~20.04.5
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
> uname -a
Linux chenhao 5.4.0-109-generic #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
version 1.73 boost::lockfree::queue crash under android 12 when destruct?
eg.
{
boost::lockfree::queue<int, boost::fixed_sized > test_que(10);
}
it does nothing else, it crash backtrace like this:
boost::lockfree::queue<int, boost::fixed_sized >::~queue()+272
the same apk in not android 12 phone, it works ok, but android 12
what happened?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.