inkooboo / thread-pool-cpp

High performance C++11 thread pool

thread-pool-cpp

• It is highly scalable and fast.
• It is header only.
• No external dependencies, only standard library needed.
• It implements both work-stealing and work-distribution balancing startegies.
• It implements cooperative scheduling strategy.

Example run: Post job to thread pool is much faster than for boost::asio based thread pool.

Benchmark job reposting
***thread pool cpp***
reposted 1000001 in 61.6754 ms
reposted 1000001 in 62.0187 ms
reposted 1000001 in 62.8785 ms
reposted 1000001 in 70.2714 ms
***asio thread pool***
reposted 1000001 in 1381.58 ms
reposted 1000001 in 1390.35 ms
reposted 1000001 in 1391.84 ms
reposted 1000001 in 1393.19 ms

See benchmark/benchmark.cpp for benchmark code.

All code except MPMCBoundedQueue is under MIT license.

thread-pool-cpp's Issues

Difference between packaged task and fixed function

Hi, I couldn't find any documentation related to the project so I looked into tests and found 2 different ways to send tasks to thread pool if I'm guessing right. So what's the difference b/w using fixed function vs using packaged task here?

Question - Sample & clear example

Dear Andrey

do you have any manual or sample for the usage of your thread pool implementation?
I can not find anything here, about how can i use it or the architecture and options that exist in your pool and what is the exact behavior of all these things.

Regards,

change the implementation of thread_pool_options to a .cpp file

I need to use the ThreadPoolOptions to set the thread count and the queue size. Since the ThreadPool's ctor only accepts the ThreadPoolOptions, I met an "multiple definition of" error when linking in my program.
May you remove the implementation of ThreadPoolOptions to antoher .cpp file or add a ctor in ThreadPool to control the queue size and thread count?

Deadlock on dll/exe load when using tp::ThreadPool as global variable

I tried to declare the thread pool in global scope like this:

tp::ThreadPool g_threadPool;

The process never reaches the main function.

link error:fixed_function undefined reference

OS:centos 7
g++ version: 8.3.1 set -std=c++17
gtest installed by such way:sudo yum install epel-release
sudo yum install dnf
sudo dnf install dnf-plugins-core
sudo dnf install gtest gtest-devel
it throw link error:
[root@DESKTOP-M5G7V2R thread-pool-cpp]# mkdir build
[root@DESKTOP-M5G7V2R thread-pool-cpp]# cd build/; cmake -DCMAKE_BUILD_TYPE=Release ..;clear ;make VERBOSE=1
-- The CXX compiler identification is GNU 8.3.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done (1.2s)
-- Generating done (0.0s)
/usr/local/bin/cmake -S/home/thread-pool-cpp -B/home/thread-pool-cpp/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/bin/cmake -E cmake_progress_start /home/thread-pool-cpp/build/CMakeFiles /home/thread-pool-cpp/build//CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory /home/thread-pool-cpp/build' make -f CMakeFiles/HEADER_ONLY_TARGET.dir/build.make CMakeFiles/HEADER_ONLY_TARGET.dir/depend make[2]: Entering directory /home/thread-pool-cpp/build'
cd /home/thread-pool-cpp/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /home/thread-pool-cpp /home/thread-pool-cpp /home/thread-pool-cpp/build /home/thread-pool-cpp/build /home/thread-pool-cpp/build/CMakeFiles/HEADER_ONLY_TARGET.dir/DependInfo.cmake "--color="
make[2]: Leaving directory /home/thread-pool-cpp/build' make -f CMakeFiles/HEADER_ONLY_TARGET.dir/build.make CMakeFiles/HEADER_ONLY_TARGET.dir/build make[2]: Entering directory /home/thread-pool-cpp/build'
[ 11%] Linking CXX static library libHEADER_ONLY_TARGET.a
/usr/local/bin/cmake -P CMakeFiles/HEADER_ONLY_TARGET.dir/cmake_clean_target.cmake
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/HEADER_ONLY_TARGET.dir/link.txt --verbose=1
/usr/bin/ar qc libHEADER_ONLY_TARGET.a
/usr/bin/ranlib libHEADER_ONLY_TARGET.a
make[2]: Leaving directory /home/thread-pool-cpp/build' [ 11%] Built target HEADER_ONLY_TARGET make -f tests/CMakeFiles/fixed_function_test.dir/build.make tests/CMakeFiles/fixed_function_test.dir/depend make[2]: Entering directory /home/thread-pool-cpp/build'
cd /home/thread-pool-cpp/build && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /home/thread-pool-cpp /home/thread-pool-cpp/tests /home/thread-pool-cpp/build /home/thread-pool-cpp/build/tests /home/thread-pool-cpp/build/tests/CMakeFiles/fixed_function_test.dir/DependInfo.cmake "--color="
make[2]: Leaving directory /home/thread-pool-cpp/build' make -f tests/CMakeFiles/fixed_function_test.dir/build.make tests/CMakeFiles/fixed_function_test.dir/build make[2]: Entering directory /home/thread-pool-cpp/build'
[ 22%] Building CXX object tests/CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o
cd /home/thread-pool-cpp/build/tests && /usr/bin/g++ -I/usr/local/include/gtest -I/home/thread-pool-cpp/include -I/home/thread-pool-cpp/tests -std=c++17 -Wall -Wextra -lgtest -O3 -DNDEBUG -MD -MT tests/CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o -MF CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o.d -o CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o -c /home/thread-pool-cpp/tests/fixed_function.t.cpp
[ 33%] Linking CXX executable fixed_function_test
cd /home/thread-pool-cpp/build/tests && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/fixed_function_test.dir/link.txt --verbose=1
/usr/bin/g++ -std=c++17 -Wall -Wextra -lgtest -O3 -DNDEBUG CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o -o fixed_function_test -lpthread -lgtest -lgtest_main
CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o: In function FixedFunction_allocDealloc_Test::TestBody()': fixed_function.t.cpp:(.text+0x15de): undefined reference to testing::internal::EqFailure(char const*, char const*, std::string const&, std::string const&, bool)'
CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o: In function _GLOBAL__sub_I__Z14test_free_funci': fixed_function.t.cpp:(.text.startup+0x1fa): undefined reference to testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, testing::internal::CodeLocation, void const*, void ()(), void ()(), testing::internal::TestFactoryBase*)'
fixed_function.t.cpp:(.text.startup+0x2c0): undefined reference to testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, testing::internal::CodeLocation, void const*, void (*)(), void (*)(), testing::internal::TestFactoryBase*)' fixed_function.t.cpp:(.text.startup+0x386): undefined reference to testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, testing::internal::CodeLocation, void const*, void ()(), void ()(), testing::internal::TestFactoryBase*)'
fixed_function.t.cpp:(.text.startup+0x44c): undefined reference to testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, testing::internal::CodeLocation, void const*, void (*)(), void (*)(), testing::internal::TestFactoryBase*)' fixed_function.t.cpp:(.text.startup+0x512): undefined reference to testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, testing::internal::CodeLocation, void const*, void ()(), void ()(), testing::internal::TestFactoryBase*)'
CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o:fixed_function.t.cpp:(.text.startup+0x5d8): more undefined references to testing::internal::MakeAndRegisterTestInfo(char const*, char const*, char const*, char const*, testing::internal::CodeLocation, void const*, void (*)(), void (*)(), testing::internal::TestFactoryBase*)' follow CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o: In function testing::AssertionResult testing::internal::CmpHelperEQ<std::string, std::string>(char const*, char const*, std::string const&, std::string const&)':
fixed_function.t.cpp:(.text.ZN7testing8internal11CmpHelperEQISsSsEENS_15AssertionResultEPKcS4_RKT_RKT0[ZN7testing8internal11CmpHelperEQISsSsEENS_15AssertionResultEPKcS4_RKT_RKT0]+0x57): undefined reference to testing::internal::EqFailure(char const*, char const*, std::string const&, std::string const&, bool)' CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o: In function testing::AssertionResult testing::internal::CmpHelperEQ<int, unsigned long>(char const*, char const*, int const&, unsigned long const&)':
fixed_function.t.cpp:(.text.ZN7testing8internal11CmpHelperEQIimEENS_15AssertionResultEPKcS4_RKT_RKT0[ZN7testing8internal11CmpHelperEQIimEENS_15AssertionResultEPKcS4_RKT_RKT0]+0x4b): undefined reference to testing::internal::EqFailure(char const*, char const*, std::string const&, std::string const&, bool)' CMakeFiles/fixed_function_test.dir/fixed_function.t.cpp.o: In function testing::AssertionResult testing::internal::CmpHelperEQ<int, int>(char const*, char const*, int const&, int const&)':
fixed_function.t.cpp:(.text.ZN7testing8internal11CmpHelperEQIiiEENS_15AssertionResultEPKcS4_RKT_RKT0[ZN7testing8internal11CmpHelperEQIiiEENS_15AssertionResultEPKcS4_RKT_RKT0]+0x4a): undefined reference to testing::internal::EqFailure(char const*, char const*, std::string const&, std::string const&, bool)' collect2: error: ld returned 1 exit status make[2]: *** [tests/fixed_function_test] Error 1 make[2]: Leaving directory /home/thread-pool-cpp/build'
make[1]: *** [tests/CMakeFiles/fixed_function_test.dir/all] Error 2
make[1]: Leaving directory `/home/thread-pool-cpp/build'
make: *** [all] Error 2

Thread start/end lambdas

I have a use-case where I need to invoke code on each of the worker threads on thread creation and shutdown.

I have already written the code in my fork and would like to PR it here if you want the feature.

The use case is for running Mozilla's SpiderMonkey JavaScript engine in the thread pool. SpiderMonkey uses TLS so the engine must run on the thread that created it.

The new signature for threadFunc looks like:

inline void Worker::threadFunc(size_t id, Worker *steal_donor, OnStart onStart, OnStop onStop)

In addition each handler() takes the thread 'id' as the first param, this allows the handler to locate the correct engine, eg:

void myHandler(size_t id) {
    auto& engine = engines[id];
    engine.Evaluate("(function() { return 42; })()"); // run some JS
}

My change is based on an old commit on your master. I've raised this as an issue now since it will take me some time to merge it and I'd rather not bother if you don't want it.

Why does the function with arguments have errors?

error C2660 “std::packaged_task<int (int)>::operator ()”: The function does not take zero arguments

#include "thread_pool.hpp"
#include  "thread"
#include "future"
#include "functional"
#include "memory"
#include "iostream"
using namespace std;

int f(int j)
{
	return j;
}

int main(int argc, char **argv)
{
	tp::ThreadPool pool;
	//
	std::packaged_task<int(int)> t(f);
	std::future<int> r = t.get_future();
	pool.post(t);
	r.get();
	//while (1);
	getchar();
	return 0;
}

The thread pool can't drain

I wan't drain all the task, but can't find the method, could you add the drain method?

Wrong sequence Destructor / Constructor and task is not processed

Hi there,

I'm trying to use this thread-pool "LIB" and before using it, I wanted to make some stress tests on it to identify its limit, its hardening and reliability.

I wrote a really short main function and set a small amount of thread (1) with also a small file queue size (2). The behavior I can see is really strange : Some destructor may be called before constructor :huh: and the same input may be treated by several tasks

Here is the small main.cpp :

#include <iostream>
#include <regex>

#include <vector>
#include <signal.h>
#include <unistd.h>
// #include "logger.h"

#include <future>
#include <utility>

#include "thread_pool.hpp"

static volatile bool         g_theEnd = false ;

#define DEBUG(...)   { printf(__VA_ARGS__) ; printf("\n") ; }
#define INFO(...)    { printf(__VA_ARGS__) ; printf("\n") ; }
#define WARNING(...) { printf(__VA_ARGS__) ; printf("\n") ; }
#define ERROR(...)   { printf(__VA_ARGS__) ; printf("\n") ; }

#define LOG_INIT(a)
#define SET_LOG_LEVEL(a)
#define LOG_END()

//
//
//
void cleanup(void)
{
    // Ask the threads to give up
    g_theEnd = true ;

    INFO("Stop test-thread-pool") ;
    LOG_END() ;
}

//
//
//
class NgapMessageDecode
{
public:
    NgapMessageDecode(int fd) : m_fd(fd)
    {
        DEBUG("Constructor %.8ld, fd=%.2d", this, fd) ;
    }

    virtual ~NgapMessageDecode()
    {
        DEBUG("Destructor  %.8ld, fd=%.2d", this, m_fd) ;
    }

    void operator()()
    {
        DEBUG("Decode %.2d, this=%ld, thread=%.8ld", m_fd, this, pthread_self()) ;
        // sleep(1) ;
    }

private:
    // std::promise<void> *    m_waiter ;
    int                     m_fd ;
};


int main(int argc, char * argv[])
{
    LOG_INIT("test-thread-pool") ;
    SET_LOG_LEVEL(_LOG_DEBUG) ;

    INFO("%s", "") ;
    INFO("Start test-thread-pool") ;

    tp::ThreadPoolOptions   threadPoolOption ;

    threadPoolOption.setThreadCount(1) ;
    threadPoolOption.setQueueSize(2) ;

    tp::ThreadPool  threadPool(threadPoolOption);

    for(int i=0; i<100; i++)
    {
        try
        {
            threadPool.post(NgapMessageDecode(i)) ;
        }
        catch(std::runtime_error & e)
        {
            std::cout << e.what() << std::endl ;
        }
    }
    sleep(1) ;
    cleanup() ;
    return 0 ;
}

The output is :

Start test-thread-pool
Constructor 140734462742560, fd=00
Destructor  140734462742176, fd=00
Destructor  140734462742560, fd=00
Constructor 140734462742560, fd=01
Destructor  140734462742176, fd=01
Destructor  140734462742560, fd=01
Constructor 140734462742560, fd=02
Destructor  140734462742560, fd=02
thread pool queue is full
Constructor 140734462742560, fd=03
Destructor  140734462742560, fd=03
thread pool queue is full
Constructor 140734462742560, fd=04
Destructor  140734462742560, fd=04
thread pool queue is full

How can the Destructor 140734462742176, fd=00 (line2) can be called before the constructor ?

More over, this same Destructor refers to the object where fd==00 whereas the corresponding constructor below refers to fd==01

Thus, how can I got the sequence described in the 3 first lines : Constructor / Destructor / Destructor with the same FD==00

So, I probably missed something or at least misunderstood how I'm supposed to use this "LIB" but I can't find out what's the good practice :/

Branch used : master
I can't use the branch round-robin-stealing 'cause it refers to std::exchange that is defined in C++14 and I must stay in C++11

Command to build the main application listed above, to facilitate :

g++ -c   -DLOG -I<path_to_thread-pool-cpp>/include/thread_pool -Wall -Wextra -g -std=c++11 -o main.o main.cpp

g++ -g   -o test-thread-pool main.o -lpthread

All kind of help would be appreciated 👍

Feature request: post tasks to front of queue (LIFO)?

I need to prefer new tasks over old tasks in the queue. Is it possible to change the mpmc queue to post to the front of the queue, or to add a new method post_front()?

license?

This looks really nice. I'm interested in adding this as a package to the hunter project (ruslo/hunter#251) https://github.com/ruslo/hunter, but I don't see a license anywhere. Are you amenable to a simplified BSD type license (or similar)? Thanks.

Explain getWorker

Hi, please explain why this is so? https://github.com/inkooboo/thread-pool-cpp/blob/master/include/thread_pool/thread_pool.hpp#L149
It seems a fairly frequent use case is pushing from one thread of a bunch of tasks.
I understand the idea of ​​locality, but at first glance it doesn't seem to work? After all, if we push a task from the thread of the thread pool, this does not necessarily mean that its number is equal to its number in the array

Maybe something like this would be work better?

getWorker:
auto idx = current thread id;
if last_idx == idx {
     return last_worker;
}
for (worker : workers) {
     if (idx == worker.idx) {
         last_idx = idx;
         last_worker = worker;
         return worker;
     }
}
Logic with m_next_worker

Or something like this

thread_local worker_index{-1}; // set in worker::start
if (worker_index != -1) {
     return workers[worker_index];
}
Logic with m_next_worker

Idling Performance

I wanted to share some results relating to the performance of this thread pool when all member threads are idle. These results are caused by the waiting loop in threads that are idle.

Though its probably not a great idea to run 500 threads on a machine with 8 logical cores, a large number of threads can be needed for blocking i/o operations, and the idle CPU usage overhead can burden and drain the battery life of client applications.

inline thread_id() does not play with multiple compilation units (GCC 4.9)

The decl of thread_id as inline static causes issues with multiple compilation units under GCC 4.9, for example:

In the test directory create a new file getWorkerIdForCurrentThread.cpp:

#include <worker.hpp>
size_t getWorkerIdForCurrentThread() { return *thread_id(); }
size_t getWorkerIdForCurrentThread2() { return Worker::getWorkerIdForCurrentThread(); }

Declare the new functions at the top of thread_pool.t.cpp:

size_t getWorkerIdForCurrentThread();
size_t getWorkerIdForCurrentThread2();

Update doTest("post job"), add the following line in the lambda after std::packaged_task<int()> ....:

printf("\nThread id(1): %lu, id(2): %lu, id(3): %lu, id(4): %lu\n", 
  Worker::getWorkerIdForCurrentThread(), *thread_id(), 
  getWorkerIdForCurrentThread(), getWorkerIdForCurrentThread2());

Build and run thread_pool.t.cpp as usual, the output will show the new line:

Thread id(1): 7, id(2): 7, id(3): 4294967295, id(4): 7

Compile with -O3 and we get even worse output (due to aggressive inlining):

Thread id(1): 7, id(2): 7, id(3): 4294967295, id(4): 4294967295

In essence getWorkerIdForCurrentThread.cpp contains another instance of tss_id per thread since it was inlined by GCC and is never initialized to a valid value by the thread pool. Now when we call these functions we always see -1ul regardless of the thread we call from.

This limits the usefulness of the thread pool - at best you have to be extremely careful what you call and where when you have multiple CPPs.

Duplicate thread_pool directory?

There is include/thread_pool and thread_pool directory with similar files, but different
version of the files. Is one of the two deprecated?

c++11 support

I am interested in using this with c++11 projects. It seems the only feature requiring c++14 is the capture-by-move initialization for the packaged_task lambda assignment here:

https://github.com/inkooboo/thread-pool-cpp/blob/master/thread_pool/thread_pool.hpp#L117

I'll need to familiarize myself with this. It seems a workaround might be possible in c++11 based on this document ("evil_wrapper"):

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3610.html

I'll take a look. Any suggestions are welcome.

Spurious Failure in Task Posting

Posting tasks to this thread pool may fail spuriously. This is caused by the MPMC Bounded Queue. It has the associated (undocumented) property that enqueue and dequeue operations may fail spuriously, regardless of whether or not the queue is actually full or empty (respectively).

I'm going to detail the enqueue version of the issue, though I believe it exists for dequeue as well. Lets say our queue is of size N, and we are dequeue-ing cell K. The issue arises when a context switch occurs between the CAS operation on dequeue_pos and the incrementing of the sequence number of K. What this means is that dequeue operations can continue to occur on other threads, effectively freeing up elements of the queue, but enqueue operations cannot take place past cell K. This entails enqueue operations failing even when there is buffer available in the queue. Below is an example of the state of the data structure that arises from this situation.

Buffer Index	Sequence #	Position Pointers
1	36993
2	36994
3	36995
4	36996
5	36997
6	36998
7	36999
8	37000
9	37001
10	37002
11	37003
12	37004
13	37005
14	37006
15	37007
16	37008
17	37009
18	37010
19	37011
20	37012
21	37013
22	37014
23	37015
24	37016
25	37017
26	37018
27	37019
28	37020
29	37021
30	36958	enqueue - 37021
31	37022
32	37023
33	37024
34	37025
35	37026
36	37027
37	37028
38	37029
39	37030
40	37031
41	37032
42	37033
43	37034
44	37035
45	37036
46	37037
47	37038
48	37039
49	37040
50	36978	dequeue - 36977
51	36979
52	36980
53	36981
54	36982
55	36983
56	36984
57	36985
58	36986
59	36987
60	36988
61	36989
62	36990
63	36991
64	36992

These values are actually pulled straight out of my debugger. As you can see, we have 20 cells available in our queue, but because of the inopportune context switch in cell index 30, we can no longer enqueue. If we allow the system to keep running, cell 30 will eventually sort itself out, but that means any call to enqueue or dequeue is not 'strong'. Meaning that it would be have to run in a loop, and that we cannot trust false outcomes actually representing the state of the data structure.

CMake option to disable tests compilation

First of all thank you for making this library installable. I use External_Project_Add() for all libraries, it compiles the library in an isolated environment and installs it in the specified directory (if you are interested in more details I can explain).

What I want is a way to disable tests when building the library.

Wrap this block in an if

thread-pool-cpp/CMakeLists.txt

Lines 11 to 22 in af95dd8

	# gtest
	enable_testing()
	add_subdirectory(googletest)
	include_directories(${CMAKE_CURRENT_SOURCE_DIR}/googletest/googletest/include)
	include_directories(${CMAKE_CURRENT_SOURCE_DIR}/include)
	# Tests
	add_subdirectory(tests)
	# Benchmark
	add_subdirectory(benchmark)

like this

https://github.com/Microsoft/GSL/blob/5cbde3008aa43a9c5f6c219ee15b8388336d4433/CMakeLists.txt#L71-L75

then I will be able to control that block with -DENABLE_TESTS=ON/OFF

ThreadPool must execute all tasks posted to it

Currently, ThreadPool's destructor does not wait for completion of all tasks posted to it, because Worker relies on m_running_flag to stay in the loop, as a result this flag is set to false as soon as ThreadPool destructor is invoked, as the threads exit without further processing the tasks from the queue.

One solution is to remove m_running_flag from the worker, and instead add a poison task to the queue and each worker reads this poison task and exit but before exiting enqueue it again, so that other workers can exit in similar mannner.

Also, ThreadPool may provide a function called wait() function which will add the poison task. Also, the destructor should call this wait() function.

Thread Queue Filling Semantics

Just wanted to point out some behavior in the way task queues fill up and report that the thread pool is full.

We have a thread pool with T threads and Q queue size per thread. Under certain call patterns its possible to have the entire thread pool fill with only 2 threads working, and only Q + 2 tasks in the thread pool (As opposed to the optimal T concurrently working threads, and Q*T + T submitted tasks.

This is caused by 2 factors:

1. Each queue is only serviced by 2 threads, as opposed to something along the lines of round robin work stealing.
2. Task posting aggressively submits new tasks to the queue on the same thread.

If we submit task A to thread Tn, and task A then creates k new tasks, all k tasks will be submitted to Tn's queue. Only T(n-1) and Tn will be active in servicing the k tasks, and if k > Q, the attempt by A to post more than Q tasks will return false, even if the rest of the thread pool is idle.

can't compile with vc2015

static thread_local size_t tss_id = -1u;
the code will cause an error C4146.

File mpsc_bounded_queue.hpp is misnamed

It actually contains MPMC queue, not MPSC.

Task may drop when thread pool destruct

thread pool call worker's stop method when destruct and set worker's runing flag to false
worker's thread proc exit when runing flag is set to false, ignore weather more tasks in its task queue

can worker wait all tasks finish when thread pool destruct