number of online processors when some cores are isolated via kernel about pigz

[madler]

You can specify the number of threads using -p, so perhaps you can use the output of numactl to deduce the number of available cores and feed that to the command line option.

However I wonder what difference it makes. There is likely not much penalty in specifying more cores than you have, other than using more memory for the buffers in all the threads. You should test to see the difference between -p 7 and -p 32, or -p 100 for that matter. In fact, pigz invokes two more threads than requested (expect when using -p 1), one being the main thread and one being a thread to write the output. The n cores in -p n is only the number of cores for compression.

Of course, if you want the best performance, you don't want to specify fewer cores than you have, so the number of actual cores should always be safe.

[MichaelReitsma]

Not being a unix expert in anyway i was wondering why it would be necessary, apart from this example, to specify the number of processors/cores it uses ? Doesn't unix have a multi-core load balancer or something ?
Or is it best to have one core free for other tasks ?
So if i have a 8 core system and pigz uses 2 cores for other purposes then i should specify a maximum of 5 with -p5?

Edit:
After some more testing i found the processing time to be different between -p8 and -p6 (or even -p7) on a 8 core system. This suggests that specifying 6 or 8 cores is not the same in terms of load. 6 cores should be the same snce pigz claimes an additional 2 for administration overhead.

[madler]

@MichaelReitsma You don't need to specify the number of cores. By default pigz will set up twice as many compression threads as there are cores. This is done to have a high probability of keeping all of the cores busy. Other processes on the system will continue to get their fair share of time on the cores. Though their fair share will be less while pigz is trying to keep all the cores busy.

If you have a need to assure that there are some number of cores that pigz will not monopolize (not sure what that need would be), then I would use the number given to -p — not that number plus two. The other two threads are to take advantage of parallel input/output operations, not computation, so those two threads will not monopolize cores.