Threaded destination vs Worker threads about syslog-ng

[bazsi]

The whole point in the mainloop refactor was to be able to use the same infrastructure for worker threads and io threads.

The fact that we have threadid allows LogQueueFifo to scale properly for input threads.

So we should fix the assert.

[algernon]

It's not just the assert. If we fix that, then iv_list_add_tail() will fail horribly later.

[algernon]

I spent some time thinking about this issue, and I believe that bringing worker and io threads under the same umbrella was a mistake. At least for drivers that use LogThreadedDestination, the IO thread is intentionally and forcibly separate, and does not behave the same way as worker threads do (ie, they can't execute in the main thread). Making all of them be able to run in the main thread if there is no separate thread available would be a huge undertaking, which I believe is not worth it.

The IO threads used by LogThreadedDestination should, therefore, be handled independently of worker threads.

[juhaszviktor]

I think that a good solution should be, that the input threads (which threads work into LogFifoQueue) get id from a range 1-64, and the other threads get id from a range 65-128.
With this solution the ivykis worker threads and LogQueueFifo can operate as before, and the threaded destinations (which use worker threads) can be in the same infrastructure.

I will send a pull request in a short time which hopefully solve this problem

[algernon]

Merged a fix from @juhaszviktor, closing this too.