monitor worker count = 1 bug or confusion with RedisStorage about apalis

[alexlatif]

solved via - very confusing why a worker needs a ConcurrencyLimitLayer. Could you please share the reason why a worker will use a thread pool and what the point of setting workers in the first place are? As in im confusing by having to tweak both to achieve one job at a time.

[geofmureithi]

There are several key points to consider:

• ConcurrencyLayer: This pertains to the number of jobs or tasks that can be executed simultaneously.
• Worker with Count: This relates to parallelism. Without using a thread pool, it would imply that a worker can only handle one job at a time, which is not optimal for our purposes.

Additionally, Apalis leverages Tower to delegate control to the library user. If a service is blocked (e.g., due to rate limits or concurrency constraints), workers are unable to poll for new tasks.

In short workers can consume as much as they can and you get to control that with tower layers.