Running tasks in parallel about girder_worker

[jeffbaumes]

This feature is not implemented as of yet in girder worker directly. To do this you would need to start the girder-worker server and send the various tasks through celery manually (calling run). Celery supports parallelism with groups.

I'd like to keep this issue open to track this if we want to implement a better solution that girder worker handles directly.

[rbaral]

Yes, celery does provide the parallel execution, but I need to run some tasks of the workflow itself to run in parallel. I can run the whole workflow instances in parallel using celery or raw python parallelism. The parallelism within the workflow is the good to have feature and also highlighted in the workflow patterns [http://www.workflowpatterns.com/patterns/control/]

Thanks.

[jeffbaumes]

Agreed, it would be a great feature to have.