local config about porecov

[iferres]

Thank you! And sorry for my late response. I said nanoplot as an example. I'd probably set other defaults as well, but I'd need to benchmark them before. I ran it on 12 barcodes with cores = 4 and it was pretty fast.
Nice to have the --max_cores parameter, I may or may not copy this implementation from your wf to mine :P .
Bests!

[hoelzer]

Hi @iferres, thanks for the message and for using the wf!

Yes, you are right and I agree that for some processes the default CPU usage can be reduced to allow more parallelization.

@replikation I think something like:

process {
    withLabel:  artic       { cpus = params.cores }
    withLabel:  bwa         { cpus = params.cores }
    withLabel:  fastcov     { cpus = 4 }
    withLabel:  ggplot2     { cpus = 4 }
    withLabel:  guppy_cpu   { cpus = params.cores }
    withLabel:  guppy_gpu   { cpus = params.cores }
    withLabel:  kraken2     { cpus = params.cores }
    withLabel:  krona       { cpus = 1 }
    withLabel:  minimap2    { cpus = params.cores }    
    withLabel:  nanoplot    { cpus = 2 }
    withLabel:  nextclade   { cpus = params.cores }
    withLabel:  pangolin    { cpus = 4 }
    withLabel:  president   { cpus = 1 }
    withLabel:  pycoqc      { cpus = 4 }
    withLabel:  snp_sites   { cpus = 4 }
    withLabel:  ubuntu      { cpus = 1 } 
}

I am also not sure if guppy_gpu needs params.cores bc/ it anyway runs on the GPU so does not need to block CPUs? (however, it's anyway the first step all other processes have to wait for)

And, @iferres if you are not aware of it: there is also an additional parameter --max_cores for the local profile. By that, you can configure how many CPUs are block in total on your machine. So lets say you have a local machine w/ 32 CPUs but you only want to use 24 in total, you could specify:

--cores 8 --max_cores 24

so e.g. bwa would in maximum run 3 times in parallel.

[replikation]

Hi @iferres glad you like it :)

yes, we still have #58 open, so feedback here is of course welcome :) . Hardcoding nanoplot down to 2 makes e.g. sense I think. @hoelzer the cpus are for the demultiplexing part after the calling which is slow with only a few threads.

Usually, multithreading is a bit more difficult on one machine (e.g. we want to avoid overloading) so the best way (in my experience) is using the --cores to adjust the level of "parallel" runs as @hoelzer suggested.

[replikation]

added to f919641

[replikation]

@iferres yep let me know 👍🏻