Lima execution seems unexpectedly slow on MASseq run about pbbioconda

[armintoepfer]

Maybe you are limited by IO?

You can also try to set this export PB_BAMREADER_THREADS=12

[cyrilcros]

Thanks for the quick reply. I will check that when my runs end, but it really feels like a regression was introduced. On the exact same data my run has been running for now two hours on 5x times more cores when it was done in 15-20min before. I kept the pipeline code in version control and I still have the Nextflow reports around.
Around November 23rd, I was running with conda ='bioconda::lima=2.7.1' / 24 cpus / a 1 hour walltime limit. The average CPU usage was 40%, right now I am nowhere near that. I have bumped the resources and time limit now and will see how long it takes.
I am still using the same /scratch system on our cluster which is NVMe storage drives in a BeeGFS cluster filesystem.

[armintoepfer]

It scales linear in the number of barcodes, but I really suspect that it's your system. On the Revio instrument I can demux 1M HiFi reads per minute with 96 barcodes and lima 2.9.0.

Example log:

Processed : 8715784
Throughput: 1083976/min
Run Time  : 8m 12s
CPU Time  : 2h 31m
Peak RSS  : 3.72195 GB

[cyrilcros]

Hi, I think you are absolutely right. We added some new nodes to our cluster in December where my jobs are mostly landing. I am also seeing a slow down when checking out the workflow version I used back then.