Comments (4)
I would just like to add, arriba is really quite fast, so I don't think multi-threaded execution per analysis run is the right form of parallelism either. However, assuming you have a whole bunch of different RNA-Seq samples then a more appropriate form of parallelism would be to run arriba on a whole bunch of different samples in parallel at once using say GNU parallel, I have implemented such a system in a bash script which you might find useful.
from arriba.
Hi, thanks for the positive feedback! Always good to hear that the tool is useful.
Regarding your request: I have considered support for multi-threading a number of times. So far I have always dismissed it for two reasons:
-
Some of the steps would be hard to parallelize and others would hardly benefit from parallelization, because the overhead to dispatch the work to multiple threads would exceed the actual work that has to be done.
-
Arriba is already pretty fast. Ideally the most time-consuming step (extracting reads from the BAM files) runs in parallel to alignment, so no time is lost. I understand that many users want to run Arriba separately from alignment, such that parallelized extraction of reads could reduce the runtime noticeably. But I have seen it process RNA-Seq samples with a billion reads in an hour (excluding alignment), which is still not much time.
I have never tried implementing multi-threading. So I don't actually know how much it would improve the speed, but my gut feeling tells me it is not worth pursuing. I'll reconsider, if your use case makes a compelling point that multi-threading would be of much benefit. What is your experience? Have you noticed particularly long runtimes with certain samples? This could be a bug that should be fixed, rather than using the steamroller approach "multi-threading". On a related note: The next release of Arriba comes with speed optimizations for two of the slower filters ("low_entropy" and "homologs") and faster parsing of GTF files.
BTW Arriba was not designed for DNA-Seq data (WGS/WES). It makes many assumptions which are only applicable to RNA-Seq data. If you want to find fusions in DNA-Seq data, you are probably better off using a structural variant caller. Here are a number of tools to choose from: https://bio.tools/?page=1&q=%27DNA%20structural%20variation%27&sort=score
Once you have structural variant calls, you can feed them to Arriba as additional information (see option -d
). But I would not recommend running Arriba on DNA-Seq data directly.
from arriba.
@akeeeshi: A new release is available. It comes with several speed improvements, most notably ~25% faster reading of BAM files.
I agree with @MattBashton that process-level parallelization using GNU parallel is the way to go. If you don't object, I will close this issue.
@MattBashton: Great script! So far I used to wrap Arriba in a call to xargs -P
, but yours look much more elaborate. I will give it a try. ;-)
from arriba.
I apologize for the delay in response. My Github notifications are doing something odd. After hearing everyone's input I can see how process-level parallelization is the way to go. Appreciate the help!
from arriba.
Related Issues (20)
- Split reads evidence for EWSR1-NR4A3 fusion missing in sample.arriba.fusions.discarded.tsv HOT 2
- Validating New Versions HOT 4
- How to get the position of the fused gene on the chromosome? HOT 3
- Giving known fusions list does not recover the fusions I want HOT 4
- request for simulation dataset mentioned in the paper HOT 2
- intergenic breakpoints reported without distances to genes HOT 1
- Can one read be used to support multiple fusions HOT 6
- Use draw_fusions.R to only output partial plots HOT 1
- Arriba for plate-based scRNA-seq HOT 2
- Recommendation for samtools bam>fastq conversion for Arriba HOT 1
- FastQ test data cannot be aligned HOT 2
- Recommended STAR index version HOT 10
- Missing Transcript IDs in ARRIBA Output Files HOT 11
- error: Command 'arriba' not found, did you mean: command 'ariba' from deb ariba (2.14.6+ds-3build1) HOT 1
- total=ERROR: could not find sequence of contig 'NC_007605' HOT 2
- Issue with the "relative_support" filter HOT 1
- The CBFB::MYH11D fusion gene is discarded by the 'blacklist' filter, but it is not found in the blacklist file. HOT 3
- Can I use the latest Gencode GTF file ? HOT 1
- coverage < split_reads HOT 1
- Feature request: Compatibility with Starlong HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from arriba.