labbcb / espresso-caller Goto Github PK

View Code? Open in Web Editor NEW

0.0 2.0 0.0 180 KB

Automated and reproducible tool for identifying genomic variants at scale ☕️

Home Page: https://bcblab.org/espresso-caller

License: MIT License

WDL 65.39% Python 34.61%

cromwell ga4gh tes wes bioinformatics openapi

espresso-caller's People

Contributors

Watchers

espresso-caller's Issues

GC overhead limit exceeded

Number of samples: 491
Workflow: JointDiscovery
Task: SNPsVariantRecalibratorClassic

[January 11, 2020 11:57:03 AM BRST] org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator done. Elapsed time: 249.44 minutes.
Runtime.totalMemory()=6417809408
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.ArrayList.<init>(ArrayList.java:152)
        at org.broadinstitute.hellbender.engine.FeatureCache.getCachedFeaturesUpToStopPosition(FeatureCache.java:213)
        at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:506)
        at org.broadinstitute.hellbender.engine.FeatureManager.getFeatures(FeatureManager.java:340)
        at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:172)
        at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:124)
        at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:239)
        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantDataManager.parseTrainingSets(VariantDataManager.java:386)
        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator.addDatum(VariantRecalibrator.java:615)
        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator.addVariantDatum(VariantRecalibrator.java:573)
        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator.lambda$consumeQueuedVariants$0(VariantRecalibrator.java:544)
        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator$$Lambda$79/1042273835.accept(Unknown Source)
        at java.util.ArrayList.forEach(ArrayList.java:1249)
        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator.consumeQueuedVariants(VariantRecalibrator.java:544)
        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator.apply(VariantRecalibrator.java:523)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker.lambda$traverse$1(MultiVariantWalker.java:114)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker$$Lambda$77/1808470758.accept(Unknown Source)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.Iterator.forEachRemaining(Iterator.java:116)
        at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker.traverse(MultiVariantWalker.java:112)
        at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:966)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:138)

Add verbose flag

Check `scattered.interval` files

BAM validation workflow creates wrong file name

BAM validation workflow are generating files with .validation_.txt.
We should remove the underscore.

Check sequencing date format

Must be ISO8601 format.

Add support to custom genomic intervals file

By default this tool will look for genomic intervals files designed for WGS.
We have to add support for custom intervals file for working with WES data.

Organize JSON and WDL files in data package

File structure:

workflows
- *.wdl
inputs
- *.json

Set required CPUs for tasks

Tasks

HaplotypeCaller = 4
MarkDuplicates = 16
BaseRecalibrator = 2
MergeGVCFs = 2

Add prefix to sample names from VCF files

When processing new data with pre-processed raw gVCF files, add prefix to sample names through --prefix

ls new_raw_data
100_1.fastq.gz 100_2.fastq.gz 101_1.fastq.gz 101_2.fastq.gz

ls raw_gvcfs
99.g.vcf.gz 100.gvcf.gz

esperesso all --fastq new_raw_data --vcf raw_gvcfs --prefix GVCFS_ res_dir

Will give:

CHROM .... GVCFS_99 GVCFS_100 100 101

Do not import remote WDL files

Provide all required WDL files in the workflows data package.
This tool should zip and submit imported WDL files as workflow dependency.

Wrong sample name extraction from VCF file for joint-discovery

A USER ERROR has occurred: Bad input: Found two mappings for the same sample: 25_14
/home/benilton/bioinf/tmp/cromwell-executions/JointGenotyping/44718321-e203-43ce-aba0-bb5c53dc0359/call-ImportGVCFs/shard-1049/inputs/-64135002/725_14.b37.g.vcf.gz
/home/benilton/bioinf/tmp/cromwell-executions/JointGenotyping/44718321-e203-43ce-aba0-bb5c53dc0359/call-ImportGVCFs/shard-1049/inputs/-64135002/325_14.b37.g.vcf.gz

bcftools query -l /home/benilton/bioinf/tmp/cromwell-executions/JointGenotyping/44718321-e203-43ce-aba0-bb5c53dc0359/call-ImportGVCFs/shard-1049/inputs/-64135002/725_14.b37.g.vcf.gz
bcftools query -l /home/benilton/bioinf/tmp/cromwell-executions/JointGenotyping/44718321-e203-43ce-aba0-bb5c53dc0359/call-ImportGVCFs/shard-1049/inputs/-64135002/325_14.b37.g.vcf.gz

Sample name in VCF files is ok.

Wrong names in workflow inputs JSON file, 25_14 twice.

Do not export uBAM but export ready-analysis BAM files as result

Unmapped BAM (uBAM) files are not useful at this moment.
The ready-analysis BAM files may be useful if their CRAM version are not supported.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble