bccdc-phl / tbprofiler-nf Goto Github PK

We've seen some examples of samples that have an H37Rv-like genotype, but get reported as M. canetti or M. pinnipedii, with a Percentage of ~0.1.

We should handle these cases where the Percentage is very low and correct the Species field.

If a negative control (containing reads but few/zero TB reads) is put through the pipeline, it crashes:

plot_coverage.py", line 66, in plot_coverage 
sum(1 if x > threshold else 0 for x in depths.depth)
ZeroDivisionError: division by zero

Generate a depth-of-coverage summary for each isolate, including:

• Configurable depth threshold
• Percent of genome that are below the depth threshold
• .bed file showing low-coverage regions

...and possibly also

• A .pdf plot of depth of coverage across the genome

The --versioned_outdir param hasn't proven to be useful and it clutters our publishDir directives. Remove it.

Add a flag --collect_outputs that allows the collection of tabular outputs for all samples into a single file stored at the top-level of outdir.

Also add a flag --collected_outputs_prefix that controls the prefix used for collected outputs. Default to collected

In #21 I accidentally merged before committing my last update to the check_snpit_against_tbprofiler.py script.

We switched from our initial strategy of looking for H37Rv in the tbprofiler lineages to instead looking for lineage4 as the main lineage and looking for >98% reads mapped. We found that the initial approach worked on our positive controls (which are H37Rv) but not on some samples that are close to H37Rv but don't have H37Rv in their tbprofiler reports.

Update that script to reflect the new strategy.

there is an error when converting qualimap depth of > 1000 to float, as it is represented as '1,000' in string format.

A new version of tbprofiler has been released. Update this pipeline to use tbprofiler v6.2.0

Add a testing pipeline using GitHub Actions.

When calculating depth coverage for negative control the pipeline will fail. The pipeline typically is not typically ran on negative control samples but issue should be fixed to prevent unnecessary pipeline fail.

The _tbprofiler_full_report.csv can contain multiple mutiple mutations per antibiotic. The _tbprofiler_resistance.csv file created in split_tbprofiler_csv.py only outputs the first mutation from the full report.

We'd like to capture as much 'data provenance' information as possible in our production analysis pipelines. Information such as tool versions, input file checksums, analysis timestamps and database versions should be captured in some persistent form so they can be used to track and reproduce analyses.

See jodyphelan/TBProfiler#345

TBProfiler generates a resistance report that lists each drug on a separate row. This makes it a bit difficult to gather together resistance info from multiple samples. It would be convenient to have a resistance report that is structured to be one row per library.

It is not an error of tbprofiler-nf. but some samples can cause tbprofiler to fail, for unclear reasons.
Could we add ignore errors for those samples so the pipeline can run on the rest of the samples?

Our tbprofiler_full_report.json files don't have any newlines or indentation. They're fairly large/complex documents so this makes it difficult for a person to visually read the files directly. They can be re-formatted with a text editor, but it would be simpler and more convenient to write the files in "pretty-printed" form (with newlines and indentation).

One way to do this is to just stream the file through python's json.tool module before writing to disk:

On this line:

We could instead do:

cat results/${sample_id}.results.json | python -m json.tool > ${sample_id}_tbprofiler_full_report.json

We've made some changes to our provenance format. See the BCCDC-PHL/kma-cgmlst pipeline for an example of the updated format:

https://github.com/BCCDC-PHL/kma-cgmlst?tab=readme-ov-file#provenance

It seems that in cases where the sample's genotype closely matches H37Rv, the species is reported as N/A. But it should be reported as M. tuberculosis.

We'd like to incorporate subspecies calling into our analysis, using philipfowler/snpit

In order to do this, we'll need to collect the whole genome .vcf files from TBProfiler and feed those into snpit. We had previously only been collecting the target-specific .vcf files from TBProfiler.