iobio / bam.iobio.io Goto Github PK

Amazon S3 bam.test.files.private/SRR893046_STP1N240A.bam

Is it worth allowing a defined range to be input by the user? Right now, if I look at my genome, the default range is 0-600, with the mean ~360. If I click outliers, it jumps to 0-200M, so even with zooming, I have very little control. I want to see if there is a spike from Alus which would be ~700 for this data, but can't expand the range of the default view to get out to ~800 which is what I would like. Would be nice if I could just specify that I want to look at the range 0-800.

data with many small references (e.g. microbiome) need to sample multiple references at the same time to be useful. Need to figure out how to display multiple read coverages at once to be able to implement this feature.

Discussion with Omicia led to the suggestion that we should be able to define chromosome and region and include in URL.

It looks the the 1000Genomes BAM files are ordered sequentially (e.g. ch1, ch2, ch4...), but this is not always the case with BAM files. For example, load this BAM (zebrafish)

https://s3.amazonaws.com/bam.test.files.private/zebrafish/8789X1_111118_SN141_0417_BC04PLABXX_6_STP1N60A.bam

Notice how the first chromosome in the panel is ch20.

This is not always replicatable, which is obviously a pain, but I have seen this happening a number of times recently, including at BoG.

I downloaded the HG04141bam and bai file that is the same as the default URL. However, the line chart draws a bit strangely when I load from the local file vs. the URL. See screen prints.

Line chart from bam served from URL bam http://s3.amazonaws.com/1000genomes/data/HG04141/alignment/HG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam

Line chart from same bam file loaded from my local drive

Need better error reporting when bai file is missing. Right now, when loading the bam file from a URL, if the corresponding .bai file is not present, the failure is silent. We should send an error message back to the web page that indicates that the corresponding .bai file is not found. Down the road, it might me nice to generate a .bai file on the fly if it is not present.

give the read coverage a y axis to make it more useful

This issue only occurs on specific BAM files. To reproduce this bug, please load the following BAM file:
https://s3.amazonaws.com/bam.test.files.private/SRR893047_STP1N240A.bam.

Here are the screen prints of the Read Coverage Histogram for ch1, after each time ch1 is selected. Each time, it produces an entirely different distribution (or so it appears....)

currently the references are just listed, which is impractical for files with a large number of references. Turn into a auto-complete drop down list.

For a BAM with sparse data, zoom in on a section of data where there are reads and the read coverage distribution looks good, but the average is off the chart to the left. I assume that this is showing the average of all the data, which since it is sparse, is close to zero. Would it make more sense to show the average of the data in the selected data?

From Howard Sun:

I do not think to first fix it,but more good solution is to load bam via sftp(ssh), because a large bam data is not open for anyone.
There should be many public codes available. I hope you will consider first sftp. That would be great.

Set colours for the reference sequences and font sign on the URL section to be the same as vcf.

Hi All,

Good day and hope all is well.

We have recently installed bam.iobio.io locally with the help of the installation guide at this link: https://github.com/chmille4/iobio/wiki/Local-iobio-setup

However, when loading the bam files by selecting the 'choose bam file' button it takes very long to load and visualize the data. Our bam file is indexed according to http://bam.iobio.io/help.html.

I am wondering if the bam file needs to be in the same virtual server for some reason where the webservice is running . Currently the bam files are in a separate node connected through a high speed data network.

Also is there any recommended server specification (cpu, memory etc.) for installing bam.iobio.locally?

Best Regards,

Anwar

for chromosome selection.

If I supply a file that doesn't exist, bam just sits. Console logs show the file doesn't exist, but the user has no idea.

Doing so would also let authorized users access data on Google Genomics, which will include the MSSNG autism database, TCGA, 1000 Genomes and more

Average looks off for some datasets

Example:
http://bam.iobio.io/?bam=http://s3.amazonaws.com/bam.test.files.private/TCGA-Benchmark-v4/G15511.HCC1143.1.bam

Add a range selector to fragment length so that a user can look at the outlier fragments without the chart being overwhelmed by the data in the correct fragment range

Current version of samtools fails on remote files that have indices named xxx.bai instead of xxx.bam.bai. This is fixed in samtools v1.

When I tried to load the Illumina TruSeq Sure Select Capture bed, it wouldn't load giving me the message of no matching reference. The bed file has the reference name 1, where the bam file has the reference name chr1.

Ftp doesn't support byte range requests so we can't support ftp urls.

Detect ftp data urls and do

1. Check if url is also being served over http and if so use that url
2. If above doesn't work, give error to use.

Add a set of test BAM files and a doc that provides bam.iobio links to all the test files. This can be used to check through a list of files and ensure that bam.iobio acts as expected on some constructed use cases.

It seems like the current sampling should be interrupted if the user clicks on another reference when sampling is still in progress.

Custom BEDs will only load when reference names (e.g. ch9) match. Need an alias lookup so that different naming conventions won't throw off app. Here are the 2 custom bed files I atttempted to load.

Illumina TruSeq exome capture:
truseq_exome_targeted_regions.hg19.bed

NimbleGen exome capture:
SeqCap_EZ_Exome_v3_capture.bed

When I hover over the Read Coverage Distribution, it shows the x,y which is great. However, the y (%) is not rounding, showing many digits to the right of the decimal.

Do we want these to be rotated?

I loaded the following bam file and it worked the first time and the second time. However, when I tried to increase the sample size again, the page never refreshed.

HG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam

sampling currently is solely based on regions. This means we can either over or under sample based on the read depth of the data. Need to improve sampling so that we keep sampling if we get too few reads or stop sampling once we have enough reads.

Start bam.iobio for whole genome, rather than chromosome 1 and estimate the number of unmapped reads to get proper mapping rate estimate.

Can we figure out a way to generate a bam.iobio screenshot showing the real stats for my genome (specifically, mapping rate of ~70%) for BoG poster. If the iobio suite workflow is also going to be demo'd at BoG, we need to be able to have bam.iobio run and generate these stats. Real stats are:

Total reads: 671644470
Mapped reads: 479405211 (71.3778%)
Forward strand: 431035875 (64.1762%)
Reverse strand: 240608595 (35.8238%)
Failed QC: 0 (0%)
Duplicates: 40566531 (6.03988%)
Paired-end reads: 671644470 (100%)
'Proper-pairs': 472073744 (70.2862%)
Both pairs mapped: 477749930 (71.1314%)
Read 1: 335671024
Read 2: 335973446
Singletons: 1655281 (0.246452%)

When I compare the Read Depth line charts for the same BAM file, one loaded from a client-side file, the other from a URL, the line charts closely resemble each other; however, on some chromosomes, there are empty regions on the file-loaded BAM where the URL-loaded BAM has point (depth) data. See screen prints below on the default 1000Genomes dataset (HG04141).

Chromosome 13

Chromosome 9

Allowing letters of support etc.

When the index is parsed, the reference are built up, but the read coverage across the chromosomes appears all at once after the sequences are loaded. Should these be drawn in along with the chromosome titles?

Since fragment length is only calculated with paired-end reads, let's grey out the Fragment Length Distribution chart when the BAM file has single-end reads only.

After choosing a .bam and .bai file, I click on the 'Open' button and the File chooser dialog closes, but the web page stays on the home page.

I will try to troubleshoot this a bit more tomorrow, stepping through the code to see where the error occurs.

Andrew is having trouble with bam.iobio for local files from a mounted drive. I am seeing the same problem from my local computer.

The density chart loads, but then the browser (Chrome in this case) crashes with an ‘out of memory’ error.

We should test on multiple resolutions, in particular when presenting, we want to be sure that the apps still look good. In the viz-dev branch, the scaling of the chromosome bars and the read coverage (top image) are not the same, so the coverage starts partway through the bars (no coverage above chr1) etc. in a presentation.

a link titled ‘Report Problem’; clicking it would trigger a screen capture and a popup that allowed the user to send a quick message which was sent to a group email account (e.g. - [email protected] or [email protected])

I am running into one behavior that seems odd…. I get a different number of Reads Sampled each time I load the same bam file. Is this expected behavior?

At first, I thought that the stats were different when the .bai was produced from samtools vs bamtools. However, it may just be that the number of reads sampled varies considerably for each load.

On a related note, the line charges for read depth look slightly different for the .bai generated from the same bam but with samtools vs bamtools. Specifically, I don’t see the line going from the high point back down to zero on the line chart for the samtools generated .bai file.

I’ve attached 2 screen prints showing the read depth line charts from same bam, but from .bai files generated from samtools vs. bamtools.

I pulled down a public dataset for a Zebrafish BAM. Here is the URL to the file:
http://s3.amazonaws.com/bam.test.files.private/zebrafish/8789X1_111118_SN141_0417_BC04PLABXX_6_STP1N60A.bam

No errors, warning in console and the server appears to return data for get estimated read depth; however, the all widgets stay in 'sampling...' state.

Good news - This seems to be dependent on the file loaded. For example, I do not see this problem with our HG04141.mapped.ILLUMINA.bwa.BEB.low_coverage.20130415.bam file.

I am seeing it with a public bam file I downloaded from a Univ of Utah data sharing site.
(A1754/Bam/Hg19/SRR893054_STP1N240A.bam)

Here are screen prints for each time the Read Coverage Distribution histogram is refreshed for the times I clicked on ch20:

2nd iteration

3rd iteration

The drop down for reference sits on top of the wheel and the read coverage doesn't rescale with the window. The reference sequence markers (the coloured blocks) resize as the screen is resized, but the data stays put. This is an issue, because sometimes the scaling is wrong when you load and you

have to resize the window to get them to line up.

When I pull up a bam file in bam.iobio.io, the read coverage distribution histogram works as expected. However, when I click back and forth between one reference (chromosome) and another, I occasionally see a truncated (> 100%?) tick on the Y-axis.

In the read coverage chart, the reference sequence blocks have no mouseover and the mouse symbol does not make it clear that they are clickable.

I noticed that the Read Length histogram is for K reads counted. Is the Fragment Length Y axis the same?