Comments (6)
Ooh, interesting bug! We'll look into this further (unless someone already knows the answer and I get to learn something new), but I suspect that the problem may have occurred during the generation of the bigWig files (so something wrong with the bigWig writer, not the reader). If I'm reading the bytes right, the first bigWig file internally claims to have a size of 0xffffffff8855ee2f bytes, or about 1.8e+19. As you suggest, that seems unlikely. Using only the low-order 32 bits of that value gives 2287332911, which seems much more reasonable.
from kent.
from kent.
So, I wasn't actually using bigWigSummary
. I maintain my own Python bindings to kent lib, and I was trying to read out numpy arrays of 1kb-binned tracks using the summary functionality. I was then able to reproduce the error using bigWigSummary
.
I have an easy workaround: don't use the summary functionality and just do the binning and averaging in Python. It's more accurate anyway, since the other way is really interpolating from the nearest zoom level. pyBigWig, which uses its own bigwig lib, also seems to be able to execute the same queries.
So this isn't an impediment for me, but I thought I'd report it.
from kent.
Thanks for reporting this @nvictus . It does look like this is a problem with how these files were created on the encode portal. I've sent them mail in an attempt to track down how this problem was introduced.
from kent.
I talked a but to Encode and we've been unable to track down this source of this problem. I've encouraged them to use our most recent code to build new big files since there have been several bug fixes and one may have resolved this problem.
Do let us know if you run into this again.
from kent.
I've recently run into this exact issue on additional files from Encode beyond those in @nvictus's list.
./bigWigSummary ENCFF089CVK.bigWig chr8 146250000 146300000 100
needLargeMem: trying to allocate 18446744069414602975 bytes (limit: 17179869184)
It can really throw a wrench into a processing pipeline, so if you have any additional updates on a work-around or solution either on your end or on Encode's, I'd appreciate it.
from kent.
Related Issues (20)
- path issue for doBlastzChainNet.pl HOT 2
- Certificate validation (https.c) causing issues with build in v424 HOT 9
- profile db not found in sqlProfileToMyCnf()
- bedGraphToBigWig: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory HOT 1
- twoBitToFa: Can only handle version 0 of this file. This is version 1 HOT 1
- module in src/hg can't make with MySQL 8^ HOT 1
- `Response is missing required header Content-Length: for url ` HOT 11
- Bioconda package for kent HOT 4
- wigToBigWig doesn't work for chromosomes with spaces in name. HOT 3
- src/lib/htmshell.c doesn't compile on aarch64 Ubuntu 20.04 HOT 2
- Where can I download the pslSplitOnTarget binary? HOT 2
- Provide Linux aarch64 binary for genePredToGtf HOT 17
- Incorrect number of bytes for empty slots in rWriteLeaves HOT 5
- Proposal to remove S:417/440/505 in branch specific mask list.
- Bash reporting error HOT 1
- undefined reference to sqlSetIsUcscMirror HOT 1
- undefined reference HOT 2
- Question about classNet HOT 9
- errors reported by axtchain HOT 2
- undefined reference to `knet_init_alt'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kent.