Comments (4)
Fixing that problem is easy enough with setting copy=None
. However, we get something more sinister later in the process:
(numpy-2-venv) jk@empire$ python3 -m bio2zarr vcf2zarr explode tests/data/vcf/sample.vcf.gz tmp/x.icf -p0
Scan: 100%|█████████████████████████████████████████████████████████████████| 1.00/1.00 [00:00<00:00, 52.7files/s]
Explode: 100%|███████████████████████████████████████████████████████████████████| 9.00/9.00 [00:00<00:00, 429vars/s]
(numpy-2-venv) jk@empire$ python3 -m bio2zarr vcf2zarr encode tmp/x.icf tmp/x.vcz -f -p0
Encode: 85%|███████████████████████████████████████████████████████████▊ | 792/927 [00:00<00:00, 7.81kB/s]Segmentation fault (core dumped)
(numpy-2-venv) jk@empire$ /usr/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
So, during the encode step we've got a segfault. Digging in to this now.
from bio2zarr.
Hmm, so setting copy=True above resolves this segfault (which appears to happen when accessing the genotype data). So, setting copy=True seems fine and dandy to me - I'm sure the perf difference is negligible.
from bio2zarr.
With #257 we should be basically set for numpy 2.0 and numpy 1.x compatibility. To close this issue we should add a CI job that explicitly installs numpy 2.x and runs the tests. Later, when numpy 2.0 becomes the default thing we install (due to dependencies) we can switch this to 1.x.
We're waiting on numpy 2.0 compatible wheels from msprime, so no point in making this CI job until they arrive.
from bio2zarr.
Just waiting on numpy 2.0 wheels for msprime which should arrive in a few days, and we can then ship a numpy 2.0 compatible version.
from bio2zarr.
Related Issues (20)
- Returning a string from `.mkschema` HOT 1
- Document status of Python API
- Fixup msprime based tests when packages are fixed
- Add "what about cloud?" docs
- Add explicit warning for Mac Python 3.9
- New tool: tskit2zarr HOT 1
- Document copying to cloud storage HOT 1
- Refactor docs build infrastructure
- Restructure vcf2zarr docs
- Add --no-progress (or similar) to suppress progress
- Bug in dexplode-partition
- Change dexplode-init to use ``--num-parts``/``-n`` instead of positional HOT 1
- Change dencode-init to use --num-partitions
- Hypothesis testing for vcf2zarr HOT 13
- Pin to zarr < 3
- ValueError: could not broadcast input array
- Run tests against Zarr 3 HOT 2
- Set copy=True in np.array creation for numpy 2.0 compatibility HOT 1
- ICF stores created with numpy 1.x won't work with numpy 2.x HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bio2zarr.