Comments (2)
Hey dude!
It's great to hear form you, and thank you for the heads up. I can't replicate this on my system with pysam
0.9.1.4 (in or out of a virtualenv
), and I know I have Mac users who are using pysam
>= 0.9.1.0 outside of Conda (usually in a virtualenv
)
I think this is issue is the same set of events that came up in issue 2, in that case with numpy
. The TLDR is that Conda has some strange interactions with pip
, causing Conda's sandboxes to leak. Overall, I strongly recommend using virtualenv (which plays well with pip
, and to my knowledge has raised no problems with plastid
, pysam
, or numpy
) in place of Conda. In a virtuanlenv
, I have been able to build Plastid
with every version of Pysam
>= 0.8.4
The long answer (if what I think is happening actually is) is this:
- Conda aims to create Python sandboxes within which various packages can be installed and upgraded. This is a noble goal
- When
pip
is processing dependencies for setup (fromsetup.py
), it sees the package versions that are present inside the Conda envrionment, as expected - When
pip
actually runs the compiler needed to buildplastid
, it starts a subshell that exists outside the Conda sandbox. This can cause builds to fail, e.g. becausepip
will try to link against a version ofpysam
and/ornumpy
that might be lower or higher than the version required in and detected bysetup.py
. Even if the build succeeds and both versions ofpysam
and/ornumpy
meet the minimal requirements, you may still get runtime errors ifplastid
is linked against a different version ofnumpy
orpysam
from the ones it finds in the runtime environment
In contrast, If pip
is run inside a virtualenv
(not a Conda virtual environment), subshells stay within the sandbox and this problem doesn't happen.
The way (I think) this affected your installation is that:
- During install,
pip
detected some version ofpysam
>= 0.8.4. It must have, because I hard-coded asys.exit(1)
insidesetup.py
ifpysam.__version__
< 0.8.4. This is an atypical thing to do- but I had to do it for backward compatibility with old versions ofpip
. - Within the subshell called by
pip
,gcc
linked againstpysam
0.6.x, causing build to fail - After
pysam
was upgraded,plastid
probably linked to 0.9.1 (or 0.8.4) in the subshell - During
plastid
import, it found the opposite or 0.8.4 (or 0.9.1) in its runtime environemnt - Rolling back to 0.8.4 in your made the import and build-linked versions of
pysam
match
Issues of this type have come up now several times, and it has always been an issue with Conda breaking pip
(or pip
breaking Conda, depending on how you look at it). Unfortunately, I'm powerless to fix this, though I will write a FAQ about it on the install page. I don't intend to besmirch Conda, but I strongly suggest using virtualenv
instead (detailed here.
Before I close this, could you try creating a virtualenv, and installing plastid
in that, following the instructions at http://plastid.readthedocs.io/en/latest/installation.html ?
from plastid.
Hi Josh,
That all makes sense. I tried to do what you suggested in the first place
(as per the docs) but my system fails to create the virtualenv, raising
this exception:
ImportError:
dlopen(/Users/calvinjan/virtualenvs/plastid/lib/python2.7/lib-dynload/_io.so,
2): Symbol not found: __PyCodecInfo_GetIncrementalDecoder
This is clearly not a plastid issue (and probably still a conda nuisance),
so I didn't want to bug you about it. I'll update you if I get around to
troubleshooting this.
Thanks,
Calvin
On Wed, Aug 31, 2016 at 6:46 PM, Joshua Griffin Dunn <
[email protected]> wrote:
Hey dude!
It's great to hear form you, and thank you for the heads up. I can't
replicate this on my system with pysam 0.9.1.4 (in or out of a virtualenv),
and I know I have Mac users who are using pysam >= 0.9.1.0 outside of
Conda (usually in a virtualenv)I think this is issue is the same set of events that came up in issue 2
#2, in that case with
numpy. The TLDR is that Conda has some strange interactions with pip,
causing Conda's sandboxes to leak. Overall, I strongly recommend using
virtualenv https://virtualenv.pypa.io/en/stable/ (which plays well with
pip, and to my knowledge has raised no problems with plastid, pysam, or
numpy) in place of Conda. In a virtuanlenv, I have been able to build
Plastid with every version of Pysam >= 0.8.4The long answer (if what I think is happening actually is) is this:
Conda aims to create Python sandboxes within which various packages
can be installed and upgraded. This is a noble goal
2.When pip is processing dependencies for setup (from setup.py), it sees
the package versions that are present inside the Conda envrionment, as
expected
3.When pip actually runs the compiler needed to build plastid, it starts
a subshell that exists outside the Conda sandbox. This can cause builds to
fail, e.g. because pip will try to link against a version of pysam
and/or numpy that might be lower or higher than the version required
in and detected by setup.py. Even if the build succeeds and both
versions of pysam and/or numpy meet the minimal requirements, you may
still get runtime errors if plastid is linked against a different
version of numpy or pysam from the ones it finds in the runtime
environmentIn contrast, If pip is run inside a virtualenv (not a Conda virtual
environment), subshells stay within the sandbox and this problem doesn't
happen.The way (I think) this affected your installation is that:
During install, pip detected some version of pysam >= 0.8.4. It must
have, because I hard-coded a sys.exit(1) inside setup.py if
pysam.version < 0.8.4. This is an atypical thing to do- but I had
to do it for backward compatibility with old versions of pip.
2.Within the subshell called by pip , gcc linked against pysam 0.6.x,
causing build to fail
3.After pysam was upgraded, plastid probably linked to 0.9.1 (or 0.8.4)
in the subshell
4.During plastid import, it found the opposite or 0.8.4 (or 0.9.1) in
its runtime environemnt
5.Rolling back to 0.8.4 in your made the import and build-linked
versions of pysam matchIssues of this type have come up now several times, and it has always been
an issue with Conda breaking pip (or pip breaking Conda, depending on how
you look at it). Unfortunately, I'm powerless to fix this, though I will
write a FAQ about it on the install page. I don't intend to besmirch Conda,
but I strongly suggest using virtualenv instead (detailed here
http://plastid.readthedocs.io/en/latest/FAQ.html#faq-install-fails-with-prereqs.Before I close this, could you try creating a virtualenv
https://virtualenv.pypa.io/en/stable/, and installing plastid in that,
following the instructions at http://plastid.readthedocs.io/
en/latest/installation.html ?—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#4 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AC-58BIC2_1wopgiB70_XgubTWHqSxDjks5qli5tgaJpZM4Juz6t
.
from plastid.
Related Issues (20)
- Generated .bed file has "0" score in the 3rd line - When generating .bedgraph out of this .bed, no coverage of Ribo-seq data. HOT 2
- Support Python 3.8 HOT 1
- Remove deprecated features
- Support CRAM files HOT 1
- Plotting a metagene plot for two different samples at the same time HOT 1
- Question on --gff_transcript_types and --mask_gff_transcript_types HOT 1
- RuntimeWarning HOT 1
- ‘phase_by_size’ command-line script problem with Numpy(1.18.1) HOT 1
- Undefined symbol caused by compilation error only under Python 2.7.17 (3.x, 2.7.15 are ok) HOT 3
- ModuleNotFoundError: No module named 'plastid.genomics.roitools' HOT 5
- logging dependency should be remove from setup.py when installing with Python3 HOT 1
- Error when converting GFF3 to GTF2 HOT 3
- Incompatible with BioPython 1.78+ HOT 2
- Roitools module error when running psite: "ValueError: too many values to unpack (expected 2)"
- the bam file choose HOT 1
- the p site offset problem HOT 1
- Problem install plastid 'no attribute 'PileupColumn' HOT 2
- installation problems HOT 2
- numpy 1.24.1 and plastid 0.6.1 appear to have an issue due to dropping of support for numpy.int by numpy. HOT 6
- Building wheel is failing in local install HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from plastid.