Comments (14)
It looks like python3.4 made some changes to how .pyc
files are formatted:
https://docs.python.org/3/whatsnew/3.4.html#marshal
So there's probably some new incompatibility with the way esky is building them.
The code in question is e.g. this: https://github.com/cloudmatrix/esky/blob/master/esky/bdist_esky/f_cxfreeze.py#L133
It's trying to generate the contents of a .pyc
from some python source code, so that it can be stuffed into a zipfile.
If you're feeling adventurous, you could try re-implementing this logic using https://docs.python.org/3/library/py_compile.html and see if that makes a difference. It could probably do with a refactoring into a utility function anyway.
from esky.
I'm a bit confused as to what eskycode and eskybscode are supposed to have in them. I see on line 145 that maincode will have the compiled code_source in it, but with the other two the compile function is getting passed an empty string.
from esky.
This is intentional, they're supposed to be empty modules (like putting an empty __init__.py
to mark a directory as a package) so that the import environment for the bootstrapping code looks like it has esky installed.
from esky.
I believe I've found a solution (at least for python 3.4.1). This is my first time contributing to an online project, so what are the next steps I should take? Pull request?
from esky.
Pull request?
Yes, thanks, that would be great.
from esky.
After looking at doing a pull request, I realized I don't know what the standard is when importing something that is only available in certain python versions.
Here is what I have so far, it works for the versions of everything I have installed at the moment, but I haven't tested it otherwise.
# Since Python 3.3 the .pyc file format contains the source size.
# It's not used for anything at all except to check if the file is up to date.
# We can set this value to zero to make Esky also work for Python 3.3
if sys.version_info[:2] < (3, 1):
maincode = imp.get_magic() + struct.pack("<i",0)
eskycode = imp.get_magic() + struct.pack("<i",0)
eskybscode = imp.get_magic() + struct.pack("<i",0)
maincode += marshal.dumps(compile(code_source,INITNAME+".py","exec"))
eskycode += marshal.dumps(compile("","esky/__init__.py","exec"))
eskybscode += marshal.dumps(compile("","esky/bootstrap.py","exec"))
else:
import importlib._bootstrap
loader = importlib._bootstrap.SourceLoader()
code = loader.source_to_code(code_source, '<string>')
empty_code = loader.source_to_code(b"", '<string>')
empty_bytecode = importlib._bootstrap._code_to_bytecode(empty_code)
maincode = importlib._bootstrap._code_to_bytecode(code)
eskycode = empty_bytecode
eskybscode = empty_bytecode
from esky.
Great! Using stuff from importlib
seems much nicer than trying to keep up-to-date with the evolving bytecode format.
It's OK to start a pull-request even if the code is incomplete, since you can always add more commits to the branch to update it after further discussion.
I'd go for three-way branching logic here, which is pretty ugly but safer than anything else I can think of:
if sys.version_info[:2] < (3, 1):
...existing code...
elif sys.version_info[:2] < (3, 4):
...existing code...
else:
...the new code for 3.4 and later...
The docs say that loader.source_to_code
is new in 3.4 so we may have to keep both existing branches of this code, as well as the new one.
Ideally we can refactor this into a helper function in esky.util
, e.g. something like compile_to_bytecode
where you just give it the source and it returns the bytes for the .pyc
file.
from esky.
I'm using the latest version of esky available at the moment, and it looks like this was included (through pull request or whatever), however the version on pypi is outdated and this bug is still present in that build, and now i'm having to specify the git commit here on github in my requirements.txt rather then just 'esky', is there any chance you can update the build on pypi?
from esky.
Doen't having it on pypi also help Python 3 users since 2to3 has to be run on the new code before it will work? (This is based on a fuzzy memory of having to do that a few months ago - I apologize if this is incorrect.)
from esky.
yes, esky is on pypi and it uses 2to3 to produce a python 3 version, but that just runs the same setup.py that is present in this github repo, but i like pypi more cause its just one less thing to keep track of (what git commit do i need to make this work, etc, is the current commit broken, etc)
but currently the version on pypi doesn't have this fix so anyone on python3.4+ who downloads it from pip will get a broken version and be sad =(
from esky.
Yeah I remember that sadness... took me a while to get all my stuff working
on 3.4, and esky wasn't a very smooth transition. Also had to switch to
cxfreeze from py2exe, but I don't think that was esky's fault.
from esky.
Hilariously, i also had problems with cx_freeze, because apparently python internal stuff changed with python3.4 and it builds fine, but running the generated exe gives some import error. Turns out if you just 'recompile' cx_freeze then it works again, but they don't seem to think that updating the version on pypi is worth it: https://bitbucket.org/anthony_tuininga/cx_freeze/issue/81/python-34-venv-importlib-attributeerror
why do i have to maintain so many custom copies of all these libraries =(
from esky.
Just to note thas as of 07/30/2015 the issue remais on the pypi version of esky, but it seems corrected on the master branch.
Installing from the repo fixed it for me. Thank you all for the great work and valuable contributions!
Environment Python 3.4.1 x64, cx_freeze 4.3.4, win7
from esky.
Pypi has been updated
from esky.
Related Issues (20)
- Unable to get admin rights HOT 5
- Import error with pythonnet HOT 12
- cxfreeze windows digest not euqal HOT 1
- continous integration appveyor HOT 4
- Esky falls over using 64bit python on windows HOT 1
- Esky falls over using 64bit linux HOT 1
- py2exe on windows HOT 1
- py2app on mac HOT 1
- OSX: unorderable types: str() > NoneType() while creating patch HOT 5
- unable to run bootrap file using py2exe and python3.4 HOT 4
- py2exe py2.7, failing on py3k_master branch
- Esky differential patches not working HOT 6
- extract overwritting case sensitive files
- Error for running the tutorial esky-master\tutorial\stage0\ on my Windows10 HOT 2
- Error for running python setup.py bdist_esky HOT 9
- Windows system error with auto update HOT 3
- esky in windows enviornment HOT 6
- Error running bootstrap executable created by py2exe with Python 3.4 HOT 2
- tutorials fail HOT 1
- locale.getdefaultlocale() returns None HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from esky.