pycqa / modernize Goto Github PK
View Code? Open in Web Editor NEWModernizes Python code for eventual Python 3 migration. Built on top of fissix (a fork of lib2to3)
Home Page: https://modernize.readthedocs.org/
License: Other
Modernizes Python code for eventual Python 3 migration. Built on top of fissix (a fork of lib2to3)
Home Page: https://modernize.readthedocs.org/
License: Other
and optionally also
isinstance(_, basestring)
to isinstance(_, six.string_types)
unicode
to six.text_type
str
to six.binary_type
(moved from mitsuhiko/python-modernize#18 )
python-modernize currently makes this change:
- for i in xrange(N):
+ for i in range(N):
The resulting code is inefficient on Python 2.x for large N. I would prefer just to add:
from six.moves import xrange
(moved from mitsuhiko/python-modernize#16 )
While trying to port pylint I have run into itertools.izip
and cStringIO
as things that need to get moved. It also happens that six.moves
handles both cases. Any interest in trying to add a fixer that can work with all renames that six.moves
supports?
https://pypi.python.org/pypi?name=modernize&version=0.3&:action=display_pkginfo
Download-url: UNKNOWN
License: UNKNOWN
Platform: UNKNOWN
This would guarantee cross-version semantics of the function itself along with returning the same type (file
vs. _io.TextIOWrapper
).
d.addCallback(lambda (rc, out, err), fn=fn:
self.failUnlessEqual(out, fn * 1000))
I would have expected this to be changed to something like
d.addCallback(lambda rc_and_out_and_err, fn=fn:
self.failUnlessEqual(rc_and_out_and_err[1], fn * 1000))
(moved from mitsuhiko/python-modernize#23 )
+from __future__ import print_function
#!/usr/bin/env python
Imports should only be added after any initial lines that are either blank or start with #. That would account for copyright headers etc., as well as shebang lines.
(moved from mitsuhiko/python-modernize#21 )
By default at present, unicode literals are wrapped in a call to six.u, so u'foo'
-> six.u('foo')
.
The --compat-unicode
flag leaves unicode literals untouched. The u'foo'
syntax is valid from Python 3.3, released in September 2012. I think that projects that haven't yet ported to Python 3 are unlikely to want to support Python 3.2 or older, so I would make the compat-unicode behaviour the default, and add a --six-unicode
flag to get the current behaviour.
Basically just replace /
with //
, under the assumption that integer division was intended. This makes the code clean for DeprecationWarning: classic int division
when run under python -3
.
(moved from mitsuhiko/python-modernize#22 )
Since fix_metaclass
was originally written, six.with_metaclass()
has been changed to accept multiple base classes. That would lead to much cleaner output. Compare the current solution:
class Foo(six.with_metaclass(Meta, type('NewBase', (Bar, Spam), {}))):
pass
to what six would allow for today:
class Foo(six.with_metaclass(Meta, Bar, Spam)):
pass
While a Python 2.6 and higher fix, it is still legitimate and useful (I just had to go through and do that transformation manually).
Currently python-modernize strips L suffixes off numeric literals. This is dubious because presumably there was a reason for specifying that the value should be long on Python 2.x. (For most arithmetic operations it doesn't matter, but it does affect type testing.)
six doesn't seem to have a function to do a long coercion, although one can easily be defined:
L = int if six.PY3 else long
and then change, e.g. 0xFFFFFFFFL
to L(0xFFFFFFFF)
.
(moved from mitsuhiko/python-modernize#19 )
It would be useful if the README file showed some basic usage examples. I say this as it seems unclear that there is a command "python-modernise" which gets installed and has the same syntax as "2to3".
(moved from mitsuhiko/python-modernize#29 opened by @kevcampb )
@brettcannon has been doing a lot of work improving existing code and adding new fixes. We should think about making a new release so people can use these changes.
I've renamed the milestone from 0.3.1 to 0.4, because it's definitely more than a bugfix release.
To properly match Python 3 import semantics, every import within a package should have from __future__ import absolute_import
. lib2to3.fixes.fix_import
does a reasonable thing to syntactically make an import use explicit relative imports, but that doesn't guarantee future imports will do the right thing if someone forgets to run the fixer again.
Maybe make an optional fixer that adds the __future__
statement for any code that has an import statement? Another option is to extend lib2to3.fixes.fix_import
to add the __future__
statement after calling super().transform()
(it also would also make the fixer a little more performant as it uses the __future__
import as a short-circuit). Personally I prefer the latter option for being more thorough.
There are various future_builtins not being used, e.g. fix_map. Using future_builtins would also make fix_zip work when --no-six
is specified.
If one has:
foo(u'.........'
u'.........')
it gets converted to:
foo(six.u('........')
six.u('........'))
that obviously does not work, better would be:
foo(six.u('.........'
'.........'))
(moved from mitsuhiko/python-modernize#9 opened by @ThomasWaldmann)
Then one does not need to change this manually after running python-modernize.
(moved from mitsuhiko/python-modernize#10 opened by @ThomasWaldmann)
We should end up with fixers that cover every situation that makes sense, whether listed as an issue or already committed. There is also a selfish reason for doing this as anything not covered by a fixer will need to be documented for the Python porting HOWTO when I rewrite to heavily orient it towards using python-modernize.
That way it guarantees that even calls to file()
will get the from io import open
fix.
The link to the LICENSE file broke rendering of the README rst on PyPI. I fixed it by hand for this release, but we should work out something better. Maybe just including LICENSE in the tarball is enough. Otherwise it may be easiest just to make it not a link.
People in some pre-commit hook may want to run python-modernize
over changed code and reject the commit if it would lead to an actual change. This would allow a code base to be ported and then force developers to actually code in a Python 2/3 compatible way instead of perpetually relying on tools to do it for them.
E.g.
from urlparse import ParseResult
=>
from six.moves.urllib.parse import ParseResult
and
import urlparse
urlparse.ParseResult
=>
import six.moves.urllib.parse as urlparse
urlparse.ParseResult
The idea is that regardless of what str/bytes strategy is chosen, you still need to replace calls to the unicode()
type since unicode
doesn't exist in Python 3. six.text_type()
acts as a replacement since it is unicode()
in Python 2 and str()
in Python 3.
#38 implements this fixer.
Our current test coverage is pretty poor. There should be tests covering all fixers, including corner cases, and also covering command-line argument handling.
I added support for test coverage reporting via coverage.io: https://coveralls.io/r/python-modernize/python-modernize?branch=master
As mentioned in #31 , there is a desire to document all of the fixers. Probably want to have opt-out vs. opt-in, description of what exactly is changed (e.g. fix_range
changes both range()
and xrange()
but only in calling contexts), and maybe a link to appropriate section of http://python3porting.com/differences.html for reference.
There are two possible parts to such a fixer. One is fixing calls to file()
which should probably change to io.open()
(or open()
depending on the outcome of #42 ). The other part is the use of file
for type checks, which requires using io.IOBase
, but only if io.open()
is used instead of open()
.
For now it's sufficient for this to show the differences between releases 0.2 and 0.3.
E.g. instead of
- def _got_root((root, path)):
+ def _got_root(xxx_todo_changeme):
+ (root, path) = xxx_todo_changeme
use
- def _got_root((root, path)):
+ def _got_root(root_and_path):
+ (root, path) = root_and_path
Also, the X_and_Y
convention could be used in cases like this:
- d.addCallback(lambda (root, path):
- self._get_or_create_directories(root, path, metadata))
+ d.addCallback(lambda root_path:
+ self._get_or_create_directories(root_path[0], root_path[1], metadata))
Here the generated name root_path
is inappropriate because that would mean something like "path of the root", as opposed to a (root, path) pair. root_and_path
would be better.
(moved from mitsuhiko/python-modernize#15 )
Wrapping dict.items()
, et. al. as list(dict.items())
would deal with the case where people actually want the list of items (e.g. A.items() + B.items()
). If there is a worry about the generic method names could the fixer be at least opt-in? At least with Pylint I could use this.
Other than the future import, is there any reason to have our own fixer? If not then following the fix_range solution of subclassing and adding an import statement should be enough to keep equivalent semantics and heavily cut down on code we need to maintain.
$ cat test.py
u"artificial example"
import sys
$ python-modernize --six-unicode test.py
[...]
@@ -1,2 +1,3 @@
-u"artificial example"
+six.u("artificial example")
import sys
+import six
which doesn't work because the import of six is after the first use.
(moved from mitsuhiko/python-modernize#20 )
What I'm thinking of is something like --compat=2.6
such that things like fix_next use the next
built-in instead of six.advance_iterator
so that the translated code is a bit more idiomatic and less noisy. This also allows for things such as using future_builtins
for map, etc. instead of worrying about breaking Python 2.5 code if the user only cares about e.g. Python 2.7 compatibility.
If the project owners are amenable to the idea I'm happy to create a pull request to implement this. I could create separate fixers and have logic in libmodernize.main
to select the right fixers based on --compat
/--no-six
or have all of the selection logic in the fixers themselves and expose the requirements in libmodernize.__init__
.
Since the tests directory isn't installed, it's not easy to run tests on the installed version.
If it is installed then it should probably be in libmodernize.tests
rather than a toplevel module, to avoid module name clashes.
E.g. [c]StringIO.StringIO
should be changed to six.BytesIO
.
(moved from mitsuhiko/python-modernize#17 )
As discussed in #61 , for maintenance reasons along with minimizing the possibility of not covering all reasonable argument possibilities that Python supports -- at the expense of possibly over-applying when a name of a function or method just happens to match -- we should simplify the matching patterns for function calls.
The typical pattern should probably be trailer< '(' ')' >
when no argument is ever allowed, trailer< '(' any+ ')' >
when at least one argument needs to be there, and probably something like trailer< '(' [any+] ')' >
when 0 or more (the last pattern is untested).
Looks like the comment was pasted from libmodernize/fixes/fix_next.py and not updated.
(moved from mitsuhiko/python-modernize#25 )
raw_input
should become six.moves.input
. Ties into #34.
I.e. modernize.__version__ == "0.3"
.
This is useful to check which version you have installed.
Since most uses of basestring are for isinstance() calls, could you use (unicode, str) if hasattr(__builtins__, 'unicode') else str
instead of basestring for isinstance/issubclass calls?
2to3 allows you to select fixes using short names, e.g. sys_exc
instead of lib2to3.fixes.fix_sys_exc
. Modernize should ensure that the same thing works, so a user wanting to apply a single fix doesn't have to type out the full path.
Without knowing exactly what fixers are allowed (not) to do, I'm starting to see some disagreement of whether an approach is the best solution or not. Ignoring the syntax-only fixers which are have zero controversy -- e.g. except Exception as exc: ...
-- I'm not sure what guidelines there are which ones overrule what. Some questions I know I either have or have seen in other issues:
Take the whole range()
/xrange()
issue as an example. If being idempotent is the the most important thing while also allowing Python 3 semantics, then you will want:
from six.moves import range
range(1) # was range(1)
range(2) # was xrange(2)
If you don't want to break any code but still want idempotency over Python 3 features, then you will want:
list(range(1)) # was range(1)
list(range(2)) # was xrange(2)
If you don't want to break any code while allowing Python 3 features but don't care about idempotency:
from six.moves import range
list(range(1)) # was range(1)
range(2) # was range(2)
Three possible outcomes all based on what priorities you place on the resulting code and what your plans are for the future of it.
For me personally, I want to see this project help other projects get their code modernized as much as possible: that means I prefer an approach which is idempotent so that they can benefit from newer fixers and accidental changes that will hurt Python 3 compatibility, but with a willing cost of lightly breaking code (e.g. triggering a TypeError
when you are doing from six.moves import range; range(1) + range(2)
is acceptable to me). IOW getting people to code using Python 3 features is paramount to me while idempotency is important for easier tooling.
One approach we can take is for any fixer which cannot be totally unambiguous and won't break code while stilling being idempotent then it is an optional/opt-in fixer -- do we have a preferred term for this? -- and that is the end of that. If people want help with range()
/xrange()
we won't make the choice for them and they will simply have to choose a fixer. That would mean fix_filter
, fix_map
, fix_range
, and fix_zip
would all become optional as they all change the semantics to some degree or another. Making them optional is fine by me, but as stated previously my opinion is that what they do now -- sans fix_range
stopping trying to wrap range()
in a list()
call -- is acceptable.
fix_map and fix_zip both add a from six.moves
import, without changing the code at all. This means that code is getting the iterator versions of fix and map consistently across Python 2 and 3, but it can break code which relied on getting the list version. I think we should improve this.
I think the fixers should work like this:
for i in map(...):
...
# Converts to:
from six.moves import map
for i in map(...):
...
a = map(...) # anything other than iterating over map in a loop
# Converts to:
from six.moves import map
a = list(map(...))
Now that libmodernize.fixes.fix_range
is being used instead of lib2to3.fixes.fix_xrange
, there is the problem that the fixer is no longer idempotent, i.e. list(range(1))
will become list(list(range(1)))
and from six.moves import range; range(1)
becomes from six.moves import range; list(range(1))
.
Possible solutions are:
range
and xrange
(I suspect range
is doable with some not
rule in the pattern, but xrange
might be a pain)xrange
and assume anyone using range
to make a list can do the fix themselveslib2to3.fixes.fix_xrange
which turns xrange
into range
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.