GithubHelp home page GithubHelp logo

yapf's Issues

Add examples

Before installing yapf, I'd usually want to see some examples of what it does. A weirdly formatted file should come out quite nicely, and an already well-formatted file should come out at least just as well-formatted. I suggest adding some sample input/output combinations either to the Readme or a sepeate Examples file.

The examples could just be copied from some of the tests, but it's much more nice to see some stuff without browsing though the code.

Future imports aren't taken into account when checking syntax

This means that code like this will cause a SyntaxError:

from __future__ import print_function

def call_my_function(the_function):
    the_function("hi")

if __name__ == "__main__":
    call_my_function(print)

with output/traceback like so:

INTERNAL ERROR: 
    call_my_function(print)
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/flyte/dev/yapf/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "yapf/__init__.py", line 102, in main
    print_diff=args.diff)
  File "yapf/__init__.py", line 124, in FormatFiles
    filename, style_config=style_config, lines=lines, print_diff=print_diff)
  File "yapf/yapflib/yapf_api.py", line 68, in FormatFile
    print_diff=print_diff)
  File "yapf/yapflib/yapf_api.py", line 113, in FormatCode
    reformatted_source = reformatter.Reformat(uwlines)
  File "yapf/yapflib/reformatter.py", line 73, in Reformat
    verifier.VerifyCode(formatted_code[-1])
  File "yapf/yapflib/verifier.py", line 45, in VerifyCode
    compile(normalized_code.encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 1
    call_my_function(print)
                         ^
SyntaxError: invalid syntax

The problem is that each line is taken on its own merit without taking into account imports from future.

yapf will line break right before the closing bracket of a tuple

This is somewhat of an aesthetic preference, so obviously feel free to close this as notabugwontfix, but the following code:

def aaaaaaaaaaa(*args):
    pass


bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb = aaaaaaaaaaa(
    ("...", "."), "..",
    ".............................................."
)

is reformatted as

def aaaaaaaaaaa(*args):
    pass


bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb = aaaaaaaaaaa(("...", "."
    ), "..", "..............................................")

Having the tuple argument split over the line like that with the closing bracket on the next line looks really bad. Note that putting the closing bracket on the same line as the opening bracket would not even come close to the 80 character column limit.

yapf --style=pep8 produces "E126 continuation line over-indented for hanging indent"

The following code:

class NAryTreeStrategy():

    def produce_template(self, context, pv):
        if True:
            children = tuple((self.branch_key_strategy.draw_template(
                context, pv.branch_key_parameter),
                self.produce_template(context, pv))
                for _ in range(10))
            return children

Is reformatted as:

class NAryTreeStrategy():
    def produce_template(self, context, pv):
        if True:
            children = tuple((self.branch_key_strategy.draw_template(
                context, pv.branch_key_parameter),
                              self.produce_template(context, pv))
                             for _ in range(10))
            return children

Which causes the pep8 checker to report "E126 continuation line over-indented for hanging indent"

yapf will sometimes eat comments

If I have a python file with just the following line:

#"

Then running yapf on the file will eat the comment and produce an empty file.

E122 continuation line outdented inside class body

yapf formats:

class eld(d):
    if str(geom.geom_type).upper(
    ) != self.geom_type and not self.geom_type == 'GEOMETRY':
        ror(code='om_type')

as:

class eld(d):
    if str(geom.geom_type).upper(
) != self.geom_type and not self.geom_type == 'GEOMETRY':
        ror(code='om_type')

This is both pep8 wrong and really confusing to read.

Comment between two decorators produces internal error

The following:

def foo(x):
    return x

@foo
# A comment
@foo
def inner(self):
    pass

Produces the following error:

INTERNAL ERROR: 
# A comment
@foo
Traceback (most recent call last):
  File "/home/david/.pyenv/versions/2.7.8/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/david/.pyenv/versions/2.7.8/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "/home/david/yapf/yapf/__init__.py", line 102, in main
    print_diff=args.diff)
  File "/home/david/yapf/yapf/__init__.py", line 124, in FormatFiles
    filename, style_config=style_config, lines=lines, print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 67, in FormatFile
    print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 110, in FormatCode
    reformatted_source = reformatter.Reformat(uwlines)
  File "/home/david/yapf/yapf/yapflib/reformatter.py", line 73, in Reformat
    verifier.VerifyCode(formatted_code[-1])
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 45, in VerifyCode
    compile(normalized_code.encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 2
    @foo
       ^
SyntaxError: invalid syntax

W503: yapf inserts line break before rather than after a binary operator

The following:

def w(context): {
    'e_link': (
        not is_popup and context['has_delete_permission'] and
        ange and t('e', True)
    ),
}

Gets reformatted as:

def w(context):
    {
        'e_link': (not is_popup and context['has_delete_permission'] and ange
                   and t('e', True)),
    }

Which is wrong. The "and" should not be starting a line according to pep8.

yap --style=pep8 sometimes deletes required blank lines

The following code:

def noop(x):
    return x


timeout_settings = None


# This is a comment not a blank line
# This is a comment not a blank line
@noop
@noop
def test_slow_failing_test_1(x):
    pass

is reformatted to:

def noop(x):
    return x


timeout_settings = None

# This is a comment not a blank line
# This is a comment not a blank line
@noop
@noop
def test_slow_failing_test_1(x):
    pass

With a single blank line between timeout_settings and the start of the comment. This is incorrect. ( E302 expected 2 blank lines, found 1)

Both comments and both decorators appear to be necessary to trigger this behaviour.

yapf does not explicitly handle different encodings

yapf will error if you run it on a file that cannot be handled by the encoding set by your LANG environment variable. e.g. consider the following file:

# encoding: utf-8

x = "☃"

If you set LANG=C and run yapf on it this will crash with

    Traceback (most recent call last):
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/runpy.py", line 170, in _run_module_as_main
        "__main__", mod_spec)
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
        sys.exit(yapf.main(sys.argv))
      File "/home/david/yapf/yapf/__init__.py", line 102, in main
        print_diff=args.diff)
      File "/home/david/yapf/yapf/__init__.py", line 124, in FormatFiles
        filename, style_config=style_config, lines=lines, print_diff=print_diff)
      File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 59, in FormatFile
        original_source = ReadFile(filename, logging.warning)
      File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 142, in ReadFile
        source = fd.read()
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 24: ordinal not in range(128)

(Took me forever to figure out this was what was going on here. I was experiencing a lot of crashes when running yapf in a subprocess but couldn't reproduce them. Finally figured out it was that the environment was getting overridden)

yapf will crash if you run it on a file with a byte order mark

In the interests of not having github eat the important details, here is an example file encoded as a python string literal:

"\ufeff# -*- coding: utf-8 -*-\n\nresult = 'passed'\n"

(Yes, I know that a bom in a utf-8 file doesn't make sense. Don't ask me, I just find 'em).

Running on python 3.4.2 with the latest master this will produce the following lib2to3 error:

    Traceback (most recent call last):
      File "/home/david/yapf/yapf/yapflib/pytree_utils.py", line 85, in ParseCodeToTree
        tree = parser_driver.parse_string(code, debug=False)
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/driver.py", line 106, in parse_string
        return self.parse_tokens(tokens, debug)
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/driver.py", line 71, in parse_tokens
        if p.addtoken(type, value, (prefix, start)):
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/parse.py", line 116, in addtoken
        ilabel = self.classify(type, value, context)
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/parse.py", line 172, in classify
        raise ParseError("bad token", type, value, context)
    lib2to3.pgen2.parse.ParseError: bad token: type=56, value='\ufeff', context=('', (1, 0))

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/runpy.py", line 170, in _run_module_as_main
        "__main__", mod_spec)
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
        sys.exit(yapf.main(sys.argv))
      File "/home/david/yapf/yapf/__init__.py", line 102, in main
        print_diff=args.diff)
      File "/home/david/yapf/yapf/__init__.py", line 124, in FormatFiles
        filename, style_config=style_config, lines=lines, print_diff=print_diff)
      File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 67, in FormatFile
        print_diff=print_diff)
      File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 88, in FormatCode
        tree = pytree_utils.ParseCodeToTree(unformatted_source.rstrip() + '\n')
      File "/home/david/yapf/yapf/yapflib/pytree_utils.py", line 91, in ParseCodeToTree
        tree = parser_driver.parse_string(code, debug=False)
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/driver.py", line 106, in parse_string
        return self.parse_tokens(tokens, debug)
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/driver.py", line 71, in parse_tokens
        if p.addtoken(type, value, (prefix, start)):
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/parse.py", line 116, in addtoken
        ilabel = self.classify(type, value, context)
      File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/parse.py", line 172, in classify
        raise ParseError("bad token", type, value, context)
    lib2to3.pgen2.parse.ParseError: bad token: type=56, value='\ufeff', context=('', (1, 0))

Running 2to3 itself on the file works correctly.

Another lib2to3 parse error

This one is not caused by whitespace as far as I can tell, or at least not at the end of the file:

def walk(top, topdown=True, onerror=None, followlinks=False):
            yield from walk(new_path, topdown, onerror, followlinks)

Produces the error:

Traceback (most recent call last):
  File "/home/david/yapf/yapf/yapflib/pytree_utils.py", line 85, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/usr/lib/python3.2/lib2to3/pgen2/driver.py", line 106, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/usr/lib/python3.2/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/usr/lib/python3.2/lib2to3/pgen2/parse.py", line 159, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=5, value='    ', context=('', (1, 0))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.2/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python3.2/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "/home/david/yapf/yapf/__init__.py", line 98, in main
    lines=lines))
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 88, in FormatCode
    tree = pytree_utils.ParseCodeToTree(unformatted_source.rstrip() + '\n')
  File "/home/david/yapf/yapf/yapflib/pytree_utils.py", line 91, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/usr/lib/python3.2/lib2to3/pgen2/driver.py", line 106, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/usr/lib/python3.2/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/usr/lib/python3.2/lib2to3/pgen2/parse.py", line 159, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=5, value='    ', context=('', (1, 0))

lib2to3 parse error on well formed code that 2to3 passes fine

The following code:

def response(                  using=None):
    if (context_instance is _context_instance_undefined and
        dirs is _dirs_undefined and dictionary is _dictionary_undefined):
            return to

Gives me the following exception when run under python 3.4:

Traceback (most recent call last):
  File "/home/david/yapf/yapf/yapflib/pytree_utils.py", line 85, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/driver.py", line 106, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/parse.py", line 159, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=6, value='', context=('\n', (5, 0))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "/home/david/yapf/yapf/__init__.py", line 102, in main
    print_diff=args.diff)
  File "/home/david/yapf/yapf/__init__.py", line 124, in FormatFiles
    filename, style_config=style_config, lines=lines, print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 67, in FormatFile
    print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 88, in FormatCode
    tree = pytree_utils.ParseCodeToTree(unformatted_source)
  File "/home/david/yapf/yapf/yapflib/pytree_utils.py", line 91, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/driver.py", line 106, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/lib2to3/pgen2/parse.py", line 159, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=6, value='', context=('\n', (5, 0))

Running the normal 2to3 command line utility on this file works fine, so I think there's something about what you're passing in to 2to3.

Integration as a http://pre-commit.com/ hook

This isn't really such a bug report as more of an inquisition about how to approach this.

As requested here: pre-commit/pre-commit-hooks#50 we've had some interest in integrating yapf with our pre-commit framework.

There's two basic approaches to doing this so I'd like to get your opinions on this (I'd like to contribute to making either of them happen):

  1. Provide the proper metadata in google/yapf for running hooks (basically involves checking in a file similar to this: https://github.com/pre-commit/pre-commit-hooks/blob/master/hooks.yaml (except of course it'd just list yapf)).
  2. Provide library mechanisms to fix files and we'd write a wrapper around yapf in one of our repositories.

Option 1 is ideal for us as we don't have to deal with dependencies, breaking apis, etc. especially when the project is in an early stage and potentially volatile (and I don't think yapf is on pypi yet?)

Either way, the general interface for a pre-commit hook to work with our framework is as follows:

  • Installable in an isolated environment
  • Exposes an executable which:
    • Takes a list of filenames to check / fix (in the case of yapf, either)
    • Returns 0 when everything is A-OK or no changes are made, and nonzero otherwise

I'm not sure how much of that already happens but I'd be willing to pitch in to make it work (and test it to prevent regressions, etc.).

Given those assumptions it'd be as simple as checking in a file named hooks.yaml which has contents probably similar to this:

-   id: yapf
    name: yapf
    description: "Runs yapf 'Yet Another Python Formatter' over python source"
    entry: yapf
    language: python
    files: \.py$
    args: [-i]

Thanks in advance,

Anthony

SyntaxError in yapflib.verifier

The following code:

class YapfTest(unittest.TestCase):
  def testSimple(self):
    unformatted_code = textwrap.dedent(u"""\
        print('foo')
        """)

Gives the following error:

INTERNAL ERROR: 
        unformatted_code = textwrap.dedent(u"""\
        print('foo')
        """)
Traceback (most recent call last):
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 38, in VerifyCode
    compile(textwrap.dedent(code).encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 4
    """)
      ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 41, in VerifyCode
    ast.parse(textwrap.dedent(code.lstrip('\n')).lstrip(), '<string>', 'exec')
  File "/usr/lib/python3.2/ast.py", line 36, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<string>", line 3
    """)
      ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.2/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python3.2/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "/home/david/yapf/yapf/__init__.py", line 102, in main
    print_diff=args.diff)
  File "/home/david/yapf/yapf/__init__.py", line 124, in FormatFiles
    filename, style_config=style_config, lines=lines, print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 67, in FormatFile
    print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 110, in FormatCode
    reformatted_source = reformatter.Reformat(uwlines)
  File "/home/david/yapf/yapf/yapflib/reformatter.py", line 73, in Reformat
    verifier.VerifyCode(formatted_code[-1])
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 45, in VerifyCode
    compile(normalized_code.encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 3
    """)
      ^
SyntaxError: invalid syntax

...as you can probably tell, by this point I have a fuzzer doing most of the work for me (about the first half + a few of the subsequent ones have been hand produced) and this is from turning it on the yapf codebase itself. If you want it I'd be happy to give you a copy of the fuzzer.

yapf --style=pep8 continuation line may not distinguish itself from next line

The following code:

class Foo(object):

    def bar(self):
        if self.solo_generator is None and len(
                self.generators + self.next_batch) == 1:
            pass

reformats as

class Foo(object):
    def bar(self):
        if self.solo_generator is None and len(
            self.generators + self.next_batch) == 1:
            pass

Which the pep8 error checker says "E125 continuation line with same indent as next logical line" to

Broken indentation

yapf seems to break the indentation of Python 3.4's bisect.py. Note that the lines after the else: should be indented one more level than what yapf is currently doing.

$ python -m yapf -d bisect.py
--- bisect.py   (original)
+++ bisect.py   (reformatted)
@@ -1,4 +1,5 @@
 """Bisection algorithms."""
+

 def insort_right(a, x, lo=0, hi=None):
     """Insert item x in list a, and keep it sorted assuming a is sorted.
@@ -14,12 +15,15 @@
     if hi is None:
         hi = len(a)
     while lo < hi:
-        mid = (lo+hi)//2
+        mid = (lo + hi) // 2
         if x < a[mid]: hi = mid
-        else: lo = mid+1
+        else:
+        lo = mid + 1
     a.insert(lo, x)

-insort = insort_right   # backward compatibility
+
+insort = insort_right # backward compatibility
+

 def bisect_right(a, x, lo=0, hi=None):
     """Return the index where to insert item x in list a, assuming a is sorted.
@@ -37,12 +41,15 @@
     if hi is None:
         hi = len(a)
     while lo < hi:
-        mid = (lo+hi)//2
+        mid = (lo + hi) // 2
         if x < a[mid]: hi = mid
-        else: lo = mid+1
+        else:
+        lo = mid + 1
     return lo

-bisect = bisect_right   # backward compatibility
+
+bisect = bisect_right # backward compatibility
+

 def insort_left(a, x, lo=0, hi=None):
     """Insert item x in list a, and keep it sorted assuming a is sorted.
@@ -58,9 +65,10 @@
     if hi is None:
         hi = len(a)
     while lo < hi:
-        mid = (lo+hi)//2
-        if a[mid] < x: lo = mid+1
-        else: hi = mid
+        mid = (lo + hi) // 2
+        if a[mid] < x: lo = mid + 1
+        else:
+        hi = mid
     a.insert(lo, x)


@@ -80,9 +88,10 @@
     if hi is None:
         hi = len(a)
     while lo < hi:
-        mid = (lo+hi)//2
-        if a[mid] < x: lo = mid+1
-        else: hi = mid
+        mid = (lo + hi) // 2
+        if a[mid] < x: lo = mid + 1
+        else:
+        hi = mid
     return lo

 # Overwrite above definitions with a fast C implementation

Here is a minimal example:

if True: pass
else: pass
--- foo.py  (original)
+++ foo.py  (reformatted)
@@ -1,2 +1,3 @@
 if True: pass
-else: pass
+else:
+pass

Similarly:

class Foo(object): pass
--- foo.py  (original)
+++ foo.py  (reformatted)
@@ -1 +1,2 @@
-class Foo(object): pass
+class Foo(object):
+pass

Thanks

TypeError: unorderable types because cur_token.node_split_penalty is None

When running yapf over this code:

class CookieJar(RAMCookieJar):

    def purge_old_cookies(self):
        cookies = [c for c in self.allCookies()
                   if c.isSessionCookie() or c.expirationDate() >= now]

I get:

Traceback (most recent call last):  
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main  
    "__main__", mod_spec)  
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code  
    exec(code, run_globals)  
  File "/home/florian/proj/qutebrowser/.venv/lib/python3.4/site-packages/yapf/__main__.py", line 18, in <module>  
    sys.exit(yapf.main(sys.argv))  
  File "/home/florian/proj/qutebrowser/.venv/lib/python3.4/site-packages/yapf/__init__.py", line 102, in main  
    print_diff=args.diff)  
  File "/home/florian/proj/qutebrowser/.venv/lib/python3.4/site-packages/yapf/__init__.py", line 124, in FormatFiles  
    filename, style_config=style_config, lines=lines, print_diff=print_diff)  
  File "/home/florian/proj/qutebrowser/.venv/lib/python3.4/site-packages/yapf/yapflib/yapf_api.py", line 68, in FormatFile  
    print_diff=print_diff)  
  File "/home/florian/proj/qutebrowser/.venv/lib/python3.4/site-packages/yapf/yapflib/yapf_api.py", line 101, in FormatCode  
    uwl.CalculateFormattingInformation()  
  File "/home/florian/proj/qutebrowser/.venv/lib/python3.4/site-packages/yapf/yapflib/unwrapped_line.py", line 77, in CalculateFormattingInformation  
    token.split_penalty += _SplitPenalty(prev_token, token)  
  File "/home/florian/proj/qutebrowser/.venv/lib/python3.4/site-packages/yapf/yapflib/unwrapped_line.py", line 342, in _SplitPenalty  
    if cur_token.node_split_penalty > 0:  
TypeError: unorderable types: NoneType() > int()

chokes on `rb"literal"`

yapf currently chokes on raw bytestring literals:

bash-4.3$ cat /tmp/rb_str_literal.py
rb'literal'
bash-4.3$ python3 -m yapf -i /tmp/rb_str_literal.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/yapf/yapflib/pytree_utils.py", line 96, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib2to3/pgen2/driver.py", line 106, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib2to3/pgen2/parse.py", line 159, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=3, value="'literal'", context=('', (1, 2))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.4/site-packages/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "/usr/local/lib/python3.4/site-packages/yapf/__init__.py", line 109, in main
    print_diff=args.diff, verify=args.verify)
  File "/usr/local/lib/python3.4/site-packages/yapf/__init__.py", line 133, in FormatFiles
    print_diff=print_diff, verify=verify)
  File "/usr/local/lib/python3.4/site-packages/yapf/yapflib/yapf_api.py", line 72, in FormatFile
    verify=verify)
  File "/usr/local/lib/python3.4/site-packages/yapf/yapflib/yapf_api.py", line 95, in FormatCode
    tree = pytree_utils.ParseCodeToTree(unformatted_source.rstrip() + '\n')
  File "/usr/local/lib/python3.4/site-packages/yapf/yapflib/pytree_utils.py", line 101, in ParseCodeToTree
    tree = parser_driver.parse_string(code, debug=False)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib2to3/pgen2/driver.py", line 106, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib2to3/pgen2/driver.py", line 71, in parse_tokens
    if p.addtoken(type, value, (prefix, start)):
  File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib2to3/pgen2/parse.py", line 159, in addtoken
    raise ParseError("bad input", type, value, context)
lib2to3.pgen2.parse.ParseError: bad input: type=3, value="'literal'", context=('', (1, 2))

This is using revision 7a01d00.

yapf --style=pep8 does not put spaces after a trailing comma

Reformatting the following (pep8 correct) code:

trailing_comma = [
    1,
]

Results in the following:

trailing_comma = [1,]

PEP8 wants whitespace after the trailing comma. I can't immediately find a reference in PEP8 itself, but the pep8 checker program calls this E231.

Consistent indentation for peer values

Many Django projects have code like this:

urlpatterns = patterns('',
    url(r'^$', 'homepage_view'),
    url(r'^/login/$', 'login_view'),
    url(r'^/login/$', 'logout_view'),
    url(r'^/user/(?P<username>\w+)/$', 'profile_view')
)

With the PEP-8 preset, this shows the alignment algorithm's attempts to fit as much as possible on each line which tends to require a little more care scanning to distinguish the arguments:

urlpatterns = patterns('', url(r'^$', 'homepage_view'), url(r'^/login/$',
                                                            'login_view'),
                       url(r'^/login/$',
                           'logout_view'), url(r'^/user/(?P<username>\w+)/$',
                                               'profile_view')
                      )

A quick look through style.py doesn't seem to offer a way to have it attempt to fit everything on a line but, when impossible, use a common starting position all of the elements at the same level to avoid what happens to the last element in the previous example:

urlpatterns = patterns(
    '',
    url(r'^$', 'homepage_view'),
    url(r'^/login/$', 'login_view'),
    url(r'^/login/$', 'logout_view'),
    url(r'^/user/(?P<username>\w+)/$', 'profile_view'),
)

or a more compact version which avoids wrapping within a term:

urlpatterns = patterns('', url(r'^$', 'homepage_view'), 
                       url(r'^/login/$', 'login_view'),
                       url(r'^/login/$', 'logout_view'),
                       url(r'^/user/(?P<username>\w+)/$', 'profile_view'))

Pathologically slow example

The following takes 25 seconds to reformat on my machine. In comparison, many significantly larger files run in under a second:

specialElements = frozenset((
    (namespaces['html'], 'address'),
    (namespaces['html'], 'applet'),
    (namespaces['html'], 'area'),
    (namespaces['html'], 'article'),
    (namespaces['html'], 'aside'),
    (namespaces['html'], 'base'),
    (namespaces['html'], 'basefont'),
    (namespaces['html'], 'bgsound'),
    (namespaces['html'], 'blockquote'),
    (namespaces['html'], 'body'),
    (namespaces['html'], 'br'),
    (namespaces['html'], 'button'),
    (namespaces['html'], 'caption'),
    (namespaces['html'], 'center'),
    (namespaces['html'], 'col'),
    (namespaces['html'], 'colgroup'),
    (namespaces['html'], 'command'),
    (namespaces['html'], 'dd'),
    (namespaces['html'], 'details'),
    (namespaces['html'], 'dir'),
    (namespaces['html'], 'div'),
    (namespaces['html'], 'dl'),
    (namespaces['html'], 'dt'),
    (namespaces['html'], 'embed'),
    (namespaces['html'], 'fieldset'),
    (namespaces['html'], 'figure'),
    (namespaces['html'], 'footer'),
    (namespaces['html'], 'form'),
    (namespaces['html'], 'frame'),
    (namespaces['html'], 'frameset'),
    (namespaces['html'], 'h1'),
    (namespaces['html'], 'h2'),
    (namespaces['html'], 'h3'),
    (namespaces['html'], 'h4'),
    (namespaces['html'], 'h5'),
    (namespaces['html'], 'h6'),
    (namespaces['html'], 'head'),
    (namespaces['html'], 'header'),
    (namespaces['html'], 'hr'),
    (namespaces['html'], 'html'),
    (namespaces['html'], 'iframe'),
    # Note that image is commented out in the spec as "this isn't an
    # element that can end up on the stack, so it doesn't matter,"
    (namespaces['html'], 'image'),
    (namespaces['html'], 'img'),
    (namespaces['html'], 'input'),
    (namespaces['html'], 'isindex'),
    (namespaces['html'], 'li'),
    (namespaces['html'], 'link'),
    (namespaces['html'], 'listing'),
    (namespaces['html'], 'marquee'),
    (namespaces['html'], 'menu'),
    (namespaces['html'], 'meta'),
    (namespaces['html'], 'nav'),
    (namespaces['html'], 'noembed'),
    (namespaces['html'], 'noframes'),
    (namespaces['html'], 'noscript'),
    (namespaces['html'], 'object'),
    (namespaces['html'], 'ol'),
    (namespaces['html'], 'p'),
    (namespaces['html'], 'param'),
    (namespaces['html'], 'plaintext'),
    (namespaces['html'], 'pre'),
    (namespaces['html'], 'script'),
    (namespaces['html'], 'section'),
    (namespaces['html'], 'select'),
    (namespaces['html'], 'style'),
    (namespaces['html'], 'table'),
    (namespaces['html'], 'tbody'),
    (namespaces['html'], 'td'),
    (namespaces['html'], 'textarea'),
    (namespaces['html'], 'tfoot'),
    (namespaces['html'], 'th'),
    (namespaces['html'], 'thead'),
    (namespaces['html'], 'title'),
    (namespaces['html'], 'tr'),
    (namespaces['html'], 'ul'),
    (namespaces['html'], 'wbr'),
    (namespaces['html'], 'xmp'),
    (namespaces['svg'], 'foreignObject')
))

E129 visually indented line with same indent as next logical line

yapf formats:

def response():
    if (defined and
            dirs is _dirs_undefined and dictionary is _dictionary_undefined):
        return to

as

def response():
    if (defined and dirs is _dirs_undefined and
        dictionary is _dictionary_undefined):
        return to

pep8 checker gives E129. I also agree with it that this looks misleading.

pyformat thinks the original is correct and leaves it untouched. I would personally have formatted as

def response():
    if (
        defined and
        dirs is _dirs_undefined and dictionary is _dictionary_undefined
    ):
        return to

but I'm not wedded to that and suspect you like whitespace less than I do. :-)

Invalid unicode will crash yapf with an exception rather than an error message

In the continuing strategy of "here's a python string literal for a file exhibiting this problem to avoid github being clever", the following string passes ast.parse (in python 3.4.2) but causes yapf to crash when a file with precisely these contents is passed to it and run under the same python version. I think this is because yapf has an assumption baked in that all source is valid utf-8.

String:

"# а\x91а\x96б\x9fб\x80б\x81б\x82б\x83б\x84б\x85б\x86б\x87б\x88б\x89б\x8aб\x8bб\x8cб\x8dб\x8eб\x8f <- Cyrillic characters\n'а\x8eб\x82т\x84\x96аЄ'\n"

Error:

INTERNAL ERROR: # а�а�б�б�б�б�б�б�б
б�б�б�б�б�б�б�б�б�б� <- Cyrillic characters
Traceback (most recent call last):
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 38, in VerifyCode
    compile(textwrap.dedent(code).encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 2
    б�б�б�б�б�б�б�б�б�б� <- Cyrillic characters
                       ^
SyntaxError: invalid character in identifier

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 41, in VerifyCode
    ast.parse(textwrap.dedent(code.lstrip('\n')).lstrip(), '<string>', 'exec')
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<string>", line 2
    б�б�б�б�б�б�б�б�б�б� <- Cyrillic characters
                       ^
SyntaxError: invalid character in identifier

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "/home/david/yapf/yapf/__init__.py", line 102, in main
    print_diff=args.diff)
  File "/home/david/yapf/yapf/__init__.py", line 124, in FormatFiles
    filename, style_config=style_config, lines=lines, print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 67, in FormatFile
    print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 110, in FormatCode
    reformatted_source = reformatter.Reformat(uwlines)
  File "/home/david/yapf/yapf/yapflib/reformatter.py", line 73, in Reformat
    verifier.VerifyCode(formatted_code[-1])
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 45, in VerifyCode
    compile(normalized_code.encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 1
    б�б�б�б�б�б�б�б�б�б� <- Cyrillic characters
                       ^
SyntaxError: invalid character in identifier

yapf --style=pep8 will line break in the middle of a keyword argument (E251)

The following pep8 correct code:

class A(object):
    def b(self):
        if self.aaaaaaaaaaaaaaaaaaaa not in self.bbbbbbbbbb(
            cccccccccccccccccccc=True
        ):
            pass

Will be reformatted to the following non pep8 compliant (and ugly) code:

class A(object):
    def b(self):
        if self.aaaaaaaaaaaaaaaaaaaa not in self.bbbbbbbbbb(cccccccccccccccccccc
                                                            =True):
            pass

The line break around the equals is not kosher. pep8 requires keeping kwargs together without spaces.

exec used as a method breaks lib2to3

Formatting the following crashes (under both 2.7.8 and 3.4.2):

methods.exec(sys.modules[name])

The error is:

lib2to3.pgen2.parse.ParseError: bad input: type=1, value='exec', context=('', (1, 8))

Running 2to3 on the same code produces the same parse error, so this is a limitation of the library.

yapf: disable disables just enough to generate invalid syntax

The following syntactically valid python:

class SplitPenaltyTest(unittest.TestCase):
  def testUnbreakable(self):
    self._CheckPenalties(tree, [
    ])  # yapf: disable

Is reformatted into the following invalid python:

class SplitPenaltyTest(unittest.TestCase):
    def testUnbreakable(self):
    self._CheckPenalties(tree, [
    ])  # yapf: disable

yapf cannot handle 3.x metaclass syntax when run under 2.7

The following code will produce an internal error when yapf is run under 2.7:

class ABC(metaclass=type):
    pass

The error is:

INTERNAL ERROR: class ABC(metaclass=type):
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "/home/david/yapf/yapf/__init__.py", line 102, in main
    print_diff=args.diff)
  File "/home/david/yapf/yapf/__init__.py", line 124, in FormatFiles
    filename, style_config=style_config, lines=lines, print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 67, in FormatFile
    print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 110, in FormatCode
    reformatted_source = reformatter.Reformat(uwlines)
  File "/home/david/yapf/yapf/yapflib/reformatter.py", line 73, in Reformat
    verifier.VerifyCode(formatted_code[-1])
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 45, in VerifyCode
    compile(normalized_code.encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 1
    class ABC(metaclass=type):
                       ^
SyntaxError: invalid syntax

Exception when formatting list comprehensions on Python 3

The following code causes an error when yapf is run under python 3 (I've tried on both 3.2 and 3.4):

def given(y):
    [k for k in () if k in y]

It gives the following error:

Traceback (most recent call last):
  File "/home/david/.pyenv/versions/3.4.2/bin/yapf", line 11, in <module>
    sys.exit(run_main())
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/site-packages/yapf/__init__.py", line 154, in run_main
    sys.exit(main(sys.argv))
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/site-packages/yapf/__init__.py", line 102, in main
    print_diff=args.diff)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/site-packages/yapf/__init__.py", line 124, in FormatFiles
    filename, style_config=style_config, lines=lines, print_diff=print_diff)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/site-packages/yapf/yapflib/yapf_api.py", line 68, in FormatFile
    print_diff=print_diff)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/site-packages/yapf/yapflib/yapf_api.py", line 101, in FormatCode
    uwl.CalculateFormattingInformation()
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/site-packages/yapf/yapflib/unwrapped_line.py", line 77, in CalculateFormattingInformation
    token.split_penalty += _SplitPenalty(prev_token, token)
  File "/home/david/.pyenv/versions/3.4.2/lib/python3.4/site-packages/yapf/yapflib/unwrapped_line.py", line 342, in _SplitPenalty
    if cur_token.node_split_penalty > 0:
TypeError: unorderable types: NoneType() > int()

I've tried this both against the latest version on pip and the current version of master.

yapf --style=pep8 sometimes inserts a blank line it shouldn't

This code is pep8 correct, because timeout is a local function definition and thus has only one blank line in front of it:

import signal


try:
    signal.SIGALRM
    # ..................................................................
    # ...............................................................

    def timeout(seconds=1):
        pass
except:
    pass

However yapf reformats it as:

import signal

try:
    signal.SIGALRM

    # ..................................................................
    # ...............................................................


    def timeout(seconds=1):
        pass
except:
    pass

With two blank lines before timeout.

pep8 checker says "E303 too many blank lines (2)"

yapf does not strip blank lines from the end of a file

I suppose this one is more of a feature request. pyformat will strip blank lines from the end of a file (pep8 checker reports the blank line at the end of the file as W391), but yapf will not, meaning a file that could easily be formatted into PEP8 compliance will not be.

yapf with --style=pep8 puts imports on one line even if that breaks the character limit

Consider e.g.

from __future__ import division, print_function, absolute_import, unicode_literals

(there's nothing special about this being a from future import, it's just an example of a long import from my project)

This import line is more than 82 characters but yapf puts it all on one line. It does this even if the code was originally broken across multiple lines.

Feature request: Error early on unsupported python versions

i.e. just exit immediately with a big message saying "yapf doesn't support running on this version of python", mostly so you don't get twits like me running it on 3.2 and getting confused when it produces strange error messages later. :-)

Closing bracket can be lined up with the wrong opening bracket

This produces the same pep error as #26 but I think is not the same bug. Obviously feel free to close it as a duplicate if I'm wrong.

The following code:

class A(object):

    def f(self, aaaaaaaaa, bbbbbbbbbbbbb, row):
        if True:
            if row[4] is None or row[5] is None:
                bbbbbbbbbbbbb['..............'] = row[
                    5] if row[5] is not None else 5

is reformatted by yapf --style=pep8 as

class A(object):
    def f(self, aaaaaaaaa, bbbbbbbbbbbbb, row):
        if True:
            if row[4] is None or row[5] is None:
                bbbbbbbbbbbbb['..............'] = row[5] if row[5
                             ] is not None else 5

Note that the closing bracket on the following line is being lined up with the opening bracket from an entirely different indexing expression. This is both a pep8 violation and also rather misleading.

PEP8 style recommends two spaces before comment, yapf does one.

From an email from Terry Reedy:

[...] style.py has

The number of spaces required before a trailing comment.

SPACES_BEFORE_COMMENT=1,

PEP8 actually says "Inline comments should be separated by at least two spaces from the statement.", which is what you have in the google style.

Please don't recommend 'sudo python setup.py install'

Installing packages system-wide as root is problematic for several reasons. (Clobbering or shadowing system versions of the module, system packages clobbering or shadowing user-installed modules, version management issues.)

Instead of recommending 'sudo', recommend users use 'python setup.py install --user' -- or better yet, a virtualenv + pip.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.