GithubHelp home page GithubHelp logo

redbaron's People

Contributors

acieroid avatar astalaseven avatar b5y avatar cbenz avatar duncf avatar eyalev avatar felixonmars avatar gotgenes avatar graingert avatar gtors avatar hanaasagi avatar ibizaman avatar jwilk avatar kgashok avatar mineo avatar monkeywithacupcake avatar movermeyer avatar psycojoker avatar remram44 avatar rojaster avatar scottbelden avatar sylvainde avatar techdragon avatar toddrme2178 avatar vstinner avatar webknjaz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redbaron's Issues

Questionable behavior for NodeList.index_on_parent

Due to the (this line)[https://github.com/Psycojoker/redbaron/blob/master/redbaron.py#L935], the following code can insert lines at wrong places:

item.parent.insert(item.index_on_parent, foo)

This is because the index_on_parent was used by the "path" feature. The path specific behavior should be refactored inside the Path class.

A temporary fix is to use:

item.parent.insert(item.parent.index(item), foo)

Cannot parse line continuations onto an empty line

Python accepts code like this:

>>> 3\
... 
3

But RedBaron does not:

$ python -c 'import redbaron; redbaron.RedBaron("3\\")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/rmmh/.local/lib/python2.7/site-packages/redbaron.py", line 2717, in __init__
    self.node_list = NodeList.from_fst(baron.parse(source_code), parent=self, on_attribute="root")
  File "/home/rmmh/.local/lib/python2.7/site-packages/baron/baron.py", line 51, in parse
    tokens = tokenize(source_code, False)
  File "/home/rmmh/.local/lib/python2.7/site-packages/baron/baron.py", line 72, in tokenize
    return mark_indentation(inner_group(space_group(_tokenize(group(split(pouet)), print_function))))
  File "/home/rmmh/.local/lib/python2.7/site-packages/baron/inner_formatting_grouper.py", line 112, in group
    return list(group_generator(sequence))
  File "/home/rmmh/.local/lib/python2.7/site-packages/baron/inner_formatting_grouper.py", line 189, in group_generator
    raise UnExpectedFormattingToken(debug_text)
baron.inner_formatting_grouper.UnExpectedFormattingToken: Unexpected 'space' token:

   1 3\
   2 <--- here

Should have been grouped on either ('INT', '3') (before) or ('ENDMARKER', '') (after) token.

Node.decrease_indentation does not work

Strings can only be added, not subtracted:

>>> RedBaron('''
... f(a=b,
...     c=d)''').find_all('call_argument').map(lambda x: x.decrease_indentation(2))
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/Library/Python/2.7/site-packages/redbaron.py", line 386, in map
    return NodeList([function(x) for x in self.data])
  File "<stdin>", line 3, in <lambda>
  File "/Library/Python/2.7/site-packages/redbaron.py", line 1010, in decrease_indentation
    self.get_indentation_node().indent -= number_of_spaces * " "
TypeError: unsupported operand type(s) for -=: 'str' and 'str'

whitespace changes on code insertion

when i insert code, fr some reason whitespace in all parts of the affected file gets added in the form of newlines before functions, classes and sometimes inline comments

Deleting a node deletes more than needed

Using redbaron from master branch, deleting a node deletes more than I wanted:

>>> red = RedBaron('''def inline_temp(red_cache, jscript):
    """
    :type red_cache: RedCache
    :type jscript: jedi.Script
    """
    assigns = jscript.goto_assignments()
    if not assigns:
        raise NoAssignmentsFound()
    if len(assigns) > 1:
        raise MultipleAssignmentsFound()
    jassign = assigns[0]
    rdef = jdef_to_rdef(jassign, red_cache)
    rdef = up_to_assignment(rdef)
    assigned_expr_raw = rdef.value.dumps()
    rusages = collect_usages(jscript, red_cache)
    for rusage in rusages:
        if rusage.parent == rdef:
            continue
        rusage.replace(assigned_expr_raw)

    # del rdef.parent[rdef.index_on_parent]
    index = rdef.index_on_parent
    par = rdef.parent
    # this line triggers a bug: some parts are deleted, but they shouldn't be deleted
    del par[index]
    return red_cache
''')
>>> red[0]
def inline_temp(red_cache, jscript):
    """
    :type red_cache: RedCache
    :type jscript: jedi.Script
    """
    assigns = jscript.goto_assignments()
    if not assigns:
        raise NoAssignmentsFound()
    if len(assigns) > 1:
        raise MultipleAssignmentsFound()
    jassign = assigns[0]
    rdef = jdef_to_rdef(jassign, red_cache)
    rdef = up_to_assignment(rdef)
    assigned_expr_raw = rdef.value.dumps()
    rusages = collect_usages(jscript, red_cache)
    for rusage in rusages:
        if rusage.parent == rdef:
            continue
        rusage.replace(assigned_expr_raw)

    # del rdef.parent[rdef.index_on_parent]
    index = rdef.index_on_parent
    par = rdef.parent
    # this line triggers a bug: some parts are deleted, but they shouldn't be deleted
    del par[index]
    return red_cache
>>> red[0][8]
rusages = collect_usages(jscript, red_cache)
>>> del red[0][8]
>>> red[0]
def inline_temp(red_cache, jscript):
    """
    :type red_cache: RedCache
    :type jscript: jedi.Script
    """
    assigns = jscript.goto_assignments()
    if not assigns:
        raise NoAssignmentsFound()
    if len(assigns) > 1:
        raise MultipleAssignmentsFound()
    jassign = assigns[0]
    rdef = jdef_to_rdef(jassign, red_cache)
    rdef = up_to_assignment(rdef)
    assigned_expr_raw = rdef.value.dumps()
    for rusage in rusages:
        if rusage.parent == rdef:
            continue
        rusage.replace(assigned_expr_raw)

    # del rdef.parent[rdef.index_on_parent]

As you can see, I only wanted to delete the line rusages = collect_usages(jscript, red_cache), but redbaron also deleted everything after # del rdef.parent[rdef.index_on_parent].
Using redbaron 0.5.1 also deletes more than needed (but not exactly the same way as master branch).

Tuple white space problem

Hello,

this code: print("__loader__ present:", __loader__ is not None) was false modified
by the following code: tuple_node.value.node_list[-1].second_formatting = ""

This worked for many cases, as far I can see, but in this case, of tuple (not having
print_function from __future__ there, btw), it fails and changes code to:
print("__loader__ present:", __loader__ is notNone)

Not sure, if that space should be removable, and would be a bug itself. What I
then did was to do formatted output with help of these:

print("__loader__ present:", __loader__ is not None)
print("__loader__ present:", __loader__ is not None )

Notice the trailing space of the tuple, the one I am trying to remove.

node.help(deep = True, with_formatting = True) on these gives the same output.

Not so sure, if second_formatting is the wrong space here, but I am nearly sure, it's not OK for the trailing space to not show up.

Please advise,
Kay

Long integer literal crashes RedBaron

This is valid Python2 code, but not to RedBaron. It may not happen all the time, maybe just in this context.

3L
Traceback (most recent call last):
  File "./misc/autoformat.py", line 12, in <module>
    red = RedBaron(source_code.read().rstrip()+"\n")
  File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 2597, in __init__
    self.node_list = NodeList.from_fst(baron.parse(source_code), parent=self, on_attribute="root")
  File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 266, in from_fst
    return klass(map(lambda x: Node.from_fst(x, parent=parent, on_attribute=on_attribute), node_list), parent=parent, on_attribute=on_attribute)
  File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 266, in <lambda>
    return klass(map(lambda x: Node.from_fst(x, parent=parent, on_attribute=on_attribute), node_list), parent=parent, on_attribute=on_attribute)
  File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 437, in from_fst
    return globals()[class_name](node, parent=parent, on_attribute=on_attribute)
  File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 2163, in __init__
    self.value = int(self.value)
ValueError: invalid literal for int() with base 10: '3L'

find() does not find empty call node

As reported by @ronxin here PyCQA/baron#75

I try to find the first function call in a block of code:

from redbaron import RedBaron
red = RedBaron('f()')
print red.find('call')

I was expecting an empty CallNode (i.e., ()), but instead I got None.

However, findAll seems to be able to find empty call nodes:

print red.findAll('call')[0]

which gives me () as expected.

Just wanted to confirm if this is intentional, as the behavior seems a bit counter-intuitive.

Comment not parsed

Maybe this is connected to issue #67 but it's worse than just having the wrong node type, because it breaks the round trip promise:

In [27]: source = '''#!/usr/bin/env python
# coding: utf8
'''

In [28]: red = RedBaron(source)

In [29]: red
Out[29]: 
0   '#!/usr/bin/env python\n'
1   '#!/usr/bin/env python\n'


In [30]: red[0]
Out[30]: '#!/usr/bin/env python\n'

In [31]: red[1]
Out[31]: '#!/usr/bin/env python\n'

In [32]: red.dumps()
Out[32]: '#!/usr/bin/env python\n\n'

The second node seemingly repeats the first comment but when rendered back to source even that is gone, as is the coding comment. Tested with Python 2.7, baron 0.6.2, and redbaron 0.6.1

Can't get return in assignments

Here is code"

a = \
    2

It is impossible to get return. Here is what I am doing:

red = RedBaron(_source_)
assignments = red.find_all('AssignmentNode')
for ind, body_line in enumerate(assignments.data):
    if body_line.second_formatting.data[0].value == '\n':
        print True

It doesn't print True. Is it bug or I am wrong? How can I detect that assignment is defined in a separated lines?

[Doc] Terminal color codes makes examples harder to read

Terminal color code in documentation makes things harder to read

Example: from https://redbaron.readthedocs.org/en/latest/index.html :

In [5]: red.help()  # helper function that discribe nodes content so you don't have to read the doc
0 -----------------------------------------------------
�[38;5;148mAssignmentNode�[39m�[38;5;197m(�[39m�[38;5;197m)�[39m
�[38;5;15m  �[39m�[38;5;242m# identifiers: assign, assignment, assignment_, assignmentnode�[39m
�[38;5;15m  �[39m�[38;5;15moperator�[39m�[38;5;197m=�[39m�[38;5;186m''�[39m
�[38;5;15m  �[39m�[38;5;15mtarget�[39m�[38;5;15m �[39m�[38;5;197m->�[39m
�[38;5;15m    �[39m�[38;5;148mNameNode�[39m�[38;5;197m(�[39m�[38;5;197m)�[39m
�[38;5;15m      �[39m�[38;5;242m# identifiers: name, name_, namenode�[39m
�[38;5;15m      �[39m�[38;5;15mvalue�[39m�[38;5;197m=�[39m�[38;5;186m'some_value'�[39m
�[38;5;15m  �[39m�[38;5;15mvalue�[39m�[38;5;15m �[39m�[38;5;197m->�[39m
�[38;5;15m    �[39m�[38;5;148mIntNode�[39m�[38;5;197m(�[39m�[38;5;197m)�[39m
�[38;5;15m      �[39m�[38;5;242m# identifiers: int, int_, intnode�[39m
�[38;5;15m      �[39m�[38;5;15mvalue�[39m�[38;5;197m=�[39m�[38;5;186m'42'�[39m

I'd much rather see:

In [5]: red.help()  # helper function that discribe nodes content so you don't have to read the doc
0 -----------------------------------------------------
AssignmentNode()
  # identifiers: assign, assignment, assignment_, assignmentnode
  operator=''
  target ->
    NameNode()
      # identifiers: name, name_, namenode
      value='some_value'
  value ->
    IntNode()
      # identifiers: int, int_, intnode
      value='42'

handle indented cases in CommaProxyList

Quite a lot of work, we need to list/detect common situation where we could find an identation in a CommaList and see how we have to behave. Here is an example of a failing situation.

A "quick win" would be to have a default behavior (even a bad one) instead of an exception (which sucks).

In [7]: RedBaron("def __init__(self, a,\n   b, c): pass")[0].arguments.pop()
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-7-4d3f9b02d5b0> in <module>()
----> 1 RedBaron("def __init__(self, a,\n   b, c): pass")[0].arguments.pop()

/home/psycojoker/code/python/redbaron/redbaron/base_nodes.pyc in pop(self, index)
   1348         else:
   1349             self.data.pop()
-> 1350         self._synchronise()
   1351 
   1352     def remove(self, value):

/home/psycojoker/code/python/redbaron/redbaron/base_nodes.pyc in _synchronise(self)
   1325 
   1326     def _synchronise(self):
-> 1327         self.node_list.data = self._generate_expected_list()[:]
   1328         self.data = self._build_inner_list(self.node_list.data)
   1329 

/home/psycojoker/code/python/redbaron/redbaron/base_nodes.pyc in _generate_expected_list(self)
   1493                     if self.style == "indented":
   1494                         if not expected_list[-1].second_formatting.endl:
-> 1495                             raise Exception("Unhandled case")
   1496                         elif expected_list[-1].second_formatting.endl.indent != self.parent.indentation + " "*4:
   1497                             expected_list[-1].second_formatting.endl.indent = self.parent.indentation + " "*4

Exception: Unhandled case

Cannot modify whitespace from function declarations

Hello,

having a function:

def f( a, b ):
pass

I would like to set third_formatting and fourth_formatting on it, but it says "unhandled case", when I set this to "". My intent is to write code, which automatically formats function definition arguments to
contain new-lines as necessary only, and no leading or trailing space for arguments.

Can you please add support for these.

RedBaron fails to handle dict comprehension

An example speaks for itself:

>>> baron.parse("{k: v for k, v in {0: 1}.items()}")
[{'first_formatting': [], 'fourth_formatting': [], 'second_formatting': [], 'third_formatting': [], 'generators': [{'first_formatting': [{'type': 'space', 'value': ' '}], 'third_formatting': [{'type': 'space', 'value': ' '}], 'target': {'type': 'atomtrailers', 'value': [{'first_formatting': [], 'fourth_formatting': [], 'value': [{'first_formatting': [], 'type': 'dictitem', 'value': {'section': 'number', 'type': 'int', 'value': '1'}, 'key': {'section': 'number', 'type': 'int', 'value': '0'}, 'second_formatting': [{'type': 'space', 'value': ' '}]}], 'second_formatting': [], 'third_formatting': [], 'type': 'dict'}, {'first_formatting': [], 'type': 'dot', 'second_formatting': []}, {'type': 'name', 'value': 'items'}, {'first_formatting': [], 'fourth_formatting': [], 'value': [], 'second_formatting': [], 'third_formatting': [], 'type': 'call'}]}, 'iterator': {'first_formatting': [], 'fourth_formatting': [], 'value': [{'type': 'name', 'value': 'k'}, {'first_formatting': [], 'type': 'comma', 'second_formatting': [{'type': 'space', 'value': ' '}]}, {'type': 'name', 'value': 'v'}], 'second_formatting': [], 'third_formatting': [], 'with_parenthesis': False, 'type': 'tuple'}, 'ifs': [], 'fourth_formatting': [{'type': 'space', 'value': ' '}], 'type': 'comprehension_loop', 'second_formatting': [{'type': 'space', 'value': ' '}]}], 'result': {'first_formatting': [], 'value': {'type': 'name', 'value': 'v'}, 'key': {'type': 'name', 'value': 'k'}, 'second_formatting': [{'type': 'space', 'value': ' '}]}, 'type': 'dict_comprehension'}]
>>> red = RedBaron("{k: v for k, v in {0: 1}.items()}")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "redbaron.py", line 296, in __init__
    self.data = map(to_node, baron.parse(source_code))
  File "redbaron.py", line 17, in to_node
    return type(class_name, (Node,), {})(node)
  File "redbaron.py", line 77, in __init__
    setattr(self, key, to_node(value))
  File "redbaron.py", line 13, in to_node
    class_name = "".join(map(lambda x: x.capitalize(), node["type"].split("_"))) + "Node"
KeyError: 'type'

-- note

baron.parse("{k: v for k, v in {0: 1}.items()}")[0]["result"]

Should have a "type" key with the value "dictitem".

parent is not set on nested find_all

from redbaron import RedBaron

template = """\
class A(object):
    a = 1
    b = 2
    c = 3
"""

red = RedBaron(template)
B = red[0].copy()
red[0].insert_after(B)

print B.find('assign').index_on_parent
print B.find_all('assign')[0].index_on_parent
print B('assign')[0].index_on_parent
print '-----'
print B.find_all('assign').find_all('name', 'a').index_on_parent
print B('assign')('name', 'a').index_on_parent

output:

0
0
0
-----
None
None

I'm trying to write a function to change attributes of the class, e.g.

def set_attribute(red, name, value):
    assignment_node = red('assign')('name', name).parent
    assignment_node.value.value = -1

Python 3(.4) is doing weird annoying things with strings again

Python 3.4.2 (default, Oct  8 2014, 10:45:20) 
Type "copyright", "credits" or "license" for more information.

IPython 3.2.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from redbaron import RedBaron

In [2]: test = RedBaron("hello = 'Hello world!'\nprint(hello)")

In [3]: test
Out[3]: ---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/home/thomas/projets/ve/lib/python3.4/site-packages/IPython/core/formatters.py in __call__(self, obj)
    688                 type_pprinters=self.type_printers,
    689                 deferred_pprinters=self.deferred_printers)
--> 690             printer.pretty(obj)
    691             printer.flush()
    692             return stream.getvalue()

/home/thomas/projets/ve/lib/python3.4/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    407                             if callable(meth):
    408                                 return meth(obj, self, cycle)
--> 409             return _default_pprint(obj, self, cycle)
    410         finally:
    411             self.end_group()

/home/thomas/projets/ve/lib/python3.4/site-packages/IPython/lib/pretty.py in _default_pprint(obj, p, cycle)
    527     if _safe_getattr(klass, '__repr__', None) not in _baseclass_reprs:
    528         # A user-provided repr. Find newlines and replace them with p.break_()
--> 529         _repr_pprint(obj, p, cycle)
    530         return
    531     p.begin_group(1, '<')

/home/thomas/projets/ve/lib/python3.4/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    709     """A pprint that just redirects to the normal repr function."""
    710     # Find newlines and replace them with p.break_()
--> 711     output = repr(obj)
    712     for idx,output_line in enumerate(output.splitlines()):
    713         if idx:

/home/thomas/projets/ve/lib/python3.4/site-packages/redbaron.py in __repr__(self)
   1308     def __repr__(self):
   1309         if in_a_shell():
-> 1310             return self.__str__()
   1311 
   1312         return "<%s %s, \"%s\" %s, on %s %s>" % (

/home/thomas/projets/ve/lib/python3.4/site-packages/redbaron.py in __str__(self)
   1339         to_return = ""
   1340         for number, value in enumerate(self.data):
-> 1341             to_return += ("%-3s " % number) + "\n    ".join(value.__repr__().split("\n"))
   1342             to_return += "\n"
   1343         return to_return

TypeError: Type str doesn't support the buffer API

Apparently the fix is (.code stuff):

to_return += ("%-3s " % number) + "\n    ".join(value.__repr__().decode("Utf-8").split("\n"))

Test needs to be written.

parent of IfelseblockNode doesn't work correctly

I have the current test case:

import urllib, sys, string
from string import maketrans

bbb = 0

f = urllib.urlopen("http://www.pythonchallenge.com/pc/def/equality.html")
while 1:
    buf = f.read(200000)
    if not len(buf):
        break
    for x in range(len(buf)):
        if buf[x] in string.ascii_lowercase:
           if buf[x+1] in string.ascii_uppercase:
               if buf[x-1] in string.ascii_uppercase:
                   if buf[x+2] in string.ascii_uppercase:
                       if buf[x-2] in string.ascii_uppercase:
                           if buf[x+3] in string.ascii_uppercase:
                               if buf[x-3] in string.ascii_uppercase:
                                   if buf[x+4] in string.ascii_lowercase:
                                       if buf[x-4] in string.ascii_lowercase:
                                           bbb = x
    sys.stdout.write(buf)
    print(buf[bbb-3:bbb+4])

And I am trying to iterate through IfelseblockNode to the parent node to count nesting depth. I know that this node is unaffected.

When I get all IfelseblockNodes from code and jump to parent every time, something go wrong on step where if buf[x+3] in string.ascii_uppercase: is defined. I don't know why (yet), but it counts twice current parent and only after that goes higher.
It happens when it starts from if buf[x-4] in string.ascii_lowercase: node.

Here how I took all nodes:
if_else_node = red.find_all('IfelseblockNode')

And how I iterate through nodes to the parent node (this is piece of code - may be changed by request):

while parent:
    if isinstance(parent.parent, redbaron.RedBaron) or isinstance(parent, redbaron.RedBaron):
        break
    else:
        parent = parent.parent
        k += parent.index_on_parent_raw

String are greedy about surrounding formatting tokens, this create inconsistance

From #37

For example:

call( 1 )

And:

call( "a" )

Will have different formatting strings repartitions which is not acceptable.

Details differences:

For show('call( 1 )'):

[
    {
        "type": "atomtrailers", 
        "value": [
            {
                "type": "name", 
                "value": "call"
            }, 
            {
                "first_formatting": [], 
                "fourth_formatting": [], 
                "value": [
                    {
                        "first_formatting": [], 
                        "type": "call_argument", 
                        "target": {}, 
                        "value": {
                            "section": "number", 
                            "type": "int", 
                            "value": "1"
                        }, 
                        "second_formatting": []
                    }
                ], 
                "second_formatting": [
                    {
                        "type": "space", 
                        "value": " "
                    }
                ], 
                "third_formatting": [
                    {
                        "type": "space", 
                        "value": " "
                    }
                ], 
                "type": "call"
            }
        ]
    }
]

For call( "a" ):

[
    {
        "type": "atomtrailers", 
        "value": [
            {
                "type": "name", 
                "value": "call"
            }, 
            {
                "first_formatting": [], 
                "fourth_formatting": [], 
                "value": [
                    {
                        "first_formatting": [], 
                        "type": "call_argument", 
                        "target": {}, 
                        "value": {
                            "first_formatting": [], 
                            "type": "string", 
                            "value": "\"a\"", 
                            "second_formatting": [
                                {
                                    "type": "space", 
                                    "value": " "
                                }
                            ]
                        }, 
                        "second_formatting": []
                    }
                ], 
                "second_formatting": [
                    {
                        "type": "space", 
                        "value": " "
                    }
                ], 
                "third_formatting": [], 
                "type": "call"
            }
        ]
    }
]

Replacing ImportNode with FromImport Results in Error

I've been using RedBaron to modify some import statements, I was successful in being able to change FromImportNodes into ImportNodes, but not the other way around. When trying to make this change:

import flask.ext.foo --> from flask_foo import foo

I got an error which is in the below gist, as is the code that caused the problem. I was able to get something working for my purposes, but the prompt asked for an issue submission so here it is!

https://gist.github.com/keyanp/a9f86e3bd8f7feb68fbc

insert_before() fails with comment

The insert_before() method does not succeed if its string contains a single comment. Same deal for insert_after().

For example:

>>> s = """
... def f():
...   a = 10
...   for i in range(10):
...     j += i
... """
>>> 
>>> r=redbaron.RedBaron(s)
>>> fornode=r.find_all('ForNode')[0]
>>> fornode.insert_before('#comment')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1013, in insert_before
    self.parent.insert(self.index_on_parent - offset, value)
  File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1242, in insert
    value = self._convert_input_to_node_object(value, parent=self.node_list, on_attribute=self.on_attribute)
  File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1184, in _convert_input_to_node_object
    return self.node_list.parent._convert_input_to_node_object_list(value, parent, on_attribute).filtered()[0]
  File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 165, in _convert_input_to_node_object_list
    return self._string_to_node_list(value, parent=parent, on_attribute=on_attribute)
  File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 2172, in _string_to_node_list
    return super(DefNode, self)._string_to_node_list(string, parent, on_attribute)
  File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1022, in _string_to_node_list
    return self.parse_code_block(string, parent=parent, on_attribute=on_attribute)
  File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1038, in parse_code_block
    fst = baron.parse("def a():\n%s\n" % clean_string)[0]["value"]
  File "/usr/local/lib/python2.7/site-packages/baron/baron.py", line 51, in parse
    tokens = tokenize(source_code, False)
  File "/usr/local/lib/python2.7/site-packages/baron/baron.py", line 72, in tokenize
    return mark_indentation(inner_group(space_group(_tokenize(group(split(pouet)), print_function))))
  File "/usr/local/lib/python2.7/site-packages/baron/indentation_marker.py", line 24, in mark_indentation
    return list(mark_indentation_generator(sequence))
  File "/usr/local/lib/python2.7/site-packages/baron/indentation_marker.py", line 67, in mark_indentation_generator
    indentations.append(get_space(i))
  File "/usr/local/lib/python2.7/site-packages/baron/indentation_marker.py", line 31, in get_space
    if len(node[3]) == 0:
IndexError: tuple index out of range

[Doc] Same command twice in install documentation

Hi,
I've just started to read the doc (after an email about the project was sent to the python code quality mailing list) and found an tiny issue on :
https://redbaron.readthedocs.org/en/latest/index.html
This part repeats the same command line:

pip install redbaron[pygments]
Or if you don’t want to have syntax highlight in your shell or don’t need it:
pip install redbaron[pygments]

Which I assume to be not correct. For the reference, the readme says:

pip install redbaron[pygments]
Or if you don't want to have syntax highlight in your shell or don't need it:
pip install redbaron

Inconsistance in numerical node behavior

As far as I know, number literals can be one of IntNode, FloatNode, OctaNode, HexaNode, LongNode. However they don't appear in the code or doc, except for IntNode, which works differently because its value is a number instead of str.

So I'm left with:

def get_int_value(node):
    NUMBERS = IntNode, FloatNode, OctaNode, HexaNode, LongNode

    if isinstance(node, NUMBERS):
        if isinstance(node, IntNode):
            return node.value
        else:
            return ast.literal_eval(node.value)
    else:
        raise ValueError

code = RedBaron('a = 12\na = 0xC')
get_int_value(code[0].assign.value), get_int_value(code[1].assign.value)

Can't insert_after a node inserted previously

Test case:

from redbaron import RedBaron
rb = RedBaron('''
  class A(object):
      a = 1
      b = 2
      c = 3
''')

node_b = rb.find('assignment', target=lambda x: x.dumps() == 'b')
node_b.insert_after('d = -1')
node_d = rb.find('assignment', target=lambda x: x.dumps() == 'd')
node_d.insert_after('o = 3')

Result:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-681d36ee36d7> in <module>()
----> 1 node_d.insert_after('o = 3')

/usr/local/lib/python2.7/dist-packages/redbaron.py in insert_after(self, value, offset)
   1014 
   1015     def insert_after(self, value, offset=0):
-> 1016         self.parent.insert(self.index_on_parent + 1 + offset, value)
   1017 
   1018 

TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

Debugging the code, it seems the problem is that when d is inserted, the parent gets set to the node_list, instead of the ClassNode that the other attributes have as their parent. That prevents the index_on_parent from being calculated for that node.

The following patch seems to fix it, but I'm not sure it's safe:

diff --git a/redbaron.py b/redbaron.py
index 1890cf6..0aec875 100644
--- a/redbaron.py
+++ b/redbaron.py
@@ -1293,7 +1293,7 @@ class ProxyList(object):
         return len(self.data)

     def insert(self, index, value):
-        value = self._convert_input_to_node_object(value, parent=self.node_list, on_attribute=self.on_attribute)
+        value = self._convert_input_to_node_object(value, parent=self.parent, on_attribute=self.on_attribute)
         self.data.insert(index, value)
         self._diff_augmented_list()

Cannot remove spaces after called object

I was trying to remove spaces from inbetween called object and call brace:

from redbaron import RedBaron
red = RedBaron("called (it)")

call (it)

The "call" node seems to have it:

red[0]1
red[0][1].type
'call'

The node_list doesn't have it:

red[0][1].node_list
0 it
red[0][1].node_list[0]
it

So, I was trying this:

red[0][1].first_formatting = ""
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 1708, in setattr
super(CallNode, self).setattr(key, value)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 891, in setattr
value = self._convert_input_to_node_object_list(value, self, name)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 151, in _convert_input_to_node_object_list
return self._string_to_node_list(value, parent=parent, on_attribute=on_attribute)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 1705, in _string_to_node_list
raise Exception("Unhandled case")

I am assuming that this ought to work, second, and third formatting neither
work. Please consider implementing it. :-)

Can't use redbaron in IPython notebook

redbaron calls fileno() on sys.stdout which raises unhandled UnsupportedOperation.

---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
<ipython-input-10-7a726da5bd0a> in <module>()
----> 1 str(tree)

C:\Python2.7\lib\site-packages\redbaron.pyc in __str__(self)
   1270         to_return = ""
   1271         for number, value in enumerate(self.data):
-> 1272             to_return += ("%-3s " % number) + "\n    ".join(value.__repr__().split("\n"))
   1273             to_return += "\n"
   1274         return to_return

C:\Python2.7\lib\site-packages\redbaron.pyc in __repr__(self)
    852     def __repr__(self):
    853         # the isinstance here is for building sphinx doc
--> 854         if isinstance(sys.stdout, StringIO) or os.isatty(sys.stdout.fileno()):
    855             return self.__str__()
    856 

C:\Python2.7\lib\site-packages\IPython\kernel\zmq\iostream.pyc in fileno(self)
    192 
    193     def fileno(self):
--> 194         raise UnsupportedOperation("IOStream has no fileno.")
    195 
    196     def write(self, string):

UnsupportedOperation: IOStream has no fileno.

http://nbviewer.ipython.org/gist/remram44/449041cd67ad390037b8

Assignment to atomtrailers index handled incorrectly

Hi, I'm playing with the idea of creating a refactoring library on top of redbaron, and I've encountered a problem when assigning to an index of an AtomtrailersNode.

You can see for yourself:

>>> red = RedBaron('foo.real')
>>> red[0]
foo.real
>>> red[0].type
'atomtrailers'
>>> red[0][0] = 'random()'
>>> red[0] # expected result: random().real
().real
>>> red = RedBaron('foo.real')
>>> red[0][1] = 'random()'
>>> red[0] # expected result: foo.random()
foo()

Note that if I use replace, the result is as expected:

>>> red = RedBaron('foo.real')
>>> red[0][0].replace('random()')
>>> red[0]
random().real
>>> red = RedBaron('foo.real')
>>> red[0][1].replace('random()')
>>> red[0]
foo.random()

[Question, feature request?] After modifying the tree, how to print elements back ?

Hi,
So I'm using RedBaron to help me edit code, that's pretty exciting. My difficulty now is to re-write the tree I modified, sometimes just nodes. It doesn't seem to be a method on redbaron objects for that.

For instance, let's take the code

def foo(self, key=[], second): pass

I parse it, I sort the arguments and I want to print them back. It's easy with simple arguments that have no value, but with "key=[]" I don't find its textual representation, I find its "first_formatting" that is a python list.

{'first_formatting': [], 'type': 'def_argument', 'target': {'type': 'name', 'value': 'key'}, 'value': {'type': 'name', 'value': 'val'}, 'second_formatting': []}

So I could treat this "[]" as a special case and return the string "[]", but how will I handle a list with something in it, or a crazy argument with nested lists ?

Hope I was clear;
thanks !

broken in bpython repl

>>> import redbaron
>>> 
>>> src =open('app/controllers/......py').read()
>>> red  = redbaron.RedBaron(src)
>>> red
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File ".../site-packages/redbaron.py", line 1257, in __repr__
    if isinstance(sys.stdout, StringIO) or os.isatty(sys.stdout.fileno()):
AttributeError: 'FakeStream' object has no attribute 'fileno'

printing of RedBaron instances in ipython breaks in Python 3.

I haven't figured out exactly what set of interacting components causes this. Running redbaron==0.5.1 under ipython==3.1.0 on Python 3.4.3, the following code gives me an error:

from redbaron import RedBaron
RedBaron("1")

The error:

Out[5]: ---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/IPython/core/formatters.py in __call__(self, obj)
    688                 type_pprinters=self.type_printers,
    689                 deferred_pprinters=self.deferred_printers)
--> 690             printer.pretty(obj)
    691             printer.flush()
    692             return stream.getvalue()

/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    405                             if callable(meth):
    406                                 return meth(obj, self, cycle)
--> 407             return _default_pprint(obj, self, cycle)
    408         finally:
    409             self.end_group()

/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/IPython/lib/pretty.py in _default_pprint(obj, p, cycle)
    525     if _safe_getattr(klass, '__repr__', None) not in _baseclass_reprs:
    526         # A user-provided repr. Find newlines and replace them with p.break_()
--> 527         _repr_pprint(obj, p, cycle)
    528         return
    529     p.begin_group(1, '<')

/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    707     """A pprint that just redirects to the normal repr function."""
    708     # Find newlines and replace them with p.break_()
--> 709     output = repr(obj)
    710     for idx,output_line in enumerate(output.splitlines()):
    711         if idx:

/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/redbaron.py in __repr__(self)
   1308     def __repr__(self):
   1309         if in_a_shell():
-> 1310             return self.__str__()
   1311 
   1312         return "<%s %s, \"%s\" %s, on %s %s>" % (

/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/redbaron.py in __str__(self)
   1339         to_return = ""
   1340         for number, value in enumerate(self.data):
-> 1341             to_return += ("%-3s " % number) + "\n    ".join(value.__repr__().split("\n"))
   1342             to_return += "\n"
   1343         return to_return

TypeError: 'str' does not support the buffer interface

In [6]: str(RedBaron("1"))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-ed3dc85689a9> in <module>()
----> 1 str(RedBaron("1"))

/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/redbaron.py in __str__(self)
   1339         to_return = ""
   1340         for number, value in enumerate(self.data):
-> 1341             to_return += ("%-3s " % number) + "\n    ".join(value.__repr__().split("\n"))
   1342             to_return += "\n"
   1343         return to_return

TypeError: 'str' does not support the buffer interface

This seems to be a problem with ipython specifically. I can't reproduce it in the normal python console, nor by invoking str or repr explicitly.

def.arguments.__slice__ is buggy

# where z is a FuncdefNode

z.arguments.extend(z.arguments[1:])  # works

z.arguments[1:].extend(z.arguments[1:])

AttributeError                            Traceback (most recent call last)
<ipython-input-10-fa58bfbd216f> in <module>()
----> 1 z.arguments[1:].extend(z.arguments[1:])

/home/psycojoker/code/python/redbaron/redbaron.py in extend(self, values)
   1302 
   1303     def extend(self, values):
-> 1304         self.data.extend(self._convert_input_to_node_object_list(values, parent=self.node_list, on_attribute=self.on_attribute))
   1305         self._diff_augmented_list()
   1306 

/home/psycojoker/code/python/redbaron/redbaron.py in _convert_input_to_node_object_list(self, value, parent, on_attribute)
   1242             return self.node_list.parent._convert_input_to_node_object_list(value, parent, on_attribute)
   1243         else:
-> 1244             return NodeList([self._convert_input_to_node_object(x, parent, on_attribute) for x in value])
   1245 
   1246     def _generate_expected_list(self):

/home/psycojoker/code/python/redbaron/redbaron.py in _convert_input_to_node_object(self, value, parent, on_attribute)
   1232 
   1233     def _convert_input_to_node_object(self, value, parent, on_attribute):
-> 1234         lst = self.node_list.parent._convert_input_to_node_object_list(value, parent, on_attribute)
   1235         if all(i.type == 'endl' for i in lst):
   1236             return lst[0]

AttributeError: 'NoneType' object has no attribute '_convert_input_to_node_object_list'

AST does not correctly reflect left associativity of /

This might be more a problem for baron, but I did encounter it while using RedBaron.

>>> r = redbaron.RedBaron("4 / 5 / 6")
>>> r[0].help()                                                                              
BinaryOperatorNode()                                                                         
  # identifiers: binary_operator, binary_operator_, binaryoperator, binaryoperatornode       
  value='/'                                                                                  
  first ->                                                                                   
    IntNode()                                                                                
      # identifiers: int, int_, intnode                                                      
      value=4                                                                                
  second ->                                                                                  
    BinaryOperatorNode()                                                                     
      # identifiers: binary_operator, binary_operator_, binaryoperator, binaryoperatornode   
      value='/'                                                                              
      first ->                                                                               
        IntNode() ...                                                                        
      second ->                                                                              
        IntNode() ...                                                                        
>>> r[0].second.help()                                                                       
BinaryOperatorNode()                                                                         
  # identifiers: binary_operator, binary_operator_, binaryoperator, binaryoperatornode       
  value='/'                                                                                  
  first ->                                                                                   
    IntNode()                                                                                
      # identifiers: int, int_, intnode                                                      
      value=5                                                                                
  second ->                                                                                  
    IntNode()                                                                                
      # identifiers: int, int_, intnode                                                      
      value=6                                                                                

Or in other words the AST is

    [ / ]
    /    \
[ 4 ]    [ / ]
        /       \
      [ 5 ]   [ 6 ]

i.e. it's representing 4 / (5 / 6) rather than (4 / 5) / 6

The reason I care about this is I'm trying to perform a refactor that replaces instances of the / binary operator with a function div() that executes division instead. With the parse tree as is and a naive replacement of / with div(), things like 4 / 5 / 6 are getting translated as div(4, div(5, 6)).

I'm no parsing expert, but I would have thought the AST should reflect the associativity of the underlying grammar?

Cannot modify spaces after trailing comma

The following code:

(self, )

Note the trailing space after the comma. I was writing a modifier that removes trailing and leading spaces. And came up with this:

if "\n" not in tuple_node.dumps():
    tuple_node.second_formatting = ""
    tuple_node.third_formatting = ""

    for argument in tuple_node.value:
        if argument.type in ("string", "binary_string"):
            argument.second_formatting = ""

Note, that I find it strange, that I need to drop leading spaces from contained strings (same for
call nodes, strings seem to attract whitespace for no good reason, where I would naturally attribute it to the outside). I tried setting all four formattings to "x#", and this resulted a string containing them, plus still a space after the comma. Where is it hiding? The name typed argument doesn't indicate anything in its dict.

[Doc] Wrong link in README

(Code of Conduct)[http://meta.pycqa.org/en/latest/code-of-conduct.html].

should be

[Code of Conduct](http://meta.pycqa.org/en/latest/code-of-conduct.html).

Project structure, OOP, design patterns and style guide

What you are implementing is legendary tool to manipulate with code. But your hacking implementation is very uncomfortable to use and slowly. For instance, if you give to RedBaron file about 100 lines of logical code, it will works for more than 10 seconds.
I'm suggesting you to choose style guide and more OOP approach using design patterns.
Good example to start from is documentation of flask webframework.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.