pycqa / redbaron Goto Github PK
View Code? Open in Web Editor NEWBottom-up approach to refactoring in python
Home Page: http://redbaron.pycqa.org/
Bottom-up approach to refactoring in python
Home Page: http://redbaron.pycqa.org/
Due to the (this line)[https://github.com/Psycojoker/redbaron/blob/master/redbaron.py#L935], the following code can insert lines at wrong places:
item.parent.insert(item.index_on_parent, foo)
This is because the index_on_parent was used by the "path" feature. The path specific behavior should be refactored inside the Path class.
A temporary fix is to use:
item.parent.insert(item.parent.index(item), foo)
Python accepts code like this:
>>> 3\
...
3
But RedBaron does not:
$ python -c 'import redbaron; redbaron.RedBaron("3\\")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/rmmh/.local/lib/python2.7/site-packages/redbaron.py", line 2717, in __init__
self.node_list = NodeList.from_fst(baron.parse(source_code), parent=self, on_attribute="root")
File "/home/rmmh/.local/lib/python2.7/site-packages/baron/baron.py", line 51, in parse
tokens = tokenize(source_code, False)
File "/home/rmmh/.local/lib/python2.7/site-packages/baron/baron.py", line 72, in tokenize
return mark_indentation(inner_group(space_group(_tokenize(group(split(pouet)), print_function))))
File "/home/rmmh/.local/lib/python2.7/site-packages/baron/inner_formatting_grouper.py", line 112, in group
return list(group_generator(sequence))
File "/home/rmmh/.local/lib/python2.7/site-packages/baron/inner_formatting_grouper.py", line 189, in group_generator
raise UnExpectedFormattingToken(debug_text)
baron.inner_formatting_grouper.UnExpectedFormattingToken: Unexpected 'space' token:
1 3\
2 <--- here
Should have been grouped on either ('INT', '3') (before) or ('ENDMARKER', '') (after) token.
in a refactoring script i removed one named item in a multiline from import node,
on cases with parens and multiline, i got consistently garbled results
Strings can only be added, not subtracted:
>>> RedBaron('''
... f(a=b,
... c=d)''').find_all('call_argument').map(lambda x: x.decrease_indentation(2))
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
File "/Library/Python/2.7/site-packages/redbaron.py", line 386, in map
return NodeList([function(x) for x in self.data])
File "<stdin>", line 3, in <lambda>
File "/Library/Python/2.7/site-packages/redbaron.py", line 1010, in decrease_indentation
self.get_indentation_node().indent -= number_of_spaces * " "
TypeError: unsupported operand type(s) for -=: 'str' and 'str'
when i insert code, fr some reason whitespace in all parts of the affected file gets added in the form of newlines before functions, classes and sometimes inline comments
Using redbaron from master branch, deleting a node deletes more than I wanted:
>>> red = RedBaron('''def inline_temp(red_cache, jscript):
"""
:type red_cache: RedCache
:type jscript: jedi.Script
"""
assigns = jscript.goto_assignments()
if not assigns:
raise NoAssignmentsFound()
if len(assigns) > 1:
raise MultipleAssignmentsFound()
jassign = assigns[0]
rdef = jdef_to_rdef(jassign, red_cache)
rdef = up_to_assignment(rdef)
assigned_expr_raw = rdef.value.dumps()
rusages = collect_usages(jscript, red_cache)
for rusage in rusages:
if rusage.parent == rdef:
continue
rusage.replace(assigned_expr_raw)
# del rdef.parent[rdef.index_on_parent]
index = rdef.index_on_parent
par = rdef.parent
# this line triggers a bug: some parts are deleted, but they shouldn't be deleted
del par[index]
return red_cache
''')
>>> red[0]
def inline_temp(red_cache, jscript):
"""
:type red_cache: RedCache
:type jscript: jedi.Script
"""
assigns = jscript.goto_assignments()
if not assigns:
raise NoAssignmentsFound()
if len(assigns) > 1:
raise MultipleAssignmentsFound()
jassign = assigns[0]
rdef = jdef_to_rdef(jassign, red_cache)
rdef = up_to_assignment(rdef)
assigned_expr_raw = rdef.value.dumps()
rusages = collect_usages(jscript, red_cache)
for rusage in rusages:
if rusage.parent == rdef:
continue
rusage.replace(assigned_expr_raw)
# del rdef.parent[rdef.index_on_parent]
index = rdef.index_on_parent
par = rdef.parent
# this line triggers a bug: some parts are deleted, but they shouldn't be deleted
del par[index]
return red_cache
>>> red[0][8]
rusages = collect_usages(jscript, red_cache)
>>> del red[0][8]
>>> red[0]
def inline_temp(red_cache, jscript):
"""
:type red_cache: RedCache
:type jscript: jedi.Script
"""
assigns = jscript.goto_assignments()
if not assigns:
raise NoAssignmentsFound()
if len(assigns) > 1:
raise MultipleAssignmentsFound()
jassign = assigns[0]
rdef = jdef_to_rdef(jassign, red_cache)
rdef = up_to_assignment(rdef)
assigned_expr_raw = rdef.value.dumps()
for rusage in rusages:
if rusage.parent == rdef:
continue
rusage.replace(assigned_expr_raw)
# del rdef.parent[rdef.index_on_parent]
As you can see, I only wanted to delete the line rusages = collect_usages(jscript, red_cache)
, but redbaron also deleted everything after # del rdef.parent[rdef.index_on_parent]
.
Using redbaron 0.5.1 also deletes more than needed (but not exactly the same way as master branch).
Hello,
this code: print("__loader__ present:", __loader__ is not None)
was false modified
by the following code: tuple_node.value.node_list[-1].second_formatting = ""
This worked for many cases, as far I can see, but in this case, of tuple (not having
print_function from __future__
there, btw), it fails and changes code to:
print("__loader__ present:", __loader__ is notNone)
Not sure, if that space should be removable, and would be a bug itself. What I
then did was to do formatted output with help of these:
print("__loader__ present:", __loader__ is not None)
print("__loader__ present:", __loader__ is not None )
Notice the trailing space of the tuple, the one I am trying to remove.
node.help(deep = True, with_formatting = True)
on these gives the same output.
Not so sure, if second_formatting is the wrong space here, but I am nearly sure, it's not OK for the trailing space to not show up.
Please advise,
Kay
Hello,
this is also valid Python code: 5e5j
It's a complex literal and RedBaron needs to support it.
Yours,
Kay
This is valid Python2 code, but not to RedBaron. It may not happen all the time, maybe just in this context.
3L
Traceback (most recent call last):
File "./misc/autoformat.py", line 12, in <module>
red = RedBaron(source_code.read().rstrip()+"\n")
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 2597, in __init__
self.node_list = NodeList.from_fst(baron.parse(source_code), parent=self, on_attribute="root")
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 266, in from_fst
return klass(map(lambda x: Node.from_fst(x, parent=parent, on_attribute=on_attribute), node_list), parent=parent, on_attribute=on_attribute)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 266, in <lambda>
return klass(map(lambda x: Node.from_fst(x, parent=parent, on_attribute=on_attribute), node_list), parent=parent, on_attribute=on_attribute)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 437, in from_fst
return globals()[class_name](node, parent=parent, on_attribute=on_attribute)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 2163, in __init__
self.value = int(self.value)
ValueError: invalid literal for int() with base 10: '3L'
As reported by @ronxin here PyCQA/baron#75
I try to find the first function call in a block of code:
from redbaron import RedBaron red = RedBaron('f()') print red.find('call')
I was expecting an empty
CallNode
(i.e.,()
), but instead I gotNone
.However,
findAll
seems to be able to find empty call nodes:print red.findAll('call')[0]
which gives me
()
as expected.Just wanted to confirm if this is intentional, as the behavior seems a bit counter-intuitive.
Somehow I get an EndlNode that renders to a comment... really not sure what is going on here.
Maybe this is connected to issue #67 but it's worse than just having the wrong node type, because it breaks the round trip promise:
In [27]: source = '''#!/usr/bin/env python
# coding: utf8
'''
In [28]: red = RedBaron(source)
In [29]: red
Out[29]:
0 '#!/usr/bin/env python\n'
1 '#!/usr/bin/env python\n'
In [30]: red[0]
Out[30]: '#!/usr/bin/env python\n'
In [31]: red[1]
Out[31]: '#!/usr/bin/env python\n'
In [32]: red.dumps()
Out[32]: '#!/usr/bin/env python\n\n'
The second node seemingly repeats the first comment but when rendered back to source even that is gone, as is the coding comment. Tested with Python 2.7, baron 0.6.2, and redbaron 0.6.1
Here is code"
a = \
2
It is impossible to get return. Here is what I am doing:
red = RedBaron(_source_)
assignments = red.find_all('AssignmentNode')
for ind, body_line in enumerate(assignments.data):
if body_line.second_formatting.data[0].value == '\n':
print True
It doesn't print True. Is it bug or I am wrong? How can I detect that assignment is defined in a separated lines?
Terminal color code in documentation makes things harder to read
Example: from https://redbaron.readthedocs.org/en/latest/index.html :
In [5]: red.help() # helper function that discribe nodes content so you don't have to read the doc
0 -----------------------------------------------------
�[38;5;148mAssignmentNode�[39m�[38;5;197m(�[39m�[38;5;197m)�[39m
�[38;5;15m �[39m�[38;5;242m# identifiers: assign, assignment, assignment_, assignmentnode�[39m
�[38;5;15m �[39m�[38;5;15moperator�[39m�[38;5;197m=�[39m�[38;5;186m''�[39m
�[38;5;15m �[39m�[38;5;15mtarget�[39m�[38;5;15m �[39m�[38;5;197m->�[39m
�[38;5;15m �[39m�[38;5;148mNameNode�[39m�[38;5;197m(�[39m�[38;5;197m)�[39m
�[38;5;15m �[39m�[38;5;242m# identifiers: name, name_, namenode�[39m
�[38;5;15m �[39m�[38;5;15mvalue�[39m�[38;5;197m=�[39m�[38;5;186m'some_value'�[39m
�[38;5;15m �[39m�[38;5;15mvalue�[39m�[38;5;15m �[39m�[38;5;197m->�[39m
�[38;5;15m �[39m�[38;5;148mIntNode�[39m�[38;5;197m(�[39m�[38;5;197m)�[39m
�[38;5;15m �[39m�[38;5;242m# identifiers: int, int_, intnode�[39m
�[38;5;15m �[39m�[38;5;15mvalue�[39m�[38;5;197m=�[39m�[38;5;186m'42'�[39m
I'd much rather see:
In [5]: red.help() # helper function that discribe nodes content so you don't have to read the doc
0 -----------------------------------------------------
AssignmentNode()
# identifiers: assign, assignment, assignment_, assignmentnode
operator=''
target ->
NameNode()
# identifiers: name, name_, namenode
value='some_value'
value ->
IntNode()
# identifiers: int, int_, intnode
value='42'
See #52 for a test showing the problem.
test_bounding_box_with_proxy_list
works but test_bounding_box_of_attribute_with_proxy_list
doesn't.
Quite a lot of work, we need to list/detect common situation where we could find an identation in a CommaList and see how we have to behave. Here is an example of a failing situation.
A "quick win" would be to have a default behavior (even a bad one) instead of an exception (which sucks).
In [7]: RedBaron("def __init__(self, a,\n b, c): pass")[0].arguments.pop()
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
<ipython-input-7-4d3f9b02d5b0> in <module>()
----> 1 RedBaron("def __init__(self, a,\n b, c): pass")[0].arguments.pop()
/home/psycojoker/code/python/redbaron/redbaron/base_nodes.pyc in pop(self, index)
1348 else:
1349 self.data.pop()
-> 1350 self._synchronise()
1351
1352 def remove(self, value):
/home/psycojoker/code/python/redbaron/redbaron/base_nodes.pyc in _synchronise(self)
1325
1326 def _synchronise(self):
-> 1327 self.node_list.data = self._generate_expected_list()[:]
1328 self.data = self._build_inner_list(self.node_list.data)
1329
/home/psycojoker/code/python/redbaron/redbaron/base_nodes.pyc in _generate_expected_list(self)
1493 if self.style == "indented":
1494 if not expected_list[-1].second_formatting.endl:
-> 1495 raise Exception("Unhandled case")
1496 elif expected_list[-1].second_formatting.endl.indent != self.parent.indentation + " "*4:
1497 expected_list[-1].second_formatting.endl.indent = self.parent.indentation + " "*4
Exception: Unhandled case
Hello,
having a function:
def f( a, b ):
pass
I would like to set third_formatting and fourth_formatting on it, but it says "unhandled case", when I set this to "". My intent is to write code, which automatically formats function definition arguments to
contain new-lines as necessary only, and no leading or trailing space for arguments.
Can you please add support for these.
It would be nice to have some sort of autocompletion when working with a RedBaron node. At least:
red.<tab><tab>
red[0].<tab><tab>
Clues:
An example speaks for itself:
>>> baron.parse("{k: v for k, v in {0: 1}.items()}")
[{'first_formatting': [], 'fourth_formatting': [], 'second_formatting': [], 'third_formatting': [], 'generators': [{'first_formatting': [{'type': 'space', 'value': ' '}], 'third_formatting': [{'type': 'space', 'value': ' '}], 'target': {'type': 'atomtrailers', 'value': [{'first_formatting': [], 'fourth_formatting': [], 'value': [{'first_formatting': [], 'type': 'dictitem', 'value': {'section': 'number', 'type': 'int', 'value': '1'}, 'key': {'section': 'number', 'type': 'int', 'value': '0'}, 'second_formatting': [{'type': 'space', 'value': ' '}]}], 'second_formatting': [], 'third_formatting': [], 'type': 'dict'}, {'first_formatting': [], 'type': 'dot', 'second_formatting': []}, {'type': 'name', 'value': 'items'}, {'first_formatting': [], 'fourth_formatting': [], 'value': [], 'second_formatting': [], 'third_formatting': [], 'type': 'call'}]}, 'iterator': {'first_formatting': [], 'fourth_formatting': [], 'value': [{'type': 'name', 'value': 'k'}, {'first_formatting': [], 'type': 'comma', 'second_formatting': [{'type': 'space', 'value': ' '}]}, {'type': 'name', 'value': 'v'}], 'second_formatting': [], 'third_formatting': [], 'with_parenthesis': False, 'type': 'tuple'}, 'ifs': [], 'fourth_formatting': [{'type': 'space', 'value': ' '}], 'type': 'comprehension_loop', 'second_formatting': [{'type': 'space', 'value': ' '}]}], 'result': {'first_formatting': [], 'value': {'type': 'name', 'value': 'v'}, 'key': {'type': 'name', 'value': 'k'}, 'second_formatting': [{'type': 'space', 'value': ' '}]}, 'type': 'dict_comprehension'}]
>>> red = RedBaron("{k: v for k, v in {0: 1}.items()}")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "redbaron.py", line 296, in __init__
self.data = map(to_node, baron.parse(source_code))
File "redbaron.py", line 17, in to_node
return type(class_name, (Node,), {})(node)
File "redbaron.py", line 77, in __init__
setattr(self, key, to_node(value))
File "redbaron.py", line 13, in to_node
class_name = "".join(map(lambda x: x.capitalize(), node["type"].split("_"))) + "Node"
KeyError: 'type'
-- note
baron.parse("{k: v for k, v in {0: 1}.items()}")[0]["result"]
Should have a "type" key with the value "dictitem".
from redbaron import RedBaron
template = """\
class A(object):
a = 1
b = 2
c = 3
"""
red = RedBaron(template)
B = red[0].copy()
red[0].insert_after(B)
print B.find('assign').index_on_parent
print B.find_all('assign')[0].index_on_parent
print B('assign')[0].index_on_parent
print '-----'
print B.find_all('assign').find_all('name', 'a').index_on_parent
print B('assign')('name', 'a').index_on_parent
output:
0
0
0
-----
None
None
I'm trying to write a function to change attributes of the class, e.g.
def set_attribute(red, name, value):
assignment_node = red('assign')('name', name).parent
assignment_node.value.value = -1
Python 3.4.2 (default, Oct 8 2014, 10:45:20)
Type "copyright", "credits" or "license" for more information.
IPython 3.2.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: from redbaron import RedBaron
In [2]: test = RedBaron("hello = 'Hello world!'\nprint(hello)")
In [3]: test
Out[3]: ---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/thomas/projets/ve/lib/python3.4/site-packages/IPython/core/formatters.py in __call__(self, obj)
688 type_pprinters=self.type_printers,
689 deferred_pprinters=self.deferred_printers)
--> 690 printer.pretty(obj)
691 printer.flush()
692 return stream.getvalue()
/home/thomas/projets/ve/lib/python3.4/site-packages/IPython/lib/pretty.py in pretty(self, obj)
407 if callable(meth):
408 return meth(obj, self, cycle)
--> 409 return _default_pprint(obj, self, cycle)
410 finally:
411 self.end_group()
/home/thomas/projets/ve/lib/python3.4/site-packages/IPython/lib/pretty.py in _default_pprint(obj, p, cycle)
527 if _safe_getattr(klass, '__repr__', None) not in _baseclass_reprs:
528 # A user-provided repr. Find newlines and replace them with p.break_()
--> 529 _repr_pprint(obj, p, cycle)
530 return
531 p.begin_group(1, '<')
/home/thomas/projets/ve/lib/python3.4/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
709 """A pprint that just redirects to the normal repr function."""
710 # Find newlines and replace them with p.break_()
--> 711 output = repr(obj)
712 for idx,output_line in enumerate(output.splitlines()):
713 if idx:
/home/thomas/projets/ve/lib/python3.4/site-packages/redbaron.py in __repr__(self)
1308 def __repr__(self):
1309 if in_a_shell():
-> 1310 return self.__str__()
1311
1312 return "<%s %s, \"%s\" %s, on %s %s>" % (
/home/thomas/projets/ve/lib/python3.4/site-packages/redbaron.py in __str__(self)
1339 to_return = ""
1340 for number, value in enumerate(self.data):
-> 1341 to_return += ("%-3s " % number) + "\n ".join(value.__repr__().split("\n"))
1342 to_return += "\n"
1343 return to_return
TypeError: Type str doesn't support the buffer API
Apparently the fix is (.code stuff):
to_return += ("%-3s " % number) + "\n ".join(value.__repr__().decode("Utf-8").split("\n"))
Test needs to be written.
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 158, in _convert_input_to_node_object
raise NotImplemented
This should be NotImplementedError.
I have the current test case:
import urllib, sys, string
from string import maketrans
bbb = 0
f = urllib.urlopen("http://www.pythonchallenge.com/pc/def/equality.html")
while 1:
buf = f.read(200000)
if not len(buf):
break
for x in range(len(buf)):
if buf[x] in string.ascii_lowercase:
if buf[x+1] in string.ascii_uppercase:
if buf[x-1] in string.ascii_uppercase:
if buf[x+2] in string.ascii_uppercase:
if buf[x-2] in string.ascii_uppercase:
if buf[x+3] in string.ascii_uppercase:
if buf[x-3] in string.ascii_uppercase:
if buf[x+4] in string.ascii_lowercase:
if buf[x-4] in string.ascii_lowercase:
bbb = x
sys.stdout.write(buf)
print(buf[bbb-3:bbb+4])
And I am trying to iterate through IfelseblockNode to the parent node to count nesting depth. I know that this node is unaffected.
When I get all IfelseblockNodes from code and jump to parent every time, something go wrong on step where if buf[x+3] in string.ascii_uppercase:
is defined. I don't know why (yet), but it counts twice current parent and only after that goes higher.
It happens when it starts from if buf[x-4] in string.ascii_lowercase:
node.
Here how I took all nodes:
if_else_node = red.find_all('IfelseblockNode')
And how I iterate through nodes to the parent node (this is piece of code - may be changed by request):
while parent:
if isinstance(parent.parent, redbaron.RedBaron) or isinstance(parent, redbaron.RedBaron):
break
else:
parent = parent.parent
k += parent.index_on_parent_raw
From #37
For example:
call( 1 )
And:
call( "a" )
Will have different formatting strings repartitions which is not acceptable.
Details differences:
For show('call( 1 )')
:
[
{
"type": "atomtrailers",
"value": [
{
"type": "name",
"value": "call"
},
{
"first_formatting": [],
"fourth_formatting": [],
"value": [
{
"first_formatting": [],
"type": "call_argument",
"target": {},
"value": {
"section": "number",
"type": "int",
"value": "1"
},
"second_formatting": []
}
],
"second_formatting": [
{
"type": "space",
"value": " "
}
],
"third_formatting": [
{
"type": "space",
"value": " "
}
],
"type": "call"
}
]
}
]
For call( "a" )
:
[
{
"type": "atomtrailers",
"value": [
{
"type": "name",
"value": "call"
},
{
"first_formatting": [],
"fourth_formatting": [],
"value": [
{
"first_formatting": [],
"type": "call_argument",
"target": {},
"value": {
"first_formatting": [],
"type": "string",
"value": "\"a\"",
"second_formatting": [
{
"type": "space",
"value": " "
}
]
},
"second_formatting": []
}
],
"second_formatting": [
{
"type": "space",
"value": " "
}
],
"third_formatting": [],
"type": "call"
}
]
}
]
script.dumps()
works fine, but script[1:3].dumps()
seems to be missing some line endings...
I'm not sure if there's a good solution for this, but I habitually use a black-on-white terminal and some colors for redbaron in iPython are almost invisible against the light background.
I've been using RedBaron to modify some import statements, I was successful in being able to change FromImportNodes into ImportNodes, but not the other way around. When trying to make this change:
import flask.ext.foo
--> from flask_foo import foo
I got an error which is in the below gist, as is the code that caused the problem. I was able to get something working for my purposes, but the prompt asked for an issue submission so here it is!
Hello,
this crashes RedBaron:
[1
for d in e, f
for g in h
]
Removing the ",f" seems to make it work as do braces around "e, f", but they
are not required by the language.
Enjoy the riddle,
Kay
The insert_before() method does not succeed if its string contains a single comment. Same deal for insert_after().
For example:
>>> s = """
... def f():
... a = 10
... for i in range(10):
... j += i
... """
>>>
>>> r=redbaron.RedBaron(s)
>>> fornode=r.find_all('ForNode')[0]
>>> fornode.insert_before('#comment')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1013, in insert_before
self.parent.insert(self.index_on_parent - offset, value)
File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1242, in insert
value = self._convert_input_to_node_object(value, parent=self.node_list, on_attribute=self.on_attribute)
File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1184, in _convert_input_to_node_object
return self.node_list.parent._convert_input_to_node_object_list(value, parent, on_attribute).filtered()[0]
File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 165, in _convert_input_to_node_object_list
return self._string_to_node_list(value, parent=parent, on_attribute=on_attribute)
File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 2172, in _string_to_node_list
return super(DefNode, self)._string_to_node_list(string, parent, on_attribute)
File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1022, in _string_to_node_list
return self.parse_code_block(string, parent=parent, on_attribute=on_attribute)
File "/usr/local/lib/python2.7/site-packages/redbaron.py", line 1038, in parse_code_block
fst = baron.parse("def a():\n%s\n" % clean_string)[0]["value"]
File "/usr/local/lib/python2.7/site-packages/baron/baron.py", line 51, in parse
tokens = tokenize(source_code, False)
File "/usr/local/lib/python2.7/site-packages/baron/baron.py", line 72, in tokenize
return mark_indentation(inner_group(space_group(_tokenize(group(split(pouet)), print_function))))
File "/usr/local/lib/python2.7/site-packages/baron/indentation_marker.py", line 24, in mark_indentation
return list(mark_indentation_generator(sequence))
File "/usr/local/lib/python2.7/site-packages/baron/indentation_marker.py", line 67, in mark_indentation_generator
indentations.append(get_space(i))
File "/usr/local/lib/python2.7/site-packages/baron/indentation_marker.py", line 31, in get_space
if len(node[3]) == 0:
IndexError: tuple index out of range
Hi,
I've just started to read the doc (after an email about the project was sent to the python code quality mailing list) and found an tiny issue on :
https://redbaron.readthedocs.org/en/latest/index.html
This part repeats the same command line:
pip install redbaron[pygments]
Or if you don’t want to have syntax highlight in your shell or don’t need it:
pip install redbaron[pygments]
Which I assume to be not correct. For the reference, the readme says:
pip install redbaron[pygments]
Or if you don't want to have syntax highlight in your shell or don't need it:
pip install redbaron
As far as I know, number literals can be one of IntNode
, FloatNode
, OctaNode
, HexaNode
, LongNode
. However they don't appear in the code or doc, except for IntNode
, which works differently because its value is a number instead of str.
So I'm left with:
def get_int_value(node):
NUMBERS = IntNode, FloatNode, OctaNode, HexaNode, LongNode
if isinstance(node, NUMBERS):
if isinstance(node, IntNode):
return node.value
else:
return ast.literal_eval(node.value)
else:
raise ValueError
code = RedBaron('a = 12\na = 0xC')
get_int_value(code[0].assign.value), get_int_value(code[1].assign.value)
Test case:
from redbaron import RedBaron
rb = RedBaron('''
class A(object):
a = 1
b = 2
c = 3
''')
node_b = rb.find('assignment', target=lambda x: x.dumps() == 'b')
node_b.insert_after('d = -1')
node_d = rb.find('assignment', target=lambda x: x.dumps() == 'd')
node_d.insert_after('o = 3')
Result:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-681d36ee36d7> in <module>()
----> 1 node_d.insert_after('o = 3')
/usr/local/lib/python2.7/dist-packages/redbaron.py in insert_after(self, value, offset)
1014
1015 def insert_after(self, value, offset=0):
-> 1016 self.parent.insert(self.index_on_parent + 1 + offset, value)
1017
1018
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
Debugging the code, it seems the problem is that when d
is inserted, the parent gets set to the node_list
, instead of the ClassNode
that the other attributes have as their parent. That prevents the index_on_parent
from being calculated for that node.
The following patch seems to fix it, but I'm not sure it's safe:
diff --git a/redbaron.py b/redbaron.py
index 1890cf6..0aec875 100644
--- a/redbaron.py
+++ b/redbaron.py
@@ -1293,7 +1293,7 @@ class ProxyList(object):
return len(self.data)
def insert(self, index, value):
- value = self._convert_input_to_node_object(value, parent=self.node_list, on_attribute=self.on_attribute)
+ value = self._convert_input_to_node_object(value, parent=self.parent, on_attribute=self.on_attribute)
self.data.insert(index, value)
self._diff_augmented_list()
Surprisingly, those 2 (now 3) helpers aren't documented. Documentation should be pretty straightforward.
https://github.com/Psycojoker/redbaron/blob/master/redbaron.py#L141-149
I was trying to remove spaces from inbetween called object and call brace:
from redbaron import RedBaron
red = RedBaron("called (it)")
call (it)
The "call" node seems to have it:
red[0]1
red[0][1].type
'call'
The node_list doesn't have it:
red[0][1].node_list
0 it
red[0][1].node_list[0]
it
So, I was trying this:
red[0][1].first_formatting = ""
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 1708, in setattr
super(CallNode, self).setattr(key, value)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 891, in setattr
value = self._convert_input_to_node_object_list(value, self, name)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 151, in _convert_input_to_node_object_list
return self._string_to_node_list(value, parent=parent, on_attribute=on_attribute)
File "/usr/local/lib/python2.7/dist-packages/redbaron.py", line 1705, in _string_to_node_list
raise Exception("Unhandled case")
I am assuming that this ought to work, second, and third formatting neither
work. Please consider implementing it. :-)
redbaron calls fileno()
on sys.stdout
which raises unhandled UnsupportedOperation
.
---------------------------------------------------------------------------
UnsupportedOperation Traceback (most recent call last)
<ipython-input-10-7a726da5bd0a> in <module>()
----> 1 str(tree)
C:\Python2.7\lib\site-packages\redbaron.pyc in __str__(self)
1270 to_return = ""
1271 for number, value in enumerate(self.data):
-> 1272 to_return += ("%-3s " % number) + "\n ".join(value.__repr__().split("\n"))
1273 to_return += "\n"
1274 return to_return
C:\Python2.7\lib\site-packages\redbaron.pyc in __repr__(self)
852 def __repr__(self):
853 # the isinstance here is for building sphinx doc
--> 854 if isinstance(sys.stdout, StringIO) or os.isatty(sys.stdout.fileno()):
855 return self.__str__()
856
C:\Python2.7\lib\site-packages\IPython\kernel\zmq\iostream.pyc in fileno(self)
192
193 def fileno(self):
--> 194 raise UnsupportedOperation("IOStream has no fileno.")
195
196 def write(self, string):
UnsupportedOperation: IOStream has no fileno.
http://nbviewer.ipython.org/gist/remram44/449041cd67ad390037b8
This is valid Python2, but not for RedBaron.
def nested_args_function_with_defaults((a,b) = default_giver(), c = 5):
return a, b, c
long literals are written 123L its valid syntax, but cant be passd into int directly
Hi, I'm playing with the idea of creating a refactoring library on top of redbaron, and I've encountered a problem when assigning to an index of an AtomtrailersNode.
You can see for yourself:
>>> red = RedBaron('foo.real')
>>> red[0]
foo.real
>>> red[0].type
'atomtrailers'
>>> red[0][0] = 'random()'
>>> red[0] # expected result: random().real
().real
>>> red = RedBaron('foo.real')
>>> red[0][1] = 'random()'
>>> red[0] # expected result: foo.random()
foo()
Note that if I use replace
, the result is as expected:
>>> red = RedBaron('foo.real')
>>> red[0][0].replace('random()')
>>> red[0]
random().real
>>> red = RedBaron('foo.real')
>>> red[0][1].replace('random()')
>>> red[0]
foo.random()
Hi,
So I'm using RedBaron to help me edit code, that's pretty exciting. My difficulty now is to re-write the tree I modified, sometimes just nodes. It doesn't seem to be a method on redbaron objects for that.
For instance, let's take the code
def foo(self, key=[], second): pass
I parse it, I sort the arguments and I want to print them back. It's easy with simple arguments that have no value, but with "key=[]" I don't find its textual representation, I find its "first_formatting" that is a python list.
{'first_formatting': [], 'type': 'def_argument', 'target': {'type': 'name', 'value': 'key'}, 'value': {'type': 'name', 'value': 'val'}, 'second_formatting': []}
So I could treat this "[]" as a special case and return the string "[]", but how will I handle a list with something in it, or a crazy argument with nested lists ?
Hope I was clear;
thanks !
@ibizaman do you think that would be a good idea? funcdef bothers me for not being intuitive.
>>> import redbaron
>>>
>>> src =open('app/controllers/......py').read()
>>> red = redbaron.RedBaron(src)
>>> red
Traceback (most recent call last):
File "<input>", line 1, in <module>
File ".../site-packages/redbaron.py", line 1257, in __repr__
if isinstance(sys.stdout, StringIO) or os.isatty(sys.stdout.fileno()):
AttributeError: 'FakeStream' object has no attribute 'fileno'
I haven't figured out exactly what set of interacting components causes this. Running redbaron==0.5.1 under ipython==3.1.0 on Python 3.4.3, the following code gives me an error:
from redbaron import RedBaron
RedBaron("1")
The error:
Out[5]: ---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/IPython/core/formatters.py in __call__(self, obj)
688 type_pprinters=self.type_printers,
689 deferred_pprinters=self.deferred_printers)
--> 690 printer.pretty(obj)
691 printer.flush()
692 return stream.getvalue()
/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/IPython/lib/pretty.py in pretty(self, obj)
405 if callable(meth):
406 return meth(obj, self, cycle)
--> 407 return _default_pprint(obj, self, cycle)
408 finally:
409 self.end_group()
/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/IPython/lib/pretty.py in _default_pprint(obj, p, cycle)
525 if _safe_getattr(klass, '__repr__', None) not in _baseclass_reprs:
526 # A user-provided repr. Find newlines and replace them with p.break_()
--> 527 _repr_pprint(obj, p, cycle)
528 return
529 p.begin_group(1, '<')
/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
707 """A pprint that just redirects to the normal repr function."""
708 # Find newlines and replace them with p.break_()
--> 709 output = repr(obj)
710 for idx,output_line in enumerate(output.splitlines()):
711 if idx:
/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/redbaron.py in __repr__(self)
1308 def __repr__(self):
1309 if in_a_shell():
-> 1310 return self.__str__()
1311
1312 return "<%s %s, \"%s\" %s, on %s %s>" % (
/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/redbaron.py in __str__(self)
1339 to_return = ""
1340 for number, value in enumerate(self.data):
-> 1341 to_return += ("%-3s " % number) + "\n ".join(value.__repr__().split("\n"))
1342 to_return += "\n"
1343 return to_return
TypeError: 'str' does not support the buffer interface
In [6]: str(RedBaron("1"))
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-ed3dc85689a9> in <module>()
----> 1 str(RedBaron("1"))
/home/david/.pyenv/versions/3.4.3/lib/python3.4/site-packages/redbaron.py in __str__(self)
1339 to_return = ""
1340 for number, value in enumerate(self.data):
-> 1341 to_return += ("%-3s " % number) + "\n ".join(value.__repr__().split("\n"))
1342 to_return += "\n"
1343 return to_return
TypeError: 'str' does not support the buffer interface
This seems to be a problem with ipython specifically. I can't reproduce it in the normal python console, nor by invoking str or repr explicitly.
# where z is a FuncdefNode
z.arguments.extend(z.arguments[1:]) # works
z.arguments[1:].extend(z.arguments[1:])
AttributeError Traceback (most recent call last)
<ipython-input-10-fa58bfbd216f> in <module>()
----> 1 z.arguments[1:].extend(z.arguments[1:])
/home/psycojoker/code/python/redbaron/redbaron.py in extend(self, values)
1302
1303 def extend(self, values):
-> 1304 self.data.extend(self._convert_input_to_node_object_list(values, parent=self.node_list, on_attribute=self.on_attribute))
1305 self._diff_augmented_list()
1306
/home/psycojoker/code/python/redbaron/redbaron.py in _convert_input_to_node_object_list(self, value, parent, on_attribute)
1242 return self.node_list.parent._convert_input_to_node_object_list(value, parent, on_attribute)
1243 else:
-> 1244 return NodeList([self._convert_input_to_node_object(x, parent, on_attribute) for x in value])
1245
1246 def _generate_expected_list(self):
/home/psycojoker/code/python/redbaron/redbaron.py in _convert_input_to_node_object(self, value, parent, on_attribute)
1232
1233 def _convert_input_to_node_object(self, value, parent, on_attribute):
-> 1234 lst = self.node_list.parent._convert_input_to_node_object_list(value, parent, on_attribute)
1235 if all(i.type == 'endl' for i in lst):
1236 return lst[0]
AttributeError: 'NoneType' object has no attribute '_convert_input_to_node_object_list'
This might be more a problem for baron, but I did encounter it while using RedBaron.
>>> r = redbaron.RedBaron("4 / 5 / 6")
>>> r[0].help()
BinaryOperatorNode()
# identifiers: binary_operator, binary_operator_, binaryoperator, binaryoperatornode
value='/'
first ->
IntNode()
# identifiers: int, int_, intnode
value=4
second ->
BinaryOperatorNode()
# identifiers: binary_operator, binary_operator_, binaryoperator, binaryoperatornode
value='/'
first ->
IntNode() ...
second ->
IntNode() ...
>>> r[0].second.help()
BinaryOperatorNode()
# identifiers: binary_operator, binary_operator_, binaryoperator, binaryoperatornode
value='/'
first ->
IntNode()
# identifiers: int, int_, intnode
value=5
second ->
IntNode()
# identifiers: int, int_, intnode
value=6
Or in other words the AST is
[ / ]
/ \
[ 4 ] [ / ]
/ \
[ 5 ] [ 6 ]
i.e. it's representing 4 / (5 / 6) rather than (4 / 5) / 6
The reason I care about this is I'm trying to perform a refactor that replaces instances of the / binary operator with a function div() that executes division instead. With the parse tree as is and a naive replacement of / with div(), things like 4 / 5 / 6 are getting translated as div(4, div(5, 6)).
I'm no parsing expert, but I would have thought the AST should reflect the associativity of the underlying grammar?
The following code:
(self, )
Note the trailing space after the comma. I was writing a modifier that removes trailing and leading spaces. And came up with this:
if "\n" not in tuple_node.dumps():
tuple_node.second_formatting = ""
tuple_node.third_formatting = ""
for argument in tuple_node.value:
if argument.type in ("string", "binary_string"):
argument.second_formatting = ""
Note, that I find it strange, that I need to drop leading spaces from contained strings (same for
call nodes, strings seem to attract whitespace for no good reason, where I would naturally attribute it to the outside). I tried setting all four formattings to "x#", and this resulted a string containing them, plus still a space after the comma. Where is it hiding? The name typed argument doesn't indicate anything in its dict.
(Code of Conduct)[http://meta.pycqa.org/en/latest/code-of-conduct.html].
should be
[Code of Conduct](http://meta.pycqa.org/en/latest/code-of-conduct.html).
What you are implementing is legendary tool to manipulate with code. But your hacking implementation is very uncomfortable to use and slowly. For instance, if you give to RedBaron file about 100 lines of logical code, it will works for more than 10 seconds.
I'm suggesting you to choose style guide and more OOP approach using design patterns.
Good example to start from is documentation of flask webframework.
In https://redbaron.readthedocs.org/en/latest/tuto.html#misc-things, the examples for .map, .apply, and .filter show TypeErrors rather than output.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.