jellezijlstra / ast_decompiler Goto Github PK
View Code? Open in Web Editor NEWAn AST-to-code library for Python
License: Apache License 2.0
ast_decompiler's People
Contributors
Stargazers
Watchers
ast_decompiler's Issues
Remove relative imports from tests
While packaging this module for openSUSE, we have encountered this error:
[ 24s] + PYTHONPATH=/home/abuild/rpmbuild/BUILDROOT/python-ast-decompiler-0.7.0-0.x86_64/usr/lib/python3.8/site-packages
[ 24s] + PYTHONDONTWRITEBYTECODE=1
[ 24s] + pytest-3.8 --ignore=_build.python38 --ignore=_build.python39 --ignore=_build.python310 -v tests/
[ 24s] ============================= test session starts ==============================
[ 24s] platform linux -- Python 3.8.15, pytest-7.1.2, pluggy-1.0.0 -- /usr/bin/python3.8
[ 24s] cachedir: .pytest_cache
[ 24s] rootdir: /home/abuild/rpmbuild/BUILD/ast_decompiler-0.7.0
[ 24s] collecting ... collected 0 items / 7 errors
[ 24s]
[ 24s] ==================================== ERRORS ====================================
[ 24s] _____________________ ERROR collecting tests/test_basic.py _____________________
[ 24s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/ast_decompiler-0.7.0/tests/test_basic.py'.
[ 24s] Hint: make sure your test modules/packages have valid Python names.
[ 24s] Traceback:
[ 24s] /usr/lib64/python3.8/importlib/__init__.py:127: in import_module
[ 24s] return _bootstrap._gcd_import(name[level:], package, level)
[ 24s] tests/test_basic.py:3: in <module>
[ 24s] from .tests import assert_decompiles, check, only_on_version
[ 24s] E ImportError: attempted relative import with no known parent package
[ 24s] __________________ ERROR collecting tests/test_indentation.py __________________
[ 24s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/ast_decompiler-0.7.0/tests/test_indentation.py'.
[ 24s] Hint: make sure your test modules/packages have valid Python names.
[ 24s] Traceback:
[ 24s] /usr/lib64/python3.8/importlib/__init__.py:127: in import_module
[ 24s] return _bootstrap._gcd_import(name[level:], package, level)
[ 24s] tests/test_indentation.py:1: in <module>
[ 24s] from .tests import assert_decompiles
[ 24s] E ImportError: attempted relative import with no known parent package
[ 24s] __________________ ERROR collecting tests/test_line_length.py __________________
[ 24s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/ast_decompiler-0.7.0/tests/test_line_length.py'.
[ 24s] Hint: make sure your test modules/packages have valid Python names.
[ 24s] Traceback:
[ 24s] /usr/lib64/python3.8/importlib/__init__.py:127: in import_module
[ 24s] return _bootstrap._gcd_import(name[level:], package, level)
[ 24s] tests/test_line_length.py:1: in <module>
[ 24s] from .tests import assert_decompiles
[ 24s] E ImportError: attempted relative import with no known parent package
[ 24s] ____________________ ERROR collecting tests/test_literal.py ____________________
[ 24s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/ast_decompiler-0.7.0/tests/test_literal.py'.
[ 24s] Hint: make sure your test modules/packages have valid Python names.
[ 24s] Traceback:
[ 24s] /usr/lib64/python3.8/importlib/__init__.py:127: in import_module
[ 24s] return _bootstrap._gcd_import(name[level:], package, level)
[ 24s] tests/test_literal.py:1: in <module>
[ 24s] from .tests import assert_decompiles, only_on_version
[ 24s] E ImportError: attempted relative import with no known parent package
[ 24s] _____________________ ERROR collecting tests/test_patma.py _____________________
[ 24s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/ast_decompiler-0.7.0/tests/test_patma.py'.
[ 24s] Hint: make sure your test modules/packages have valid Python names.
[ 24s] Traceback:
[ 24s] /usr/lib64/python3.8/importlib/__init__.py:127: in import_module
[ 24s] return _bootstrap._gcd_import(name[level:], package, level)
[ 24s] tests/test_patma.py:1: in <module>
[ 24s] from .tests import check, skip_before
[ 24s] E ImportError: attempted relative import with no known parent package
[ 24s] __________________ ERROR collecting tests/test_precedence.py ___________________
[ 24s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/ast_decompiler-0.7.0/tests/test_precedence.py'.
[ 24s] Hint: make sure your test modules/packages have valid Python names.
[ 24s] Traceback:
[ 24s] /usr/lib64/python3.8/importlib/__init__.py:127: in import_module
[ 24s] return _bootstrap._gcd_import(name[level:], package, level)
[ 24s] tests/test_precedence.py:1: in <module>
[ 24s] from .tests import check, only_on_version
[ 24s] E ImportError: attempted relative import with no known parent package
[ 24s] __________________ ERROR collecting tests/test_py3_syntax.py ___________________
[ 24s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/ast_decompiler-0.7.0/tests/test_py3_syntax.py'.
[ 24s] Hint: make sure your test modules/packages have valid Python names.
[ 24s] Traceback:
[ 24s] /usr/lib64/python3.8/importlib/__init__.py:127: in import_module
[ 24s] return _bootstrap._gcd_import(name[level:], package, level)
[ 24s] tests/test_py3_syntax.py:1: in <module>
[ 24s] from .tests import check, skip_before, skip_after
[ 24s] E ImportError: attempted relative import with no known parent package
[ 24s] =========================== short test summary info ============================
[ 24s] ERROR tests/test_basic.py
[ 24s] ERROR tests/test_indentation.py
[ 24s] ERROR tests/test_line_length.py
[ 24s] ERROR tests/test_literal.py
[ 24s] ERROR tests/test_patma.py
[ 24s] ERROR tests/test_precedence.py
[ 24s] ERROR tests/test_py3_syntax.py
[ 24s] !!!!!!!!!!!!!!!!!!! Interrupted: 7 errors during collection !!!!!!!!!!!!!!!!!!!!
[ 24s] ============================== 7 errors in 0.11s ===============================
Complete build log showing all packages used with their versions and steps taken.
In my opinion the problem are relative imports. And really, when I apply this patch:
---
tests/test_basic.py | 2 +-
tests/test_indentation.py | 2 +-
tests/test_line_length.py | 2 +-
tests/test_literal.py | 2 +-
tests/test_patma.py | 2 +-
tests/test_precedence.py | 2 +-
tests/test_py3_syntax.py | 2 +-
7 files changed, 7 insertions(+), 7 deletions(-)
--- a/tests/test_basic.py
+++ b/tests/test_basic.py
@@ -1,6 +1,6 @@
import ast
from ast_decompiler import decompile
-from .tests import assert_decompiles, check, only_on_version
+from tests import assert_decompiles, check, only_on_version
def test_non_module() -> None:
--- a/tests/test_indentation.py
+++ b/tests/test_indentation.py
@@ -1,4 +1,4 @@
-from .tests import assert_decompiles
+from tests import assert_decompiles
def test_indentation() -> None:
--- a/tests/test_line_length.py
+++ b/tests/test_line_length.py
@@ -1,4 +1,4 @@
-from .tests import assert_decompiles
+from tests import assert_decompiles
def check_split(original: str, multiline: str, length_reduction: int = 2) -> None:
--- a/tests/test_literal.py
+++ b/tests/test_literal.py
@@ -1,4 +1,4 @@
-from .tests import assert_decompiles, only_on_version
+from tests import assert_decompiles, only_on_version
def test_With() -> None:
--- a/tests/test_patma.py
+++ b/tests/test_patma.py
@@ -1,4 +1,4 @@
-from .tests import check, skip_before
+from tests import check, skip_before
@skip_before((3, 10))
--- a/tests/test_precedence.py
+++ b/tests/test_precedence.py
@@ -1,4 +1,4 @@
-from .tests import check, only_on_version
+from tests import check, only_on_version
def test_Yield() -> None:
--- a/tests/test_py3_syntax.py
+++ b/tests/test_py3_syntax.py
@@ -1,4 +1,4 @@
-from .tests import check, skip_before, skip_after
+from tests import check, skip_before, skip_after
@skip_before((3, 5))then the problem goes away.
Goal of the project
Hi @JelleZijlstra! This project looks amazing, can you share some of the goals and needs of it?
I was thinking I could use it for a generating code, so my script would generate ast from something and then use ast_decompiler to generate python code, would it make sense? Or should I stick with using jinja for generating code? 😊
Unnecessary parens in subscripting
Currently, it produces:
def contextmanager(func: Callable[(_P, Iterator[_T])]) -> Callable[(_P, _GeneratorContextManager[_T])]:
The parentheses should not be there.
Nested f-strings aren't handled properly
Test cases like check("""f'{f"hello"}'""") fail. This is somewhat tricky to implement and it's not very pretty code, so I'm going to not worry about it for now.
Docstrings are not preserved correctly.
Currently it's impossible to correctly preserve docstrings. This test shows the issue:
from .tests import assert_decompiles
def test_docstrings():
assert_decompiles('''
def a():
"""
Docstring.
"""
return 1
''', '''def a():
"""
Docstring.
"""
return 1
''', indentation=1)
Result:
>>> expected
def a():
"""
Docstring.
"""
return 1
>>> actual
def a():
'\n Docstring.\n '
return 1
Thoughts: when processing the function body in write_function_def we should look at the very first node in the function's body. If the node is of type Expr, and contains a string, then it's most likely docstring of this function and should be processed accordingly.
Add license
Support python3.9
Suboptimal results when decompiling subscripts on Python <3.9
Minimal repro on Python 3.7:
>>> import ast
>>> from ast_decompiler import decompile
>>> decompile(ast.parse('Union[int, str]'))
'Union[(int, str)]\n'I'm guessing this is a side-effect of the fact that the representation of subscript slices changed in 3.9. Python 3.7. Previously:
>>> ast.dump(ast.parse('Union[int, str]').body[0].value)
"Subscript(value=Name(id='Union', ctx=Load()), slice=Index(value=Tuple(elts=[Name(id='int', ctx=Load()), Name(id='str', ctx=Load())], ctx=Load())), ctx=Load())"On more modern Pythons:
>>> ast.dump(ast.parse('Union[int, str]').body[0].value)
"Subscript(value=Name(id='Union', ctx=Load()), slice=Tuple(elts=[Name(id='int', ctx=Load()), Name(id='str', ctx=Load())], ctx=Load()), ctx=Load())"Redundant parentheses
Noticed the following:
lst[::-1]gets decompiled aslst[::-(1)][k for k, v in x]becomes[k for (k, v) in x]f(x for x in y)becomesf((x for x in y))
What is the license of this lib?
Without a license, we can use it only in toy projects, it's a shame, because I think it's a much needed library. Thank you!
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.