tadaboody / good_smell

A linting/refactoring library for python best practices and lesser-known tricks

Good Smell - it makes your code smell good!

A linting/refactoring library for python best practices and lesser-known tricks

---

This Tool tries to find bits of code that are possible to make more pythonic, more beautiful by using the language features and standard library functions you might not know about

For example Directly nested for loops (nested-for)

for i in seq_a:
    for j in seq_b:
        print(i, j)

will be flattened to a nested comprehension

for i, j in ((i,j) for i in seq_a for j in seq_b):
    print(i, j)

For a full list - check the list of implemented smells

Installing:

pip install good_smell 

Usage

To issue warnings, good_smell installs itself as a flake8 plugin with error codes starting with SML.

To automatically fix the code use good_smell fix:

good_smell fix PATH >PATH
good_smell fix PATH [--starting-line STARTING_LINE] [--end-line END_LINE]

Developing

See contributing guide

good_smell's Issues

Create project website

Nothing too fancy, something on github.io. A nice side project when the project is more well-formed

Run tests on open source samples

Ideally to check the library works (and is needed) I'd like to take some big open source projects, with defined test suites and many loops, and run the library on it finding statistics and checking the refactorings don't hurt any of the test suites

Replace nested for with product

New smell

Smelly code

for i in iter_a:
    for j in iter_b:
        ...

Fixed code

from itertools import product
for i, j in product(iter_a,iter_b):
    ...

Why it's smelly:

The less indent the better

Use with open instead of manually opening and closing file handler

New smell

Smelly code

fp = open(path,args,kwargs)
# Body
fp.close

Fixed code

with open(path,args,kwargs as fp:
    # Body

Why is it smelly?

Without a context manager you need to manually close the file, which is eror-prone.

Properly test warnings

Originally posted by @Tadaboody in #22
Maybe do this by creating an automatic test collector like pycodestyle has

Don't flatten a nested for with dependencies

For example in

good_smell/tests/test_flake8_ext.py

Lines 5 to 6 in 50c06d9

	for num in range(1, 5):
	for digits in range(1, 10 ** num):

This loop shouldn't be nested because they have a dependency between them - the target of the first loop num

Replace for-append with generator

New smell

Smelly code

li = list()
for x in other_iterable:
    # Body
    li += expr

Fixed code

def generate_li():
    # Body
    yield expr
li = list(generate_li())

Why is it smelly?

Not as smelly:tm: as other things, but in some cases it makes more sense, and more people should be aware of the possibility.

`smells` package not included in dist

[...]
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.7/site-packages/good_smell/__init__.py", line 3, in <module>
    from .smells import implemented_smells
ModuleNotFoundError: No module named 'good_smell.smells'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/sbin/flake8", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python3.7/site-packages/flake8/main/cli.py", line 16, in main
    app.run(argv)
  File "/usr/lib/python3.7/site-packages/flake8/main/application.py", line 412, in run
    self._run(argv)
  File "/usr/lib/python3.7/site-packages/flake8/main/application.py", line 399, in _run
    self.initialize(argv)
  File "/usr/lib/python3.7/site-packages/flake8/main/application.py", line 381, in initialize
    self.find_plugins()
  File "/usr/lib/python3.7/site-packages/flake8/main/application.py", line 197, in find_plugins
    self.check_plugins.load_plugins()
  File "/usr/lib/python3.7/site-packages/flake8/plugins/manager.py", line 434, in load_plugins
    plugins = list(self.manager.map(load_plugin))
  File "/usr/lib/python3.7/site-packages/flake8/plugins/manager.py", line 319, in map
    yield func(self.plugins[name], *args, **kwargs)
  File "/usr/lib/python3.7/site-packages/flake8/plugins/manager.py", line 432, in load_plugin
    return plugin.load_plugin()
  File "/usr/lib/python3.7/site-packages/flake8/plugins/manager.py", line 189, in load_plugin
    raise failed_to_load
flake8.exceptions.FailedToLoadPlugin: Flake8 failed to load plugin "SML" due to No module named 'good_smell.smells'.

➜ tree /usr/lib/python3.7/site-packages/good_smell
/usr/lib/python3.7/site-packages/good_smell
├── flake8_ext.py
├── __init__.py
├── lint_smell.py
├── main.py
├── __pycache__
│   ├── flake8_ext.cpython-37.pyc
│   ├── __init__.cpython-37.pyc
│   ├── lint_smell.cpython-37.pyc
│   ├── main.cpython-37.pyc
│   └── smell_warning.cpython-37.pyc
└── smell_warning.py

1 directory, 10 files

Simplify with yield from

New smell

Smelly code

for x in iterable:
    yield x
    ...

Fixed code

yield from iterable

Why is it smelly?

Flatter, simpler code

adding to an existing generator exp (important for chaining with #7 )

adding to an existing generator exp (important for chaining with #7 )

Originally posted by @Tadaboody in #40

Creating the error code calls log10(0)

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.7/site-packages/flake8/checker.py", line 682, in _run_checks
    return checker.run_checks()
  File "/usr/lib/python3.7/site-packages/flake8/checker.py", line 612, in run_checks
    self.run_ast_checks()
  File "/usr/lib/python3.7/site-packages/flake8/checker.py", line 520, in run_ast_checks
    for (line_number, offset, text, check) in runner:
  File "/usr/lib/python3.7/site-packages/good_smell/flake8_ext.py", line 29, in run
    for warning in warnings
  File "/usr/lib/python3.7/site-packages/good_smell/flake8_ext.py", line 29, in <genexpr>
    for warning in warnings
  File "/usr/lib/python3.7/site-packages/good_smell/flake8_ext.py", line 35, in leading_digit_str
    missing_0s = digits - int(math.log10(num)) - 1
ValueError: math domain error
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/sbin/flake8", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python3.7/site-packages/flake8/main/cli.py", line 16, in main
    app.run(argv)
  File "/usr/lib/python3.7/site-packages/flake8/main/application.py", line 412, in run
    self._run(argv)
  File "/usr/lib/python3.7/site-packages/flake8/main/application.py", line 400, in _run
    self.run_checks()
  File "/usr/lib/python3.7/site-packages/flake8/main/application.py", line 318, in run_checks
    self.file_checker_manager.run()
  File "/usr/lib/python3.7/site-packages/flake8/checker.py", line 338, in run
    self.run_parallel()
  File "/usr/lib/python3.7/site-packages/flake8/checker.py", line 302, in run_parallel
    for ret in pool_map:
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 774, in next
    raise value
ValueError: math domain error

Tuple object has no attribute id

When running good_smell on a project, I get this error:

Traceback (most recent call last):
  File "/usr/local/bin/flake8", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/flake8/main/cli.py", line 18, in main
    app.run(argv)
  File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 394, in run
    self._run(argv)
  File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 382, in _run
    self.run_checks()
  File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 301, in run_checks
    self.file_checker_manager.run()
  File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 330, in run
    self.run_serial()
  File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 314, in run_serial
    checker.run_checks()
  File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 608, in run_checks
    self.run_ast_checks()
  File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 504, in run_ast_checks
    for (line_number, offset, text, check) in runner:
  File "/usr/local/lib/python3.7/site-packages/good_smell/flake8_ext.py", line 20, in run
    warnings = smell(tree=self.tree, path=self.filename).check_for_smell()
  File "/usr/local/lib/python3.7/site-packages/good_smell/ast_smell.py", line 29, in check_for_smell
    transformer.visit(self.tree)
  File "/usr/local/lib/python3.7/site-packages/good_smell/ast_smell.py", line 20, in visit
    return self.generic_visit(node)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 317, in generic_visit
    value = self.visit(value)
  File "/usr/local/lib/python3.7/site-packages/good_smell/ast_smell.py", line 20, in visit
    return self.generic_visit(node)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 317, in generic_visit
    value = self.visit(value)
  File "/usr/local/lib/python3.7/site-packages/good_smell/ast_smell.py", line 22, in visit
    return super().visit(node)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/usr/local/lib/python3.7/site-packages/good_smell/smells/filter.py", line 42, in visit_For
    node.iter, node.target, gen_target)
  File "/usr/local/lib/python3.7/site-packages/good_smell/smells/filter.py", line 24, in replace_name_with_node
    return NameReplacer(old_val, new_val).visit(node)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 326, in generic_visit
    new_node = self.visit(old_value)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 326, in generic_visit
    new_node = self.visit(old_value)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 262, in visit
    return visitor(node)
  File "/usr/local/lib/python3.7/site-packages/good_smell/smells/filter.py", line 14, in visit_Name
    if node.id == self.old.id:
AttributeError: 'Tuple' object has no attribute ‘id'

Properly warn about smells like a linting tool

Right now the only functionality is as an all incompassing refactoring tool.
We set out to make a linter - lets make one!
Warning messages will be formatted similarly to those in pylint,pycodestyle and flake8. taking the best format.
Hopefully after doing this editor integration will be easy peasy

Convert AstSmell to use instancing instead of inheritance

Instead inheriting from ASTSmell and rewriting the same 3 properties, instancing a new AstSmell with the props as parameters would work nicely

Make mypy-compliant typing

This will both add security to the package and be a fun expirement

Move map to Iterator

New smell

Smelly code

for x in seq:
    a = transform(x)
    # Only use a or transform(x)

Fixed code

for a in (transform(x) for x in seq):
    # Only use a or transform(x)

Needs to be compatible with the rest of the smell fixes, especially #6

Use f-string instead of string.format

New smell

Smelly code

"a={},b={}".format(a,b)

Fixed code

f"a={a},b={b}"

Potential gotchas:

• warn only on py3.6+
• if the expression has quotes in it make sure to alternate single quotes '' and double quotes ""
• Don't warn if the formatted string isn't called directly (if the string is waiting for later)

Use dropwhile

New smell

Smelly code

for x in iterable:
    # No body
    if pred(x):
        continue
    # Body

Fixed code

import itertools
for x in itertools.dropwhile(pred,iterable):
    # Body

Why is it smelly?

Old for had iteration logic in the body, where business logic should be

Add a pre-commit hook config

Allow users to run the fixer with a pre-commit hook

Support replacing range arguments with enumerate

An extension of #4
The function range(start,stop,step) can be replaced with enumerate(islice(iterable,start,stop,step)) or even just islice if the index isn't used out of the assign.
for example

for i in range(3,len(seq),2):
    x = seq[i]
    ...

can be replaced with

for x in islice(seq,start=3,stop=None,step=2):
    ...

or in the case of a sequence (and not just an iterator) even

for x in seq[3:-1:2]:
    ...

Missing filter

New smell

Smelly code

for x in seq:
    if pred(x):
       # The whole body

Fixed code

for x in (x for x in seq if pred(x)):
       # The whole body

Missing takewhile

New smell

Smelly code

for x in seq:
    if pred(x):
        break
    # Body

Fixed code

from itertools import takewhile # If not already imported
for x in takewhile(seq,pred):
    # Body

Fix smells in-place

This should probably happen after #21

Support multiple python versions

Run tests for them with tox. Maybe use a backport for f-strings

Fix range_len_assign smell

New smell

Smelly code

for i in range(len(sequence)):
    x = sequence[i]
    ...

Fixed code

for i,x in enumerate(sequence):
    ...

Add black pre-commit hook

Add requirements.txt

Add requirements.txt in order to make it easy to install all the needed modules

Use argparse instead of fire for a cleaner interface and less dependancies

@MeitarR

Breaks other flake8 plugins by manipulating the AST.

See detailed explanation here:
jamescooke/flake8-aaa#142 (comment)

Create a CHANGELOG

Ideally automatically from pull requests and git tags

Use builtin breakpoint()

New smell

Smelly code

Import pdb; pdb.set_trace

Fixed code

breakpoint() 

Why is it smelly?

Have the fixer work with other auto-fomratters

Receive a path to the formatter executable and run it after fixing any smells

Imploy a Ci/Cd pipeline

This pipeline should

• run tests before every pull request
• deploy to pypi after every merge tag

Crash under very specific scenario

Description

Good Smell was crashing on one of my files and I managed to reduce it to a minimal reproducible case. It crashes only if there's a for using a range that takes a len and that inside the loop it appends to a list an object that is part of a list described in a Class. Code is below.

To Reproduce

1. Save the following file somewhere:

for i in range(len(random_a)):
    random_b.append(RandomClass.random_c[i])

2. Run good_smell warn on the file
3. Get error

Expected behavior

Good smell should not crash, especially when running with flake8 as it breaks all of flake8.

Desktop

• OS: Arch Linux and Windows 8.1
• Python version: 3.7.4
• Version good_smell 0.13.0

Additional context

Might be related with #61

Error log:

Traceback (most recent call last):
  File "C:\Program Files\Python37\Scripts\good_smell-script.py", line 11, in <module>
    load_entry_point('good-smell==0.13.0', 'console_scripts', 'good_smell')()
  File "c:\program files\python37\lib\site-packages\good_smell\main.py", line 46, in main
    Fire({"fix": print_fixed_smell, "warn": print_smell_warnings})
  File "c:\program files\python37\lib\site-packages\fire\core.py", line 127, in Fire
    component_trace = _Fire(component, args, context, name)
  File "c:\program files\python37\lib\site-packages\fire\core.py", line 366, in _Fire
    component, remaining_args)
  File "c:\program files\python37\lib\site-packages\fire\core.py", line 542, in _CallCallable
    result = fn(*varargs, **kwargs)
  File "c:\program files\python37\lib\site-packages\good_smell\main.py", line 14, in print_smell_warnings
    for warning in smell_warnings(Path(path).read_text(), path)
  File "c:\program files\python37\lib\site-packages\good_smell\main.py", line 13, in <genexpr>
    warning.warning_string()
  File "c:\program files\python37\lib\site-packages\good_smell\main.py", line 22, in smell_warnings
    source_code=source, path=str(path)
  File "c:\program files\python37\lib\site-packages\good_smell\ast_smell.py", line 29, in check_for_smell
    transformer.visit(self.tree)
  File "c:\program files\python37\lib\site-packages\good_smell\ast_smell.py", line 20, in visit
    return self.generic_visit(node)
  File "c:\program files\python37\lib\ast.py", line 317, in generic_visit
    value = self.visit(value)
  File "c:\program files\python37\lib\site-packages\good_smell\ast_smell.py", line 22, in visit
    return super().visit(node)
  File "c:\program files\python37\lib\ast.py", line 262, in visit
    return visitor(node)
  File "c:\program files\python37\lib\site-packages\good_smell\smells\range_len_fix.py", line 57, in visit_For
    new_body = deleter.visit(node).body or [ast.Pass()]
  File "c:\program files\python37\lib\ast.py", line 262, in visit
    return visitor(node)
  File "c:\program files\python37\lib\ast.py", line 317, in generic_visit
    value = self.visit(value)
  File "c:\program files\python37\lib\ast.py", line 262, in visit
    return visitor(node)
  File "c:\program files\python37\lib\ast.py", line 326, in generic_visit
    new_node = self.visit(old_value)
  File "c:\program files\python37\lib\ast.py", line 262, in visit
    return visitor(node)
  File "c:\program files\python37\lib\ast.py", line 317, in generic_visit
    value = self.visit(value)
  File "c:\program files\python37\lib\ast.py", line 262, in visit
    return visitor(node)
  File "c:\program files\python37\lib\site-packages\good_smell\smells\range_len_fix.py", line 46, in visit_Subscript
    if self.accesses_seq(node):
  File "c:\program files\python37\lib\site-packages\good_smell\smells\range_len_fix.py", line 40, in accesses_seq
    and node.value.id == self.seq.id
AttributeError: 'Attribute' object has no attribute 'id'

Add CONTRIBUTING guide

guide should have info on

• writing tests
• submitting PRs
• writing issues

Get rid of clunky generator in test collection

To get rid of https://github.com/Tadaboody/good_smell/blob/master/tests/test_collection.py#L34 we need to be able to stop collecting one test case before starting another one
So instead of formatting tests

#:<Test description>
#<Comma-seperated list of warning symbols emmited> or "None"
<Python code emitting the smells>
# ==>
<Python code after fixing the smell>
#:<next title>

Just format them like this

#:<Test description>
#<Comma-seperated list of warning symbols emmited> or "None"
<Python code emitting the smells>
# ==>
<Python code after fixing the smell>
# Special end symbol like END or !!
<Whatever>
#:<next title>

Add option to revert changes

This will probably be done by caching code before changes in a dot folder before overwriting a file, and restoring it from the cache later

Create plugins for existing linters

Probably the best call if we want it to be usable.

The refactoring tools will be an extra

Support comment flags to disable features

Docs link broken

The link to 'implemented smells' gives a 404 on both pypi.org and github.com READMEs.

Frozen requirements.txt breaks other plugins

If I install a set of requirements that includes for example:

good-smell
wemake-python-styleguide

Then good-smell forces specific version numbers in its dependencies, which results in breaking other projects which require newer releases. In this example, good-smell installs astor 0.7.1, but wemake will error without version >=0.8.

A more flexible requirements.txt would be nice to have here. In this case, I see no errors after forcing astor to upgrade to the latest 0.8.1 release.

Move smells to sub-directory

Move smells to sub-directory in order to make the project more organised and to make it possible to dynamically load all of them

Feature: Calling good_smell with multiple files

Feature Requested: Be able to call good_smell with multiple files or with bash wildcard *.py

e.g. good_smell *.py

Current behavior: good_smell is only able to be called with one file at a time