Applied mutation causes syntax error about mutmut

[boxed]

True, imports that are broken are for static analysis tools to handle. I'll fix it.

[boxed]

Well I agree this mutant is bad but if your test suite survives a syntax error something else is wrong here. If you apply this mutant to your source code your test suite should fail. Does it?

[konrad-kocik]

My tests fail if I apply this mutant manually and run my suite directly from pytest:

E       def except_to_bool(_func=None, /, exc=Exception, to=False):
E                                      ^
E   SyntaxError: invalid syntax

but they do not fail if mutant is applied by mutmut and suit is run by it:

(exceptbool) xxx@xxx:~/projects/exceptbool$ mutmut run

- Mutation testing starting - 

These are the steps:
1. A full test suite run will be made to make sure we 
   can run the tests successfully and we know how long 
   it takes (to detect infinite loops for example)
2. Mutants will be generated and checked

Mutants are written to the cache in the .mutmut-cache 
directory. Print found mutants with `mutmut results`.

Legend for output:
🎉 Killed mutants. The goal is for everything to end up in this bucket. 
⏰ Timeout. Test suite took 10 times as long as the baseline so were killed.  
🤔 Suspicious. Tests took a long time, but not long enough to be fatal. 
🙁 Survived. This means your tests needs to be expanded. 

1. Using cached time for baseline tests, to run baseline again delete the cache file

2. Checking mutants
⠇ 6/6  🎉 5  ⏰ 0  🤔 0  🙁 1

[boxed]

That is strange. Can you supply the code?

[konrad-kocik]

https://github.com/konrad-kocik/exceptbool

Tested method can be found here: exceptbool/exceptbool.py -> except_to_bool
Tests can be found here: tests/test_execeptbool.py
Setup your virtualenv with: pip install -r requirements_dev.txt

[boxed]

I agree this is a bug, but it's a performance issue mostly. When I run mutmut on exceptbool I get just one surviving mutant like you get but it's this one:

>>mutmut show 3
--- exceptbool/exceptbool.py
+++ exceptbool/exceptbool.py
@@ -18,7 +18,7 @@
     :type: bool
     """
     def decorator(func):
-        @wraps(func)
+
         def wrapper(*args, **kwargs):
             try:
                 func(*args, **kwargs)

You're applying a mutant that was killed:

> mutmut show 1
--- exceptbool/exceptbool.py
+++ exceptbool/exceptbool.py
@@ -1,7 +1,7 @@
 from functools import wraps
 
 
-def except_to_bool(_func=None, *, exc=Exception, to=False):
+def except_to_bool(_func=None, /, exc=Exception, to=False):
     """
     Makes decorated function return bool instead of raising an exception by
     converting given exception(s) into given bool value.

If you run just this mutant with mutmut run 1 it will report this as killed.

[konrad-kocik]

Ok, my mistake. I have interpreted the output from mutmut results incorrectly. First it prints number of surviving mutants, right? In my case it was 1 mutant and I incorrectly thought that was mutant id 😁 Later today I will apply surviving mutant that you have mentioned and see whether my tests fail from under pytest. Pozdrawiam, Konrad Kocik pon., 17 gru 2018, 11:08: Anders Hovmöller <[email protected]> napisał(a):

I agree this is a bug, but it's a performance issue mostly. When I run mutmut on exceptbool I get just one surviving mutant like you get but it's this one: >>mutmut show 3 --- exceptbool/exceptbool.py +++ exceptbool/exceptbool.py @@ -18,7 +18,7 @@ :type: bool """ def decorator(func): - @wraps(func) + def wrapper(*args, **kwargs): try: func(*args, **kwargs) You're applying a mutant that was killed: > mutmut show 1 --- exceptbool/exceptbool.py +++ exceptbool/exceptbool.py @@ -1,7 +1,7 @@ from functools import wraps -def except_to_bool(_func=None, *, exc=Exception, to=False): +def except_to_bool(_func=None, /, exc=Exception, to=False): """ Makes decorated function return bool instead of raising an exception by converting given exception(s) into given bool value. If you run just this mutant with mutmut run 1 it will report this as killed. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#52 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Arjf3oFqPmm1sxH3QzIw3AWoJ6d-dmYfks5u520XgaJpZM4ZVX_n> .

pon., 17 gru 2018, 11:08: Anders Hovmöller <[email protected]> napisał(a):

…

I agree this is a bug, but it's a performance issue mostly. When I run mutmut on exceptbool I get just one surviving mutant like you get but it's this one: >>mutmut show 3 --- exceptbool/exceptbool.py +++ exceptbool/exceptbool.py @@ -18,7 +18,7 @@ :type: bool """ def decorator(func): - @wraps(func) + def wrapper(*args, **kwargs): try: func(*args, **kwargs) You're applying a mutant that was killed: > mutmut show 1 --- exceptbool/exceptbool.py +++ exceptbool/exceptbool.py @@ -1,7 +1,7 @@ from functools import wraps -def except_to_bool(_func=None, *, exc=Exception, to=False): +def except_to_bool(_func=None, /, exc=Exception, to=False): """ Makes decorated function return bool instead of raising an exception by converting given exception(s) into given bool value. If you run just this mutant with mutmut run 1 it will report this as killed. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#52 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Arjf3oFqPmm1sxH3QzIw3AWoJ6d-dmYfks5u520XgaJpZM4ZVX_n> .

[boxed]

They will not ;)

[konrad-kocik]

Well, you were right, of course :) I needed to extend my test suite to kill that mutant. But this is exactly why I wanted to perform mutation tests in the first place! So thank you for that.

Final question: will you still try to solve this problem with transforming * into / in context of keyword arguments enforcement? Or will it stay that way?

[boxed]

It should be fixed yes. It's a performance issue (albeit a small one) if nothing else.

[nedbat]

BTW, with version 1.3.1, I have a survived mutant like this:

$ mutmut show
To apply a mutant on disk:
    mutmut apply <id>

To show a mutant:
    mutmut show <id>


Survived 🙁 (9)

---- src/templite.py (8) ----

10, 27, 35, 43, 44, 56, 106, 150

---- src/__init__.py (1) ----

153

$ mutmut show 153
--- src/__init__.py
+++ src/__init__.py
@@ -1,2 +1,2 @@
-from .templite import *
+from .templite import /

I've laid out my package wrong, which is why this survives the test suite, but it still is not a valid mutation.

[boxed]

Maybe I should have a special pass for finding files that can survive syntax errors because this stuff keeps happening :P

Yea, that should definitely be fixed.

[nklapste]

Maybe we should skip mutating any import lines.

Is there a way to detect lines defining import and from foo import bar statements within a parso.python.tree. If so we could add a rule to skip mutating these lines. I see no real value in messing up imports for mutation testing, it seems much too high level error injection to validate any real unit tests.