Incorrect choice of encoding Cyrillic characters resulting in UnicodeDecodeError about flake8-bandit

[AlexArcPy]

@sobolevn , good point on encoding! I am not yet confident with flake8 development so I'd have hard time working on receiving a new parameter in the constructor. :)
Regarding the tests in this repo, we won't be able to add any unless we know the pull request could be accepted - let's wait for @tylerwince to respond first?
Also, if we will have tests, would it be possible to add CI operation - the current repo does not seem to have any associated unless I am missing something.

[tylerwince]

Cheers @sobolevn and @AlexArcPy -- Thanks for the dialog above.

I would definitely welcome a PR for some tests and can get some CI stuff going with TravisCI once we they exist.

As for the encoding stuff - I have been meaning to change the way this works for a while. Unfortunately, different editors require the plugin to work differently to lint the file. I haven't had the time to go back and refactor lines https://github.com/tylerwince/flake8-bandit/blob/master/flake8_bandit.py#L64-L69 but that is where the issue lies.

If one of you guys has a good idea on how to remove the requirement of the additional file, I'd gladly accept a PR, otherwise I will take a crack at it this weekend.

Thanks for using the plugin! Glad it is useful

[sobolevn]

Hi, @tylerwince ! Thanks for writing this plugin. We are using it as a dependency in our linter: https://github.com/wemake-services/wemake-python-styleguide

So, this issue also affects us as well.

I am pretty sure that this error is caused by these lines:
https://github.com/tylerwince/flake8-bandit/blob/master/flake8_bandit.py#L62-L73

What can be done it improve it?

1. We can just receive lines from flake8 http://flake8.pycqa.org/en/latest/plugin-development/plugin-parameters.html#indicating-desired-data they should be already in the correct format/encoding
2. We can write some tests that russian letters now work, this will ensure that this won't happen again

@tylerwince Does this sound reasonable to you?
@AlexArcPy Are you willing to provide a PR? This would be awesome!

[AlexArcPy]

@sobolevn , I've tested that changing line 72
with open(self.filename) as f:
to
with io.open(self.filename, 'r', encoding='utf-8') as f:

makes it working with utf-8 files including having Cyrillic chars (both with magic encoding and without). Tested on both 2.7 and 3.6.

Nikita, did you mean to write a test in the wemake-python-styleguide repo?

[sobolevn]

@AlexArcPy no, I meant writing tests in this repo. Currently it does not have any 🙁

The problem with with io.open(self.filename, 'r', encoding='utf-8') as f: is that you are hardcoding an encoding. Encoding might differ for different locales and users. So, we should guess it instead.

flake8 does this for you, however. So, there is no need to do anything except just receiving a new parameter in the constructor.

[tylerwince]

@AlexArcPy @sobolevn I apologize for the delay on this. I am just getting around to working on this project. I have updated the way we parse the file for the next version (see branch https://github.com/tylerwince/flake8-bandit/tree/v2.0). It appears to have fixed the original issue on this so I suggest we close it and I am going to open up a new issue for testing.

Please reopen if you are still experiencing the same issue.