[Bug] UnicodeEncodeError when replying to a post or comment that has unicode characters - Python 2.7 about rtv

[Brobin]

Replicated again with the following:

rtv -l http://www.reddit.com/r/linuxmasterrace/comments/316in4/even_rpcmasterrace_knows_you_should_be_running/

Trace

File "build/bdist.linux-x86_64/egg/rtv/page.py", line 293, in _draw_content
  File "build/bdist.linux-x86_64/egg/rtv/submission.py", line 125, in draw_item
  File "build/bdist.linux-x86_64/egg/rtv/submission.py", line 148, in draw_comment
UnicodeEncodeError: 'ascii' codec can't encode character u'\u15e7' in position 15: ordinal not in range(128)

In this example, it is crashing becuase the flair contains a unicode character Antergos [sudo ᗧ -Syu]

I'm not sure exactly why this is happening. In helpers.py we set the error handling to 'replace' which should replace the unicode characters with ascii placeholders

encoding = 'utf-8' if config.unicode else 'ascii'
string = string.encode(encoding, 'replace')

[Brobin]

I think the issue might be formatting the string before calling the clean method

Changing line 148 of submission.py

text = clean('{flair} '.format(**data))

To this

text = '{0} '.format(clean(data['flair']))

Resolves the problem.

[michael-lazar]

Yup you're right, I just figured this out as well.

Calling '{flair}'.format(**data) in python2 tries to convert the unicode data to a native string before formatting it. You need to explicitly mark the type as unicode.

clean(u'{flair} '.format(**data))

09c4865. Thanks for the detailed bug report.

[Brobin]

Looks good. I'm glad I could help 👍