UnicodeEncodeError: 'latin-1' codec can't encode character about remi

[pybokeh]

To make it work for me, I changed line 90 in server.py from:

data = data.encode('latin-1')

to

data = data.encode('utf-8')

I'm not a developer and only have intermediate Python knowledge. So not sure if this is the proper solution or if it works with Python 2.X also.

[dddomodossola]

Hi pybokeh. First of all thank you for your exhaustive report. I'm unable to reproduce the error and furthermore, the code line you changed applies only to python version less than 3. Considering you are working with 3.4, the encoding you changed should not take effect.
Maybe the detection of python version on your enviroment is faulty?

May I ask you to do this test?

import sys
sys.version_info < (3,)

[pybokeh]

@dddomodossola

import sys
sys.version_info < (3,)

It outputs "False"

[pybokeh]

@dddomodossola
My sample NBA sqlite3 database can be downloaded from here There is a cross symbol † in the player name. This cross symbol is apparently the character that is causing problems.

[dddomodossola]

@pybokeh
Yeah, I've seen. Your solution to the problem works fine, but I'm trying to solve this also for version less than 3.

[dddomodossola]

@pybokeh
Now should be fixed. Can you run your example with python2.7 ?
Thank you so much for your help.

[pybokeh]

Error: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2020' in position 14: ordinal not in range(128)

full traceback:
Traceback (most recent call last):
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "test.py", line 10, in init
super(MyApp, self).init(_args)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/SocketServer.py", line 655, in init
self.handle()
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/BaseHTTPServer.py", line 340, in handle
self.handle_one_request()
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
method()
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/server.py", line 552, in do_GET
self.process_all(path)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/server.py", line 562, in process_all
self.client.root = self.main(_self.server.userdata)
File "test.py", line 62, in main
item.append(str(id(item)), str(row_item))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2020' in position 14: ordinal not in range(128)

I installed remi via pip:
pip install git+https://github.com/dddomodossola/remi.git

So I'm assuming your latest patch was implement.
Python version 2.7.10 virtual environment installed with conda.

[dddomodossola]

The problem seems to be the str() conversion you are doing. Should be:

item.append(str(id(item)), row_item)

[pybokeh]

My database query can return data that are of type int. So herein lies the problem. Your item.append() calls the html conversion code which calls s.repr() method. An int doesn't have repr(). That is why I am forced to convert the table item to str.

Below is the traceback when I don't ensure the table item is of type str:
Traceback (most recent call last):
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/SocketServer.py", line 599, in process_request_thread
self.finish_request(request, client_address)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "test.py", line 10, in init
super(MyApp, self).init(*args)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/SocketServer.py", line 655, in init
self.handle()
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/BaseHTTPServer.py", line 340, in handle
self.handle_one_request()
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/BaseHTTPServer.py", line 328, in handle_one_request
method()
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/server.py", line 552, in do_GET
self.process_all(path)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/server.py", line 571, in process_all
html = self.client.root.repr(self.client)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 196, in repr
return super(Widget,self).repr(client, include_children)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 97, in repr
innerHTML = innerHTML + s.repr(client)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 196, in repr
return super(Widget,self).repr(client, include_children)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 97, in repr
innerHTML = innerHTML + s.repr(client)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 196, in repr
return super(Widget,self).repr(client, include_children)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 97, in repr
innerHTML = innerHTML + s.repr(client)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 196, in repr
return super(Widget,self).repr(client, include_children)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 97, in repr
innerHTML = innerHTML + s.repr(client)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 196, in repr
return super(Widget,self).repr(client, include_children)
File "/home/pybokeh/miniconda3/envs/remipy2/lib/python2.7/site-packages/remi/gui.py", line 97, in repr
innerHTML = innerHTML + s.repr(client)
AttributeError: 'int' object has no attribute 'repr'

[dddomodossola]

You can do something like this:

item.append( str(id(item)), u"%s" % row_item )

[pybokeh]

Thanks that worked. I'll have to do some studying on encodings. :-)

[dddomodossola]

It's the same for me ;-)
Thank you for the information you reported in this issue, you helped me a lot improving remi.