Transistor Image Tutorial - UnicodeEncodeError from CorpusParser about fonduer

[lukehsiao]

Which version of PostgreSQL are you using?

Can you verify that your postgres encoding is set to UTF8? Check by runnning:

$ psql stg_temp_max_figure -c "SHOW SERVER_ENCODING"

Also, can we see what encoding your database is using? List those by running:

$ psql -l

I suspect the issue is that your database is using an ascii encoding rather than unicode, as it should be. If that hypothesis is correct, you should try to rerun the tutorial after explicitly making sure the database is using unicode by dropping the database and making a new one:

$ createdb -E UTF8 stg_temp_max_figure
$ psql -l

[lukehsiao]

Thanks for the follow-up @Allen8838! I don't know offhand how to change default encoding, but I do suspect it's based on setting the templates to be UTF8 rather than ASCII.

Also note that Fonduer expects Postgres 9.6 or above, so there may be other issues you run into with using postgres 9.5.13, so I'd recommend upgrading if you can.

[lukehsiao]

Hi @Allen8838, are you using OSX or Linux? Also, is this the latest tutorials using Fonduer 0.2.3?

[Allen8838]

Hi @lukehsiao, I am using Bash on Ubuntu on Windows. Yes, this is the latest tutorial using Fonduer 0.2.3.

[lukehsiao]

I'm not able to reproduce this on my Ubuntu machine. Can you post the full error log (preferably copy-pasted as text, rather than being a screenshot)?

[Allen8838]

Please see below. It's one error log, broken into several snips.

[Allen8838]

Thanks for your help. I am using version 9.5.13.

Yea, you're right about the problem. The postgres encoding is set to SQL_ASCII.

I tried the line above in creating a database but still saw that the encoding was SQL_ASCII. In case it may be helpful for others, I ended using this line instead.

! createdb -E UTF8 -T template0 stg_temp_max_figure

Which got the Parser to work.

Would you offhand know how I can change the default encoding to UTF8? I did some digging and saw that I can make the change through initdb but was unable to find this file.

Thanks again.

[Allen8838]

Thank you.