I am not sure if this is actually a problem in python-unicodecsv or if I am unreasonably expecting it to work with what I am feeding it.
I have a CSV file (exported by MS Excel on Windows). It happens to have some weirdly-encoded text in it on one line.
rows = unicodecsv.reader(csv_file)
for row in rows:
...
Environment:
Request Method: POST
Request URL: http://v029.medcn.uwcm.ac.uk:8003/upload/
Django Version: 1.4.10
Python Version: 2.7.2
Installed Applications:
('arkestra_utilities',
'cms',
'menus',
'cms.plugins.text',
'cms.plugins.snippet',
'sekizai',
'contacts_and_people',
'vacancies_and_studentships',
'news_and_events',
'links',
'arkestra_utilities.widgets.combobox',
'arkestra_image_plugin',
'video',
'housekeeping',
'publications',
'symplectic',
'arkestra_clinical_studies',
'polymorphic',
'semanticeditor',
'mptt',
'easy_thumbnails',
'typogrify',
'filer',
'widgetry',
'south',
'form_designer',
'form_designer.contrib.cms_plugins.form_designer_form',
'treeadmin',
'inspector',
'django_easyfilters',
'pagination',
'debug_toolbar',
'inspector',
'chained_selectbox',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin',
'django.contrib.admindocs',
'django.contrib.humanize',
'django.contrib.staticfiles',
'django.contrib.redirects',
'django.contrib.markup')
Installed Middleware:
(u'debug_toolbar.middleware.DebugToolbarMiddleware',
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'cms.middleware.page.CurrentPageMiddleware',
'cms.middleware.user.CurrentUserMiddleware',
'cms.middleware.toolbar.ToolbarMiddleware',
'django.contrib.redirects.middleware.RedirectFallbackMiddleware',
'pagination.middleware.PaginationMiddleware')
Traceback:
File "/home/daniele/dev-14-05-21/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
111. response = callback(request, *callback_args, **callback_kwargs)
File "/home/daniele/dev-14-05-21/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
20. return view_func(request, *args, **kwargs)
File "/home/daniele/dev-14-05-21/src/arkestra-publications/publications/views.py" in upload
81. for row in rows:
File "/home/daniele/dev-14-05-21/local/lib/python2.7/site-packages/unicodecsv/__init__.py" in next
112. unicode_(value, encoding, encoding_errors)) for value in row]
Exception Type: UnicodeDecodeError at /upload/
Exception Value: 'utf8' codec can't decode byte 0x91 in position 52: invalid start byte