GithubHelp home page GithubHelp logo

Comments (1)

jdunck avatar jdunck commented on August 29, 2024

It may not be clear from the docs, but the input to a unicodecsv reader is expected to be bytes (str in python2), not unicode, so you should be using BytesIO rather than StringIO.

Testing with BytesIO rather than StringIO, I do see the "new-line character seen in unquoted field" error. I think this is a bug in the underlying csv module - https://docs.python.org/2/library/csv.html#csv.Dialect.lineterminator "The reader is hard-coded to recognise either '\r' or '\n' as end-of-line, and ignores lineterminator. This behavior may change in the future."

Using \r (as with your original data), I can reproduce this with a normal file, not io.BytesIO or io.StringIO -- and of course unicodecsv does generally work with files.

When I change your data to use \n rather than \r, then the code works:

from io import StringIO, BytesIO
from unicodecsv import DictReader, Dialect, QUOTE_MINIMAL

data = (
    'first_name,last_name,email\n'
    'Elmer,Fudd,[email protected]\n'
    'Jo\xc3\xa3o Ant\xc3\xb4nio,Ara\xc3\xbajo,[email protected]\n'
)

unicode_data = StringIO(unicode(data, 'utf-8-sig'), newline=None)
str_data = BytesIO(data)

class CustomDialect(Dialect):
    delimiter = ','
    doublequote = True
    escapechar = '\\'
    lineterminator = '\r'
    quotechar = '"'
    quoting = QUOTE_MINIMAL
    skipinitialspace = True

rows = DictReader(str_data, dialect=CustomDialect)

Unfortunately I don't see a way to fix this from within unicodecsv.

from python-unicodecsv.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.