NSString category: "CSVComponents" is broken if the string contains Unicode character about chcsvparser

[thkeen]

TEST: (
(
"TR\U1ea6N",
"species_code",
"Scientific name",
"Author name",
"Common name",
Family,
Description,
Habitat,
""Leaf size min (cm, 0 decimal digit)"",
""Leaf size max (cm, 0 decimal digit)"",
Distribution,
"Current National Conservation Status",
"Growth requirements",
"Horticultural features",
"U\U7300\U6500\U7300\U2c00\U4100\U7300\U7300\U6f00\U6300\U6900\U6100\U7400\U6500\U6400\U2000\U6600\U6100\U7500\U6e00\U6100\U2c00\U5200\U6500\U6600\U6500\U7200\U6500\U6e00\U6300\U6500\U2c00\U7300\U7000\U6500\U6300\U6900\U6500\U7300\U5f00\U6900\U6400"
)

[thkeen]

FYI: I had to save the string into a file then load from it using "arrayWithContentsOfCSVFile" then it worked fine. :)

Thanks.

[davedelong]

The issue is the use of [csv fastestEncoding] here: https://github.com/davedelong/CHCSVParser/blob/master/CHCSVParser/CHCSVParser.m#L78

That's the difference between reading a CSV string from disk and reading it from memory. Reading it from disk tries to discover the encoding and falls back to NSUTF8StringEncoding, while the -fastestEncoding method was returning NSUnicodeStringEncoding. I'm not sure what the difference between the two is, but changing the encoding for reading from a string to always be NSUTF8StringEncoding appears to fix the issue.

[thkeen]

Hmmmmm, so you mean I should use the method "initWithInputStream....", mimic the code in the other function to solve the issue for now right?

Thanks for your help!