Processing fails on loose quotes with alternate sep_char about text-csv_xs

[Tux]

adding auto_diag => 1 to the attributes:

# CSV_XS ERROR: 2023 - EIQ - QUO character not allowed @ rec 2 pos 71 field 18
not ok 1 - processed expected number of lines
#   Failed test 'processed expected number of lines'
#   at sandbox/issue3.pl line 22.
#          got: '1'
#     expected: '3'
1..1
# Looks like you failed 1 test of 1.

Text::CSV (hence Text::CSV_PP) is supposed to follow Text::CSV_XS, but the author has stopped following closely. I don't know why.

The problem seems to be

M|"""The ", K
         ^

Where that " is interpreted as the closing quote.
Investigating …

[leejo]

Text::CSV (hence Text::CSV_PP) is supposed to follow Text::CSV_XS, but the author has stopped following closely. I don't know why.

Ah, unfortunately makamaka's modules are falling into disrepair. Whether or not makamaka is still involved with perl, or simply doesn't have time to maintain the modules i don't know. I have offered to take over maintenance of the modules but to no avail. I will bug makamaka again.

[Tux]

On second thought: the error is correct

|"""The ", K, |
 123    4

Quote 1 is the quotation: the introduction to a quoted field
Quote 2 is an escape to Quote 3
Quote 3 is escaped by Quote 2
Quote 4 is a loose quote, allowed by allow_loose_quotes (it is not a loose escape, as the , does not need to be escaped as it has no special meaning)

Then follows some text and a |. As the pipe is the separator and there is no finishing quote before the |, this field is not terminated and hence causes an error.

Does that explain enough?

2|||||||195658|"""The Cottage"" 54"|"""The "|K|||||307652|R, M|"""The ", K, |EA MATCH
⇓
2|||||||195658|"""The Cottage"" 54"|"""The "|K|||||307652|R, M|"""The ", K, "|EA MATCH

and to make that valid, it should be allow_loose_escapes => 1 instead of allow_loose_quotes => 1.

[leejo]

Does that explain enough?

It does. We know the data we have is crap, so the bug seems to be Text::CSV_PP being a little more tolerant of crap data, and not with Text::CSV_XS.

Thanks for the info.

[Tux]

If you are to take over Text::CSV (or become co-maint), I am more than happy to help out with questions like these.
Note that the current temptation in getting Text::CSV_PP up-to-date with Text::CSV_XS is the relative new functionality of the csv () function, which integrated rather heavily into the XS code.

[leejo]

OK. I've bugged makamaka again. Let's see what the response is. I guess we can close this. Thanks!