Unescape does not handle the two byte sequence \n about jsonparser

[daboyuka]

That's very odd, as there are several tests that verify that. Can you share the exact code/JSON that triggers this?
On Jun 14, 2016 00:13, "Juho Mäkinen" [email protected] wrote:

A JSON text containing the two characters \n (bytes 0x5C and 0x6E) is not
translated to a new line (byte 0x0A).

This is explained for example in
http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf
in page 4: "There are two-character escape sequence representations of some
characters." with the following list: ", , /, \b, \f, \n, \r and \t

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#49, or mute the thread
https://github.com/notifications/unsubscribe/ADvi_vbebLGcZLVDnzbvsEFCGaCRVAfWks5qLipzgaJpZM4I08jG
.

[garo]

Of course. I also realises that I didn't explain what's actually happening: It seems that GetString converts two byte sequence \n into just a single byte n instead of a single byte new-line

func TestENDL(t *testing.T) {
  s, _ := jsonparser.GetString([]byte("{\"multiline\":\"line 1\\nline 2\"}"), "multiline")
  assert.Equal(t, string(s), "line 1\nline2") // Not equal: "line 1nline 2" != "line 1\nline2"
}

That's the test case which I used to pinpoint the problem. I also added (what I think is a correct test case) test case {in: "\\n", out: "\n", canAlloc: true},into the escape_test.go into the unescapeTest{} structure (as in this diff):

vagrant@precise64:/Development/jsonparser$ git diff
diff --git a/escape_test.go b/escape_test.go
index db18108..0fb5d22 100644
--- a/escape_test.go
+++ b/escape_test.go
@@ -125,6 +125,7 @@ var unescapeTests = []unescapeTest{
        {in: `abcde\uD800`, isErr: true},
        {in: `ab\uD800de`, isErr: true},
        {in: `\uD800abcde`, isErr: true},
+        {in: "\\n", out: "\n", canAlloc: true},
 }

which fails in the following way:

=== RUN   TestUnescape
--- FAIL: TestUnescape (0.00s)
    escape_test.go:183: Unescape(`\n`, bufsize=0) returned unescaped mismatch: expected `
        ` ([10], len 1), obtained `n` ([110], len 1)

What do you think? I didn't test any other items in the list than \n

[garo]

Here's another code snippet with the built-int encoding/json package which will encode a field with a new line character into a literal two-byte string \n

type MultiLineJSON struct {
    Foobar string
}

func TestENDL2(t *testing.T) {
    m := MultiLineJSON{"line 1\nline 2"}
    b, _ := json.Marshal(m)

    // Will print MultiLineJSON: '{"Foobar":"line 1\nline 2"}'
    fmt.Printf("MultiLineJSON: '%s'\n", b) 
}

[daboyuka]

There's no question that \n should escape to a newline.

Uhm, wow. I thought we had tests for this, but apparently they don't quite test this, and the Unescape function does indeed have the error you identified. I will submit a PR very soon to fix this. Thanks for finding this!

[buger]

Thank you both :)