Encoding map of string to interface about go

[ugorji]

Did you read any of the docs? Which ones?

I need to know which of these cases it is, as each one will require a different response from me

• Docs not clear (means I have to make changes to the docs)
• Docs not found (means I have to figure out why you couldn't find it after searching)
• Docs not read (means I have to do nothing except point you to the docs).

[achille-roussel]

Thanks for taking the time to answer.

So I had read the README and the online doc at http://godoc.org/github.com/ugorji/go/codec (the first part and enough of the API doc to get some encoding/decoding code working), did a web search for something like "golang messagepack string" but didn't find anything relevant.

This morning I came across this RawToString flag in the MsgpackHandle type, I tried it out and it seems to give the expected behavior, but I'm not sure I understand why, the comment states "RawToString controls how raw bytes are decoded into a nil interface{}".

By "raw bytes" do you mean that the encoder has produced a Binary family element (BIN8/16/32) when it encoded the string? If so then that's not what I was intuitively expecting, and I didn't see any comments about this behavior in the doc I've read.

[ugorji]

Ah. K.

The messagepack spec was updated in 2013 to include a separate string vs binary type. However, most people are not aware of this spec update, and different encoders treat the raw type as either string or binary.

By default, we maintain compatibility with the old spec and encode both []byte and string as RAW, and decode RAW as []byte.

go-codec then gives users the option (via MsgpackHandle) to configure what to decode RAW as when decoding into a nil interface (ie interface{} with no type).

In your case, the option is MsgpackHandle.RawToString as you found.

The docs for these would have been at:

• API documentation for MsgpackHandle (as you found)
• Primer (a brand new article I wrote, which may help): http://ugorji.net/blog/go-codec-primer#messagepack

For where the term 'raw' originated, it is in the msgpack spec:
https://github.com/msgpack/msgpack/blob/master/spec.md#user-content-type-system

P.S. old spec is at https://github.com/msgpack/msgpack/blob/master/spec-old.md

[ugorji]

P.S. To be fully compatible with the new spec, you should set the following options:

WriteExt=true

This way, we encode []byte using Bin(8|16|32) and string using Str(8|16|32), and decode as same.

Without this flag, we are compatible with the legacy spec, and encode both []byte and string as Str(8|16|32), and use the value of RawToString to decide how to decode Str(8|16|32).

Note: In legacy spec, only Raw existed. In new spec, Raw is renamed to Str, and a new Bin was introduced. The main goal of the updated spec was to resolve the ambiguity.

Hope this helps.

I will update the Primer section on MessagePack to make this clear, and also may introduce a helper function for creating a MessagePack handle which is configured for this like NewMsgpackHandle(legacyMode bool). I don't like this because it limits specs, so I may leave it as it is and just update the primer. Let me think some more, and I am open to suggestions.

[ugorji]

FYI: I have updated http://ugorji.net/blog/go-codec-primer#messagepack to clarify this.

[achille-roussel]

Thanks for the update :)