flight-school / messagepack Goto Github PK

If I decode a Double value as a Float, it silently reinterprets the first 32 bits and gives garbage results. What behaviour is intended? Should it throw DecodingError.typeMismatch? Should It decode Double and then cast to Float?

Example:

import MessagePack

let double = Double(1)
let encoder = MessagePackEncoder()
let data = try encoder.encode(double)
let decoder = MessagePackDecoder()
let float = try decoder.decode(Float.self, from: data)
print(float) // prints 1.875

P.S.: Float to Double case gives garbage as well
P.P.S.: there are some typos in specifying some of the DecodingError.typeMismatch types in SingleValueDecodingContainer.swift

Hi!

I have a bit of inheritance in my codable objects and MessagePack crashes when encoding one of these. I could simplify the code to reproduce the crash to this:

func testMessagePack() {
        do {
            let testerB = TesterB()

            let encoder = MessagePackEncoder()
            let data = try encoder.encode(testerB)
        } catch {
            return
        }
    }
}

class TesterA: Codable {
    var fun = 1
}

class TesterB: TesterA {
    enum CodingKeys: CodingKey {
        case foo
    }
    var foo = 2

    override func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(foo, forKey: .foo)
        try super.encode(to: encoder)
    }
}

I checked the code and saw there could be only one container. What would be the best way to add support for any number of containers? I was thinking of just storing an array of them and just set container to the array.last, what do you think?

Cheers,

S.

The func decode<T>(_ type: T.Type, from data: Data) throws -> T where T : Decodable is not public and hence not available if you depend on this library from another project using SPM

Swift version 4.2.1

This code will crash

import Foundation
import MessagePack

let packed = try! MessagePackEncoder().encode(Date.distantPast)
print(packed)
let unpacked = try! MessagePackDecoder().decode(Date.self, from: packed)
print(unpacked)

If you change Date.distantPast to just Date(), it won't crash, but will decode incorrectly, to something like 1984-12-17 02:49:05 which is 34 years ago from now :)

If you comment out all custom Date codings in this package, everything works perfectly.

Swift version 5.1.2

This code will crash on decoding

struct Sock: Codable, Equatable {
    let color: String
    let size: Float
}

struct PairOfSocks: Codable, Equatable {
    let left: Sock?
    let right: Sock?

    static var example: PairOfSocks {
        PairOfSocks(left: Sock(color: "black", size: 11),
                    right: nil)
    }
}

let value = PairOfSocks.example
let encoded = try! encoder.encode(value)
let decoded = try! decoder.decode(PairOfSocks.self, from: encoded)

Error message

DecodingError.typeMismatch(Optional<Any>,
                           DecodingError.Context(codingPath: [],
                                                 debugDescription: "cannot decode nil for key: CodingKeys(stringValue: \"left\", intValue: nil)",
                                                 underlyingError: nil))

Possible solution

In _MessagePackDecoder.KeyedContainer.decodeNil(forKey:)

try checkCanDecodeValue(forKey: key)

let nestedContainer = self.nestedContainers[key.stringValue]

if let singleValueContainer = nestedContainer as? _MessagePackDecoder.SingleValueContainer {
    return singleValueContainer.decodeNil()
} else if (nestedContainer as? _MessagePackDecoder.UnkeyedContainer) == nil,
    (nestedContainer as? _MessagePackDecoder.KeyedContainer<AnyCodingKey>) == nil {
    let context = DecodingError.Context(codingPath: self.codingPath, debugDescription: "cannot decode nil for key: \(key)")
    throw DecodingError.typeMismatch(Any?.self, context)
} else {
    return false
}

In _MessagePackDecoder.UnkeyedContainer.decodeNil()

try checkCanDecodeValue()
defer { self.currentIndex += 1 }

let nestedContainer = self.nestedContainers[self.currentIndex]

if let singleValueContainer = nestedContainer as? _MessagePackDecoder.SingleValueContainer {
    return singleValueContainer.decodeNil()
} else if (nestedContainer as? _MessagePackDecoder.UnkeyedContainer) == nil,
    (nestedContainer as? _MessagePackDecoder.KeyedContainer<AnyCodingKey>) == nil {
    let context = DecodingError.Context(codingPath: self.codingPath, debugDescription: "cannot decode nil for index: \(self.currentIndex)")
    throw DecodingError.typeMismatch(Any?.self, context)
} else {
    return false
}

According to the spec

If an object can be represented in multiple possible output formats, serializers SHOULD use the format which represents the data in the smallest number of bytes.

When I encode the number 128 like this MessagePackEncoder().encode(128) it encodes it as an Int 16 (<D1 80 00>) while it should be encoded as <CC 80>

A simple model like this fails to encode and then decode successfully:

struct Link: Codable {
    let url: URL
}

This is because like Date and Data types, URLs also need to be handled as a special case by coders.

Here's how they're handled in the Swift JSONEncoder:
https://github.com/apple/swift-corelibs-foundation/blob/943fee90d095a1708da826e7641636a3cbe9642e/Sources/Foundation/JSONEncoder.swift#L445

If left to encode themselves, URLs create a base/relative URL part, whereas the JSONEncoder just outputs the absoluteString as a normal String field.

I found FixedWidthInteger bytes implementation in book is:

extension FixedWidthInteger {
    var bytes: [UInt8] {
        let capacity = MemoryLayout<Self>.size
        var mutableValue = self.bigEndian
        return withUnsafePointer(to: &mutableValue) {
            return $0.withMemoryRebound(to: UInt8.self, capacity: capacity) {
                return Array(UnsafeBufferPointer(start: $0, count: capacity))
            }
        }
    }
}

but in this repository, var mutableValue = self.

Greetings! I've imported MessagePack via SwiftPM (version 1.2.2) and now playing with this code:

struct X: Codable {
    let y: [Int]
}

let object = X(y: [Int](repeating: 0, count: 16))
let encoded = try! MessagePackEncoder().encode(object)
let decoded = try! decoder.decode(X.self, from: encoded)

The last command gives me

Fatal error: dataCorrupted(Swift.DecodingError.Context(codingPath: [_GenericIndexKey(stringValue: "Index 1", intValue: 1)], debugDescription: "Invalid format: 220", underlyingError: nil)): file /mydir/SourcePackages/checkouts/MessagePack/Sources/MessagePack/Decoder/UnkeyedDecodingContainer.swift, line 49

220 is array 16 in msgpack spec. Are such structs (with array fields that could contain 16+ elements) unsupported by the library?

Say someone encoded the number 128 as <CC 80> (which is what the spec recommends) and you try to decode this number using this library into a Swift Int you will get an error because the unsigned integers are not covered in this switch case:

Mattt, thanks for this great library!

In order to decide on my next on-disk serialization format I did some benchmarks, in particular I compared MessagePack against Foundation's PropertyListEncoder.

MessagePack creates serialized results that are about 40% smaller than the PropertyListEncoder, however it took about 300% of encoding time in contrast to PropertyListEncoder.

I wonder where this difference comes from – naively, I had expected somewhat of a linearity between size of the result and encoding time. I did not profile yet, do you have any idea whether integrating a C-based msgpack encoder could perhaps improve the runtime?