weebly / cereal Goto Github PK
View Code? Open in Web Editor NEWSwift object serialization
License: BSD 3-Clause "New" or "Revised" License
Swift object serialization
License: BSD 3-Clause "New" or "Revised" License
I'm using Cereal to wire up some simple Swift struct serialization and deserialization to disk (in UserDefaults).
Before I write the objects to disk, I began testing serialization in-memory, and am getting a weird failure from Cereal:
// deviceRecordIn is a Swift struct which conforms to `CerealType`
// When:
// We serialize one in-memory.
let encodingKey = "deviceRecord"
var encoder = CerealEncoder()
do {
try encoder.encode(deviceRecordIn, forKey: encodingKey)
} catch {
XCTAssertTrue(false, "Error encoding data. \(error)")
}
let data = encoder.toData()
XCTAssertNotNil(data)
// Then:
// We should be able to de-serialize it:
do {
let decoder = try CerealDecoder(data: data)
let aRecord = try decoder.decode(encodingKey)
// And it should match our serialized record.
XCTAssertNotNil(aRecord)
guard let encodedRecord = aRecord else {
return
}
} catch {
XCTAssertTrue(false, "Error encoding data. \(error)")
}
My unit test fails at let aRecord = try decoder.decode(encodingKey)
, with:
error: -[InspireTests.BluetoothPairingTest testThatWeCanSerializeAPairingRecord] : XCTAssertTrue failed - Error encoding data. InvalidEncoding("Failed to instantiate CerealTypeIdentifier with k")
The call stack (sorry, this one is pretty verbose):
(lldb) bt
* Thread 1 (tid 0x43e9f0, queue "com.apple.main-thread"): 0x0000000115dade40 swift_willThrow
Stopped by breakpoint 27.1
#0 0x0000000115dade40 swift_willThrow
* #1 0x000000011573720e static CerealDecoder.(encodedString="k,4:uuid:s,36:16B6F74C-8336-48A0-B833-F920C367D0CF:k,9:sensorUID:i,1:0:k,14:connectionDate:T,16:477003543.503419", index=Swift.String.Index @ 0x00007fff550b24c0, $error=ErrorType @ 0x00007fff550b2e60)(String, startingAtIndex : String.CharacterView.Index) throws -> (type : CerealTypeIdentifier, indexPassedValue : String.CharacterView.Index) + 1694 at CerealDecoder.swift:1149
#2 0x0000000115738341 static CerealDecoder.(encodedString="k,4:uuid:s,36:16B6F74C-8336-48A0-B833-F920C367D0CF:k,9:sensorUID:i,1:0:k,14:connectionDate:T,16:477003543.503419", index=Swift.String.Index @ 0x00007fff550b2780, $error=ErrorType @ 0x00007fff550b2e60)(String, startingAtIndex : String.CharacterView.Index) throws -> (type : CerealTypeIdentifier, value : String, endIndex : String.CharacterView.Index) + 289 at CerealDecoder.swift:1159
#3 0x000000011572880c String.iterateEncodedValuesWithInstantationHandler(instantiationHandler=0x000000011573f260 Cereal`partial apply forwarder for static Cereal.CerealDecoder.((parseEncodedArrayString in _5CB6B52C5402593B67F608A3C7D8E4BB) (Swift.String) throws -> Swift.Array<Cereal.CerealRepresentable>).(closure #1) at CerealDecoder.swift, self="k,4:uuid:s,36:16B6F74C-8336-48A0-B833-F920C367D0CF:k,9:sensorUID:i,1:0:k,14:connectionDate:T,16:477003543.503419", $error=ErrorType @ 0x00007fff550b2e60) throws -> ()) throws -> () + 636 at CerealDecoder.swift:1313
#4 0x00000001157276f5 static CerealDecoder.(encodedString="k,4:uuid:s,36:16B6F74C-8336-48A0-B833-F920C367D0CF:k,9:sensorUID:i,1:0:k,14:connectionDate:T,16:477003543.503419", $error=ErrorType @ 0x00007fff550b2e60)(String) throws -> [CerealRepresentable] + 181 at CerealDecoder.swift:1176
#5 0x0000000115727563 CerealDecoder.decode(key="deviceRecord", self=Cereal.CerealDecoder @ 0x00007fff550b2c70, $error=ErrorType @ 0x00007fff550b2e60) throws -> [CerealRepresentable]? + 563 at CerealDecoder.swift:137
#6 0x00000001156f6f54 BluetoothPairingTest.testThatWeCanSerializeAPairingRecord(self=0x00007fa9fa5a7b70) -> () + 884 at BluetoothPairingTest.swift:138
#7 0x00000001156f7992 @objc BluetoothPairingTest.testThatWeCanSerializeAPairingRecord() -> () + 34 at BluetoothPairingTest.swift:0
#8 0x000000010b5b989c __invoking___ + 140
#9 0x000000010b5b96ee -[NSInvocation invoke] + 286
#10 0x000000010abdb5a7 __24-[XCTestCase invokeTest]_block_invoke_2 + 362
#11 0x000000010ac0fb2b -[XCTestContext performInScope:] + 190
#12 0x000000010abdb42c -[XCTestCase invokeTest] + 169
#13 0x000000010abdba56 -[XCTestCase performTest:] + 459
#14 0x000000010abd94cb -[XCTestSuite performTest:] + 396
#15 0x000000010abd94cb -[XCTestSuite performTest:] + 396
#16 0x000000010abd94cb -[XCTestSuite performTest:] + 396
#17 0x000000010abc62e4 __25-[XCTestDriver _runSuite]_block_invoke + 51
#18 0x000000010abe71f4 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 640
#19 0x000000010abc6229 -[XCTestDriver _runSuite] + 453
#20 0x000000010abc6fa5 -[XCTestDriver _checkForTestManager] + 259
#21 0x000000010ac10fb2 _XCTestMain + 628
#22 0x000000010ab4d20f ___lldb_unnamed_function3$$xctest + 362
#23 0x000000010dfaf92d start + 1
#24 0x000000010dfaf92d start + 1
(lldb)
Interestingly, my struct's extension of init(decoder cereal: CerealDecoder)
is not called as part of decoding.
Any pointers you can give to help me debug what's going wrong here?
Carthage support would be nice
Hi, I'm currently using Cereal 1.4.0 in a Swift 3 project on OS X ( I created an OS X target, which has been working fine for me so far, with the exception of this issue).
I'm not sure if my recent upgrade to Swift 3 and Cereal 1.4.0 introduced this issue, or not, because I'm sure it worked previously but not sure exactly when it stopped working.
Anyway, I'm serializing an array of Swift objects, in this case the array includes only one object.
The error I'm getting is ".Array / .Cereal / .Dictionary not expected", so of course at first I thought the problem was that I was deserializing an array (even though that should be supported). However on closer inspection, the issue is with the decoding of a property in the object which is a dictionary. parseEncodedDictionaryString is being called, which calls iterateEncodedValues, which eventually calls parseEncodedDictionaryString. This then fails decoding the key, apparently because the key's type is .cereal.
Here's all the data that's failing in parseEncodedDictionaryString:
keyType = CerealOSX.CerealTypeIdentifier.cereal
keyValue = "k,8:rawValue:s,4:more"
type = CerealOSX.CerealTypeIdentifier.string
value = "100"
Any idea why this is failing? Also what does the key type of .cereal mean and why is that unexpected? I'm getting no errors during the encoding/serialization, only in decoding. Is this a bug in Cereal, or something I'm doing wrong?
Any help is greatly appreciated, thanks!
Hey, I'd love to see this lib available in Swift 3 so I can implement it in my project.
Thanks 😄
i'm getting this error on decoding. Any idea why?
Unclear whether this is due to a bug in Cereal of not, but encoding an NSDate and then decoding it seems to result in a slightly less accurate date, which then in turn causes isEqualToDate()
to return false for the two dates. Here's a minimal test case to highlight the problem:
let date = NSDate()
let data = try! CerealEncoder.dataWithRootItem(date)
let comparison: NSDate = try! CerealDecoder.rootItemWithData(data)
XCTAssertTrue(date.isEqualToDate(comparison))
The final line is not true, while logically it should be. When logging out info about the dates, they seem correct, but there is actually a problem with the sub-second part of the dates, Cereal seems to be dropping some accuracy:
Original: 471798680.624343 After encode/decode: 471798680.62434
Hey,
i'm trying to encode a dictionary with variable value types. According to the Github Documentation one should cast the dictionary with .CER_casted() (since the value type is a protocol) before decoding/encoding. The decoding part works fine but for some reason i can't get the encoding to work.
The Dictionary is of the type:
[String: CerealType]
Its decoded like this:
guard let data:[String:CerealType] = try decoder.decode(dataKey)?.CER_casted() else { throw CodingError.MissingData } self.data = data
and encoded like this:
try encoder.encode(self.data.CER_casted(), forKey: dataKey);
which fails with:
Ambiguous Refrence to member 'encode(_:forKey:)
Any Ideas?
-- malte
Set support is coming soon!
Hi! I've got a fun, maybe simple question for you about using Cereal with an enum:
I've got a small class and an enum, which look like:
enum UploadEnum {
case Sending
case Sent
case Failed(error: NSError)
}
class DataClass {
let name: String
let upload: UploadEnum
}
I want to encode DataClass with Cereal, but… I don't know how, because of UploadEnum.
With other enums without associated types, I've encoded the enum's rawValue
, and then decoded it with the enum's .init(rawValue: <>)
initializer.
Can you think of any simple ways to encode UploadEnum
? I can think of some invasive approaches which require changing the enum, but I'm seeking alternatives.
Swift 4 introduces new Codable
protocol that generally do the same as this library.
Should we make a migration tool for those who want to use a new protocol and be able to keep backward compatibility with Cereal?
I’m currently working with Swift 3.0. Eventual goal is port apps from Linux to OS X.
I wrote an extension to serialize/deserialize dictionaries of cereal objects easily:
extension Dictionary where Key: CerealRepresentable, Value: CerealType {
func serialize(to filePath: String) throws {
var encoder = CerealEncoder()
try encoder.encode(self, forKey: "dictionary")
let data = encoder.toData()
try data.writeToFile(filePath, options: [.AtomicWrite])
}
mutating func deserialize(from filePath: String) throws {
guard let data = NSData(contentsOfFile: filePath) else { throw SerializationError.FileNotExists }
let decoder = try CerealDecoder(data: data)
guard let result: [Key: Value] = try decoder.decodeCereal("dictionary") else { throw SerializationError.IncorrectData }
self = result
}
}
And I faced with a very frustrating fact: when number of items to be serialized/deserialized counts in the thousands, Cereal consumes CPU a lot. For example, when I serialize 8k records, it takes about 5 seconds to run on my 6+ device:
When I replaced all string[index1..<index2]
with string.substringWithRange(index1..<index2)
+ prepared dictionary's capacity (very empirical estimation: assuming number of commas equals number of subitems):
private static func parseEncodedCerealDictionaryString<DecodedKeyType: protocol<Hashable, CerealRepresentable>, DecodedValueType: CerealType>(encodedString: String) throws -> [DecodedKeyType: DecodedValueType] {
let scanner = NSScanner(string: encodedString)
scanner.charactersToBeSkipped = nil
var scanResult: NSString?
var approximateCapacity = 0
while !scanner.atEnd {
scanner.scanUpToString(",", intoString: &scanResult)
approximateCapacity += 1
scanner.scanString(",", intoString: nil)
}
var decodedItems = Dictionary<DecodedKeyType, DecodedValueType>(minimumCapacity: approximateCapacity)
try encodedString.iterateEncodedValuesWithInstantationHandler { keyType, keyValue, type, value in
let decodedKey: DecodedKeyType = try CerealDecoder.instantiate(keyValue, ofType: keyType)
decodedItems[decodedKey] = try CerealDecoder.instantiateCereal(value, ofType: type) as DecodedValueType
}
return decodedItems
}
the picture changed a little bit, showing the real reason of high load:
There's too many CerealDecoder
instantiations.
Should cereal move to another implementation, maybe the one that is based on NSScanner
, with a very little string reallocations count, with a decoder reuse instead of reallocating.
Couldn't invent a way how to do that with backward compatibility with the existing format, maybe you can help me in pointing to the right way?
Right now thinking of making a transformer to NSKeyedArchiver
/from NSKeyedUnarchiver
since the items in CerealEncoder
are stored as strings.
1.2.1 does not work with Xcode 7.3 / Swift 2.2, but the master branch does. Could you release a 1.2.2 or 1.3?
I have following struct
struct ItemDetail {
var message:String?
var date:NSDate?
var location:String?
var status:ItemStatus //enum
init(message:String?, date:NSDate?, location:String?, status:ItemStatus) {
self.message = message
self.date = date
self.location = location
self.status = status
}
}
However Cereal doesn't allow me to encode optionals. I don't want to check for nil or use default values. How can I use Cereal to encode/decode above struct? Thanks.
Suppose the following:
enum Gender: Int {
case female
case male
}
extension Gender: CerealType {
// custom encode and decode
}
And encode
was called on a variable of type Gender
, then the RawRepresentable override version will be called, ignoring the custom serializers.
I know it's not a huge issue, but it could be fixed by providing an encodeCereal
function for example, to explicitly invoke the custom serializers.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.