uscilab / cereal Goto Github PK

View Code? Open in Web Editor NEW

4.1K 4.1K 737.0 18.06 MB

A C++11 library for serialization

License: BSD 3-Clause "New" or "Revised" License

CMake 1.36% C++ 98.24% Shell 0.18% Batchfile 0.22%

c-plus-plus cereal serialization

cereal's People

Contributors

Stargazers

Watchers

Forkers

neomantra marcantoine-arnaud bogado nevermatch wufan1980 aniwey fwiesel mtourne factoid mattyclarkson hanumantmk samana tkokof jhol traw danfengzi andrey2805 dine909 bingo235 fanxu henrywoo pjm0616 volo-zyko xw0 chloe-zen besser82 zhongxingzhi bagobor eejv stevehickman siotour johnsonlz snowwolf1 fraser999 sebastianstadelmann wchy grigorij1981 johnlanni bdhmwz ustulation rggjan rhuangab avimosher rcisc xiaoxiongzheng cshuyue hungmingwu nilsbore honggw cppfool felixzng openmvg-thirdparty wavelet-noise eschnett emaxerrno dailypipsgxj ezhangle syxolk josephwinston yongli3 henrik-muehe rayyee ilerik l3b2w1 isegal mitch-connor jmgao gnuhub headupinclouds willir sambolu ashishbhawsar2k tempbottle colingilbert limbolily loutlier abutcher-gh jonathan-beard liwangouc guker robertomalatesta kylefleming vcatechnology heamon7 arvidsson wzugang aimotive-legacy dumganhar crycrane lyager imxset21 kersson mikeseven bigseb caomw zzkristy nextgenintelligence zmij ajneu dengcy028

cereal's Issues

deserialization of JSON is little bit different from (BInary, xml)

I'm not sure that I leave the issue on here.

Actually, JSONInputArchive is not working well in my environment.
Now I'm trying to use "cereal" for my mobile program in cocos2d-x 3.0( it supports for c++11)

XML, Binary is OK for serialize and deserealize..

But, when it comes to JSONInputArchive.. It made crash.
(I already checked, JSONOutputArchive made correct "data.json" file in my folder.
like
{
"m1": {
"x": 1,
"y": 2,
"z": 3
}
But, when the class tries to set stream to JSONInputArchive, something is wrong in json.hpp
--------------json.hpp-------------
//! Construct, outputting to the provided stream
/*! @param stream The stream to output to. Can be a stringstream, a file stream, or
even cout! */
JSONInputArchive(std::istream & stream) :
InputArchive(this),
itsReadStream(stream)
{
itsDocument.ParseStream<0>(itsReadStream);
itsValueStack.push_back(itsDocument.MemberBegin()); //<----Crash

}

My source code is below;

ex)
std::string path = FileUtils::getInstance()->getWritablePath() + "data.json";
{
std::ofstream ss(path.c_str());
cereal::JSONOutputArchive archive(ss);
MyData m1;
archive(CEREAL_NVP(m1));
}

{
std::ifstream is;
is.open(path.c_str());
cereal::JSONInputArchive iarchive(is); //<---- Crash

MyData m1;
iarchive(m1);
}

MyData is just simple Data Class in cereal document

struct MyClass
{
int x, y, z;
template
void serialize(Archive & archive)
{
archive( x, y, z ); // serialize things by passing them to the archive
}
};

Consider using hardened coding standards

I'm putting this up for discussion first before trying to make a patch. Currently cereal fails quite horribly when adding -Wshadow -Wextra -pedantic. Is compatibility with those flags something you guys care about? If so, I would likely fix a few of those errors.

Add _CEREAL_NVP for loads

Will likely need to have these NVPs for loading data for issue #13 and issue #14

Exposing references to objects created outside the cereal load process.

Concept:

During load you may have dependencies on something which is already in memory loaded from another aspect of your application. That may be something like a renderer, or a mouse, or a central texture atlas which has its own load/save process unrelated to your existing object hierarchy. You may even need to hook up some loaded/saved items to objects outside of the save/load heirarchy (as in the case with a central texture atlas, if you have textures from a loaded scene coming in, maybe you want to register them with that atlas as they get created.)

Right now you can construct objects through cereal and any owned resources will be correctly loaded during the save/load(serialize) process. The issue is that handles to in-memory resources need to be reconstituted later.

For simple cases of flat single type objects this can be trivial to hook up after load.

RenderObject myRenderer;

JSONInputArchive archive;
std::shared_ptr<Rectangle> rect;
archive(cereal::make_nvp("rect", rect));
rect->setRenderer(myRenderer); //acceptable!  Unless the rectangle needs information about the viewport on load.  Maybe we have to re-design the class a bit, but probably workable.

This becomes more difficult, however when layers are added, and polymorphic types step in. What if I have a clickable element deep inside a scene graph and it needs reference to the mouse wrangler?

RenderThing myRenderer;
MouseWrangler myMouse;

JSONInputArchive archive;
std::shared_ptr<Node> root;
archive(cereal::make_nvp("root", root));
root->setRenderer(myRenderer); //Same as above example!
//root->setMouse(myMouse); //WHOOPSE!  What if setMouse only exists in clickable nodes?  Now the user of cereal has to make a choice as to how to fix the loaded object post-load, or make myMouse globally accessible!

Ideally we would be able to supply these additional requirements which the enclosing scope (enclosing of the archive process) would already have access to. Then propogate that through the load process so objects could grab that information during their creation/loading instead of after.

RenderThing myRenderer;
MouseWrangler myMouse;
Point defaultRotateOrigin; //just adding this for a "value" example of something we want copied.

JSONInputArchive archive;

archive.store( cereal::make_nvp("mouse", &myMouse), cereal::make_nvp("renderer", &myRenderer), cereal::make_nvp("defaultRotateOrigin", defaultRotateOrigin));

std::shared_ptr<Node> root;
archive(cereal::make_nvp("root", root));
//Oh, and hey, all the items are now responsible for their own loading.  If they try to access a value that doesn't exist in the archive we can throw or something as usual.

the following is some pseudo code:

class Clickable : public Node {
Clickable(const std::string &name, RenderThing* renderer, MouseWrangler &mouse, Point defaultRotateOrigin);
...

template <class Archive>
static void load_and_allocate( Archive & ar, cereal::allocate<Clickable > & allocate )
    {
      std::string name;
      Point position;

      ar( cereal::make_nvp("name", name), cereal::make_nvp("position", position) );

      RenderThing* renderer;
      MouseWrangler* mouse;
      Point defaultRotateOrigin;
      ar.extract(cereal::make_nvp("renderer", renderer), cereal::make_nvp("mouse", mouse), cereal::make_nvp("defaultRotateOrigin", defaultRotateOrigin));
      assert(mouse != null);

      allocate( name, renderer, mouse, defaultRotateOrigin );
      allocate.object().position = position;
}
...
}

'minimality' of text archives

I am a strong advocate of minimalism.
Please give this a thought. And try to imagine what Steve Jobs would think of it.

I already mentioned that for enums and my 'primitive' types your text archives produced an overhead.

Now I am seeing even more overhead in polymorphic types

Your output is e.g.

{
    "polymorphic_id": 2147483650,
    "polymorphic_name": "myClass",
    "ptr_wrapper": {
        "valid": 1,
        "data": {
            "myData1": 0,
            "myData2": 1
        }
    }
},

where all I need (and all that is required to load back) is the following

{
        "myClass": {
            "myData1": 0,
            "myData2": 1
        }
},

the code already knows that a pointer is going to be loaded and can thus act accordingly

Remark:
It should not be the duty of the individual archive to handle this. One central handling of polymorhpic pointers for text archives is enough.

Support Unordered Loading from JSON

Users should be able to load data from JSON archives that have arbitrary data so long as the correct NVPs are present to identify the data.

VS2013 Develop Branch - Trouble with binary serialization of a std::pair

I am using develop branch with VS2013 Express. An exception is risingwhen reading the following kind of binary serialized container std::vectorstd::pair<std::vector<double, std::vector>> .

size is equal to 80 and readsize to 13. Please let me know if you need more information.

#include <utility>
#include <fstream>

#include <cereal/types/vector.hpp>
#include <cereal/types/utility.hpp>
#include <cereal/archives/portable_binary.hpp>

int main()
{
    {
    std::vector<double> vK, vZ;
    std::vector<std::pair<std::vector<double>, std::vector<double>>> vTestCases;

    for (size_t i = 0; i < 10; ++i)
    {
        vK.clear();
        vZ.clear();
        for (size_t j = 0; j < 10; ++j)
        {
            vK.push_back((double)std::rand());
            vZ.push_back((double)std::rand());
        }
        vTestCases.push_back(std::make_pair(vK, vZ));
    }

    std::ofstream vFileOut("test", std::ios::trunc);
    cereal::PortableBinaryOutputArchive oarchive(vFileOut);

    oarchive(vTestCases);
    vFileOut.close();
}
{
    std::vector<std::pair<std::vector<double>, std::vector<double>>> vTestCases;
    std::ifstream vFileIn("test");
    cereal::PortableBinaryInputArchive iarchive(vFileIn);

    iarchive(vTestCases);
    vFileIn.close();
    }
}

'Name = Value' archive

Forgive me for posting another enhancement 'issue'...

I love to use a simple 'name = value' pair file for logging as well as for simple configuration files.

(My true motivation for switching from boost to cereal was this:
I found it impossible (due to code complexity) to add such a custom archive to boost. The unavailability of a JSON archive for years tells the same story.)

The definition of nvp files is this:

for simple types (arithmetic,string) a line is added:

    name = value

for containers as array/vector/list square brackets are used

   container = [ a, b, c, d ]

for complex objects (i.e. user defined classes) an object in braces is added:

   object =
   {
      name1 = value1 

      name2 = value2
      ...
   }

objects can be nested.

Having such an archive would also imply automatic "pretty printing" of std containers which is a common need.

Having such a simple bulit-in archive (to be used e.g. for config files and logging purposes) would be a distinguishing feature of your lib.
It is - once more - telling, that boost included a simple log archive as an example in the docs, but that nobody was there to finish it.

I might post some code if I get this to work - at least for output

Serialization Function Wiki Page

https://github.com/USCiLab/cereal/wiki/Serialization-Functions

Primitive types

I am missing the equivalent of
BOOST_CLASS_IMPLEMENTATION(MyClass, primitive_type)

Example:
I have classes that already know how to convert to/from a string.
so I use

template<class Archive>                                 
void save( Archive& archive ) const                         
{                                                                           
    archive( ToString() );              
}

resulting in the JSON

"testobject": {
    "value0": "testobjectAsString"
}

but WHAT I WANT is just a single line

"testobject": "testobjectAsString"

Remark: the same thing happens for ENUMs in your implementation.
They are always saved as a 2-level object, which seems unnecessary.

Polymorphic Wiki Page

https://github.com/USCiLab/cereal/wiki/Polymorphic-Types

Use std::uint64_t instead of std::size_t

To help ensure compatibility between 32/64 bit systems, move from serializing the size of containers as the direct result of their size() calls to uint64_t so that it's always the same size.

Issue with shared_from_this after utilizing load_and_allocate

This stumped me for some time. I was looking for errors in my own code for quite some time before I created the following example. enable_shared_from_this doesn't seem to get its weak_ptr reconstituted properly on load.

#include <strstream>

#include "cereal/cereal.hpp"
#include "cereal/types/map.hpp"
#include "cereal/types/vector.hpp"
#include "cereal/types/memory.hpp"
#include "cereal/types/string.hpp"
#include "cereal/types/base_class.hpp"

#include "cereal/archives/json.hpp"
#include <cereal/types/polymorphic.hpp>

#include <memory>

struct Handle;

struct Node : public std::enable_shared_from_this<Node> {
    Node(int val):data(val){}
    virtual ~Node(){}
    std::map<std::string, std::shared_ptr<Node>> children;
    std::shared_ptr<Handle> handle;

    template <class Archive>
    void serialize(Archive & archive){
        archive(CEREAL_NVP(handle), CEREAL_NVP(children), CEREAL_NVP(data));
    }

    std::shared_ptr<Node> shared(){
        return shared_from_this();
    }

    template <class Archive>
    static void load_and_allocate(Archive & archive, cereal::allocate<Node> &allocate){
        allocate(-1);
        archive(cereal::make_nvp("data", allocate->data));
    }

    int data = 0;
};

struct DerivedNode : public Node {
    DerivedNode(int val):Node(val){}

    template <class Archive>
    static void load_and_allocate(Archive & archive, cereal::allocate<DerivedNode> &allocate){
        allocate(-1);
        archive(cereal::make_nvp("data", allocate->data));
    }
};

CEREAL_REGISTER_TYPE(Node);
CEREAL_REGISTER_TYPE(DerivedNode);

struct Definition;
struct Handle : public std::enable_shared_from_this<Handle> {
    Handle(int id):id(id){}
    int id = 0;
    std::shared_ptr<Definition> definition;

    template <class Archive>
    void serialize(Archive & archive){
        int testInt = 0;
        int *pointInt = &testInt;
        archive.extract(cereal::make_nvp("testInt", testInt), cereal::make_nvp("pointInt", pointInt));
        std::cout << testInt << ": " << *pointInt << std::endl;
        archive(CEREAL_NVP(id), CEREAL_NVP(definition));
    }

    template <class Archive>
    static void load_and_allocate(Archive & archive, cereal::allocate<Handle> &allocate){
        int theId;
        archive(cereal::make_nvp("id", theId));
        allocate(theId);
    }
};

struct Definition : public std::enable_shared_from_this<Definition> {
    virtual ~Definition(){}
    Definition(int id):id(id){}
    int id = 0;
    std::vector<std::weak_ptr<Handle>> handles;

    template <class Archive>
    void serialize(Archive & archive){
        archive(CEREAL_NVP(id), CEREAL_NVP(handles));
    }

    template <class Archive>
    static void load_and_allocate(Archive & archive, cereal::allocate<Definition> &allocate){
        int theId;
        archive(cereal::make_nvp("id", theId));
        allocate(theId);
    }
};

struct DerivedDefinition : public Definition {
    DerivedDefinition(int id):Definition(id){}

    template <class Archive>
    static void load_and_allocate(Archive & archive, cereal::allocate<DerivedDefinition> &allocate){
        int theId;
        archive(cereal::make_nvp("id", theId));
        allocate(theId);
    }
};

CEREAL_REGISTER_TYPE(Definition);
CEREAL_REGISTER_TYPE(DerivedDefinition);

void saveTest(){
    int testInt = 10;
    int *pointInt = &testInt;
    std::stringstream stream;
    {
        std::shared_ptr<Node> arm = std::make_shared<DerivedNode>(1);
        std::shared_ptr<Node> rock1 = std::make_shared<DerivedNode>(2);
        std::shared_ptr<Node> rock2 = std::make_shared<DerivedNode>(3);
        arm->children["rock1"] = rock1;
        arm->children["rock2"] = rock2;

        std::shared_ptr<Definition> definition = std::make_shared<DerivedDefinition>(1);
        rock1->handle = std::make_shared<Handle>(1);
        rock2->handle = std::make_shared<Handle>(2);
        rock1->handle->definition = definition;
        rock2->handle->definition = definition;

        cereal::JSONOutputArchive archive(stream);
        archive(cereal::make_nvp("arm", arm));
    }
    std::cout << stream.str() << std::endl;
    std::cout << std::endl;
    {
        cereal::JSONInputArchive archive(stream);
        archive.add(cereal::make_nvp("testInt", testInt), cereal::make_nvp("pointInt", pointInt));
        testInt = 30;

        std::shared_ptr<DerivedNode> arm;
        archive(cereal::make_nvp("arm", arm));

        arm->shared_from_this();
        std::cout << arm->shared()->data << std::endl;

        cereal::JSONOutputArchive outArchive(stream);
        outArchive(cereal::make_nvp("arm", arm));

        std::cout << stream.str() << std::endl;
        std::cout << std::endl;

        archive(cereal::make_nvp("arm", arm));
        std::cout << "Arm Child 0 Definition ID: " << arm->children["rock1"]->handle->definition->id << std::endl;
        std::cout << "Arm Child 1 Definition ID: " << arm->children["rock2"]->handle->definition->id << std::endl;
        arm->children["rock1"]->handle->shared_from_this();
        arm->children["rock1"]->handle->definition->shared_from_this();
    }

    std::cout << "Done" << std::endl;
}

int main(){
    saveTest();
}

Optional Fields

I was wondering if support for optional serialization fields is considered/planned.

In my case I would use this feature when loading/writing settings files from/to json (or xml), where certain entries don't have to be present. But also in the scenario of binary serialization this could lead to memory savings.

Based on the boost_compat_new branch I was able to implement this for my own maybe type and the json archives. For this I had to add another variant of the templated save / load functions of the json-archive and type specific prologue/epilogue functions.

Is there any interest in officially supporting something similar?

binary archive not portable across different endian machines

Would probably be pretty easy to add an option to the binary archive to detect or be given the machine's endianness, encode it when saving, and read it in when loading.

Documentation for virtually inherited base classes on the frontpage is wrong

The documentation for virtually inherited base classes on the frontpage (http://uscilab.github.io/cereal/inheritance.html) is wrong, compared to what base_class.hpp has to offer in its documentation.

The minimal example given on the frontpage wouldn't compile, only the latter.

I more or less stumbled upon that after much confusion.

Suspecting this is simply a matter of forgot-to-update -- an update on this is much appreciated!

Thanks/xdbr

Documentation for new features

Write documentation and update examples for unordered XML and JSON loading

Write documentation on boost compatability (transitioning from boost).

Nested circular references using load_and_allocate

I'm opening this as a separate issue as it was introduced by the most recent couple check-ins on the develop branch. You should be using Visual Studio 2013 to reproduce this (I am currently on the November Compiler Prerelease as well, but I suspect it's a problem with the standard install too.)

#include <iostream>
#include <sstream>
#include <string>
#include <map>

#include "cereal/cereal.hpp"
#include "cereal/types/map.hpp"
#include "cereal/types/vector.hpp"
#include "cereal/types/memory.hpp"
#include "cereal/types/string.hpp"
#include "cereal/types/base_class.hpp"

#include "cereal/archives/json.hpp"
#include <cereal/types/polymorphic.hpp>

class BaseClass : public std::enable_shared_from_this<BaseClass> {
public:
    virtual ~BaseClass(){}

    template <class Archive>
    void serialize(Archive & archive){
        archive(CEREAL_NVP(name), CEREAL_NVP(baseMember));
    }
protected:
    BaseClass(const std::string &a_name):
        name(a_name){
    }

    std::string name;
    int baseMember; //let this have random junk so we can see if it saves right.
};

class DerivedClass : public BaseClass {
    friend cereal::access;
public:
    static std::shared_ptr<DerivedClass> make(const std::string &a_name, int a_derivedMember){
        return std::shared_ptr<DerivedClass>(new DerivedClass(a_name, a_derivedMember));
    }

    template <class Archive>
    void serialize(Archive & archive){
        archive(CEREAL_NVP(derivedMember), cereal::make_nvp("base", cereal::base_class<BaseClass>(this)));
    }
private:
    DerivedClass(const std::string &a_name, int a_derivedMember):
        BaseClass(a_name),
        derivedMember(a_derivedMember){
    }

    template <class Archive>
    static DerivedClass * load_and_allocate(Archive &archive){
        int derivedMember;
        archive(CEREAL_NVP(derivedMember));
        DerivedClass* object = new DerivedClass("", derivedMember);
        archive(cereal::make_nvp("base", cereal::base_class<BaseClass>(object)));
        return object;
    }

    int derivedMember;
};

CEREAL_REGISTER_TYPE(DerivedClass);

void saveTest(){
    std::stringstream stream;
    {
        cereal::JSONOutputArchive archive(stream);
        auto testSave = DerivedClass::make("TestName", 4);
        archive(cereal::make_nvp("test", testSave));
    }
    std::cout << stream.str() << std::endl;
    std::shared_ptr<DerivedClass> loaded;
    {
        cereal::JSONInputArchive archive(stream);
        archive(cereal::make_nvp("test", loaded));
    }
    std::stringstream stream2;
    {
        cereal::JSONOutputArchive archive(stream2);
        archive(cereal::make_nvp("test", loaded));
    }
    std::cout << stream2.str() << std::endl;
    std::cout << "TA-DA!" << std::endl;
}

int main(){
    saveTest();
}

The error I get is related to the line: if( ar.isSharedPointerValid ) in memory.hpp:

The following is what I get from my LIVE CODE, not from the above test. I'll go ahead and get the error for the above code this evening.

1>  textures.cpp
1>C:\git\external\cereal\include\cereal/types/memory.hpp(171): error C3867: 'cereal::InputArchive<cereal::JSONInputArchive,0>::isSharedPointerValid': function call missing argument list; use '&cereal::InputArchive<cereal::JSONInputArchive,0>::isSharedPointerValid' to create a pointer to member
1>          C:\git\external\cereal\include\cereal/cereal.hpp(767) : see reference to function template instantiation 'void cereal::load<AA,MV::FileTextureDefinition>(Archive &,cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &> &)' being compiled
1>          with
1>          [
1>              AA=cereal::JSONInputArchive
1>  ,            Archive=cereal::JSONInputArchive
1>          ]
1>          C:\git\external\cereal\include\cereal/cereal.hpp(692) : see reference to function template instantiation 'cereal::JSONInputArchive &cereal::InputArchive<cereal::JSONInputArchive,0>::processImpl<T>(T &)' being compiled
1>          with
1>          [
1>              T=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
1>          ]
1>          C:\git\external\cereal\include\cereal/cereal.hpp(692) : see reference to function template instantiation 'cereal::JSONInputArchive &cereal::InputArchive<cereal::JSONInputArchive,0>::processImpl<T>(T &)' being compiled
1>          with
1>          [
1>              T=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
1>          ]
1>          C:\git\external\cereal\include\cereal/cereal.hpp(558) : see reference to function template instantiation 'void cereal::InputArchive<cereal::JSONInputArchive,0>::process<_Ty>(T &&)' being compiled
1>          with
1>          [
1>              _Ty=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
1>  ,            T=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
1>          ]
1>          C:\git\external\cereal\include\cereal/cereal.hpp(558) : see reference to function template instantiation 'void cereal::InputArchive<cereal::JSONInputArchive,0>::process<_Ty>(T &&)' being compiled
1>          with
1>          [
1>              _Ty=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
1>  ,            T=cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>
1>          ]
1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(160) : see reference to function template instantiation 'ArchiveType &cereal::InputArchive<ArchiveType,0>::operator ()<cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>>(cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &> &&)' being compiled
1>          with
1>          [
1>              ArchiveType=cereal::JSONInputArchive
1>          ]
1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(160) : see reference to function template instantiation 'ArchiveType &cereal::InputArchive<ArchiveType,0>::operator ()<cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &>>(cereal::memory_detail::PtrWrapper<std::shared_ptr<MV::FileTextureDefinition> &> &&)' being compiled
1>          with
1>          [
1>              ArchiveType=cereal::JSONInputArchive
1>          ]
1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(150) : while compiling class template member function 'cereal::detail::InputBindingCreator<Archive,T>::InputBindingCreator(void)'
1>          with
1>          [
1>              Archive=cereal::JSONInputArchive
1>  ,            T=MV::FileTextureDefinition
1>          ]
1>          C:\git\external\cereal\include\cereal/details/static_object.hpp(56) : see reference to function template instantiation 'cereal::detail::InputBindingCreator<Archive,T>::InputBindingCreator(void)' being compiled
1>          with
1>          [
1>              Archive=cereal::JSONInputArchive
1>  ,            T=MV::FileTextureDefinition
1>          ]
1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(293) : see reference to class template instantiation 'cereal::detail::InputBindingCreator<Archive,T>' being compiled
1>          with
1>          [
1>              Archive=cereal::JSONInputArchive
1>  ,            T=MV::FileTextureDefinition
1>          ]
1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(290) : while compiling class template member function 'void cereal::detail::polymorphic_serialization_support<cereal::JSONInputArchive,T>::instantiate(void)'
1>          with
1>          [
1>              T=MV::FileTextureDefinition
1>          ]
1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(307) : see reference to class template instantiation 'cereal::detail::polymorphic_serialization_support<cereal::JSONInputArchive,T>' being compiled
1>          with
1>          [
1>              T=MV::FileTextureDefinition
1>          ]
1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(306) : while compiling class template member function 'void cereal::detail::bind_to_archives<MV::FileTextureDefinition>::bind(std::false_type) const'
1>          C:\git\external\cereal\include\cereal/details/polymorphic_impl.hpp(321) : see reference to function template instantiation 'void cereal::detail::bind_to_archives<MV::FileTextureDefinition>::bind(std::false_type) const' being compiled
1>          Source\Render\textures.cpp(13) : see reference to class template instantiation 'cereal::detail::bind_to_archives<MV::FileTextureDefinition>' being compiled
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

Cereal does not compile with libc++ due to use of unique_ptr<void>

Trying to compile the library with clang and libc++ I get the following error:

In file included from unittests.cpp:27:
In file included from ./include/cereal/types/memory.hpp:33:
In file included from ./include/cereal/cereal.hpp:34:
In file included from /usr/include/c++/v1/unordered_map:318:
In file included from /usr/include/c++/v1/__hash_table:16:
/usr/include/c++/v1/memory:2492:13: error: static_assert failed "default_delete can not delete incomplete type"
            static_assert(!is_void<_Tp>::value, "default_delete can not delete incomplete type");
            ^             ~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/v1/memory:2692:13: note: in instantiation of member function 'std::__1::default_delete<void>::operator()' requested here
            __ptr_.second()(__tmp);
            ^
./include/cereal/details/polymorphic_impl.hpp:172:18: note: in instantiation of member function 'std::__1::unique_ptr<void, std::__1::default_delete<void> >::reset' requested here
            dptr.reset(ptr.release());
                 ^
./include/cereal/details/static_object.hpp:54:20: note: in instantiation of member function 'cereal::detail::InputBindingCreator<cereal::JSONInputArchive, PolyDerived>::InputBindingCreator' requested here
          static T t;
                   ^
./include/cereal/details/static_object.hpp:64:18: note: in instantiation of member function 'cereal::detail::StaticObject<cereal::detail::InputBindingCreator<cereal::JSONInputArchive, PolyDerived> >::create' requested here
          return create();
                 ^
./include/cereal/details/polymorphic_impl.hpp:248:81: note: in instantiation of member function 'cereal::detail::StaticObject<cereal::detail::InputBindingCreator<cereal::JSONInputArchive, PolyDerived> >::getInstance' requested here
          return cereal::detail::StaticObject<InputBindingCreator<Archive, T>>::getInstance();
                                                                                ^
./include/cereal/details/polymorphic_impl.hpp:286:37: note: in instantiation of member function 'cereal::detail::create_bindings<cereal::JSONInputArchive, PolyDerived>::load' requested here
        create_bindings<Archive,T>::load( std::is_base_of<detail::InputArchiveBase, Archive>() );
                                    ^
./include/cereal/details/polymorphic_impl.hpp:277:36: note: in instantiation of member function 'cereal::detail::polymorphic_serialization_support<cereal::JSONInputArchive, PolyDerived>::instantiate' requested here
      typedef instantiate_function<instantiate> unused;
                                   ^
./include/cereal/details/polymorphic_impl.hpp:312:9: note: in instantiation of member function 'cereal::detail::bind_to_archives<PolyDerived>::bind' requested here
        bind( std::is_abstract<T>() );
        ^
unittests.cpp:2734:1: note: in instantiation of member function 'cereal::detail::bind_to_archives<PolyDerived>::bind' requested here
CEREAL_REGISTER_TYPE(PolyDerived);
^
./include/cereal/types/polymorphic.hpp:65:3: note: expanded from macro 'CEREAL_REGISTER_TYPE'
  CEREAL_BIND_TO_ARCHIVES(T);
  ^
./include/cereal/details/polymorphic_impl.hpp:68:26: note: expanded from macro 'CEREAL_BIND_TO_ARCHIVES'
        >::getInstance().bind();                             \
                         ^

This appears to stem from the use of unique_ptr<void>, which as best I can tell is ill-formed should the deleter ever be required to fire, as it is by the call to reset().

Can't compile using VS 2013 RC

the 'constexpr' used in 'traits.hpp' seems problematic

(I only tried this because you somewhere stated that VS 2013 Preview would be supported)

Is there a workaround?
Any hints for a workaround on VS 2012, which I would still prefer...

Add Support for Non-Normal Floating Point in JSON/XML Parsers

When I try to use JSONInputArchive I get a crash in rapidjson/Document.h line 245 (member iterators).

MemberIterator MemberBegin()            { RAPIDJSON_ASSERT(IsObject()); return data_.o.members; }

Binary and XML both seemed to work fine on all my data.
My test looks like;

class PoDStruct
{
public:
    int32_t a;
    int64_t b;
    float e;
    float c;
    double d;

    std::vector<int> test;

    template <class Archive>
    void serialize( Archive & ar )
    {
        ar(a, b, c, d, test);
    };
};

int main(int argc, char *argv[])
{
        {
            std::ofstream os1("out.json");
            cereal::JSONOutputArchive archiveJOut(os1);
            archiveJOut(myData);
        }

        {
            std::ifstream is1("out.json");
            cereal::JSONInputArchive archiveJIn(is1);
            PoDStruct test;
            archiveJIn(test);
        }

}

rapidjson compiletime errors

Writer& Null() { Prefix(kNullType); WriteNull(); return *this; }
Writer& Bool(bool b) { Prefix(b ? kTrueType : kFalseType); WriteBool(b); return *this; }

Renaming all occurences of Null and Bool fixed it for me. (GCC 4.7.2)

enum save/load split

Being a frequent user of your lib, I encountered a small issue today:

If I do the obvious, and define separate save/load functions for enums or enum class, I get a compilation error C2338
(VS2013 RC)

enum testEnum { TIT, TAT };

template<class Archive>
void save( Archive& archive, const testEnum& m )
{
    std::string a;
    archive( a );
}

template<class Archive>
void load( Archive& archive, testEnum& m )
{
    std::string a;
    archive( a );
}

If I use a single function in exactly the same place, it compiles

template<class Archive>
void serialize( Archive& archive, testEnum& m )
{
    std::string a;
    archive( a );
}

For some non-enums I tried, the spilt seems to work.

I dont know if this is known already. I don't think I'm doing it wrong. Could you just test this, please...

Added Q:
Is it possible to put the (innermost) class name for which the error occurs directly into the error message? With VS2013 I have to switch to output and scroll to find that class every time an error occurs.
link: http://stackoverflow.com/questions/6415186/integrate-type-name-in-static-assert-output

Rem:
The website has a typo in the split on-member load function. bracket instead of comma.

Can't load std::multimap<uint32_t, std::unique_ptr<SomeClass>>

I get this:

/usr/lib64/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../include/c++/4.8.1/bits/stl_pair.h:134:35: error: 
  call to deleted constructor of 'std::unique_ptr<Creature, std::default_delete<Creature> >'
    : first(std::forward<_U1>(__x)), second(__y) { }

The way you guys implement loading from map is by inserting

  std::unique_ptr<Creature, std::default_delete<Creature> > >::pair<unsigned int &, void>' requested here
    hint = map.insert(hint, {key, value} );

which violates std::unique_ptr.

  function has been explicitly marked deleted here
  unique_ptr(const unique_ptr&) = delete;

Probably this also breaks with other containers but I only tested it for std::multimap. I suggest looking into a solution that uses emplace instead of insert.

Invalid JSON output when using ostringstream

JSONOutputArchive when used with std::ostringstream outputs different (and invalid) json compared to when used with std::cout.

Example code:

#include <iostream>
#include <string>
#include <sstream>
#include <cereal/types/string.hpp>
#include <cereal/archives/json.hpp>

struct stuff
{
    stuff () :
        a("hello"), b("world")
    {
    }

    template <typename Archive>
        void serialize ( Archive& ar )
        {
            ar( a, b );
        }

    std::string a;
    std::string b;
};

int main ()
{
    stuff collection1;

    std::ostringstream o_archive_stream;
    cereal::JSONOutputArchive o_archive( o_archive_stream );
    o_archive( collection1 );
    std::string data = o_archive_stream.str();

    std::cout << "Using ostringstream:\n";
    std::cout << data << "\n";

    cereal::JSONOutputArchive o_archive2( std::cout );

    std::cout << "Using cout:\n";
    o_archive2( collection1 );
}

Output:

Using ostringstream:
{
    "value0": {
        "value0": "hello",
        "value1": "world"
    }
Using cout:
{
    "value0": {
        "value0": "hello",
        "value1": "world"
    }
}

There is a missing '}' in first output. This is with both 0.91 and current master.

g++ 4.7.3 compilation

I'm not exactly sure when this happened, but we've broken compatibility with g++ 4.7.x on the development branch. There are two things that seem to have done this:

Using emplace within std::map and std::set, which are only implemented as of g++ 4.8.x.
Changing our type traits to be compatible with Visual Studio

The only traits that are breaking right now are ones related to testing whether member serialize functions exist when they are declared private. This is related to a compiler bug that is fixed in 4.8.x.

Deserializing a complex struct fails

We have the following struct (with serialization method):

typedef struct ModelEntry {
    uint32_t modelEntryID;

    std::vector<uint32_t> hashEntriesIDs[4];

    template<class Archive>
    void serialize(Archive & archive)
    {
        archive( modelEntryID, hashEntriesIDs );
    }

} ModelEntry;

Serialization works fine but when unserializing, it tries to read way too much elements and crashes due to memory allocation problem (I tried Binary and JSON format, all fails):

std::ifstream f("file");
cereal::BinaryInputArchive input(fmodeljournal);
f.close();

Any idea?

Compiler is clang:

Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.2
Thread model: posix

Quick Start Wiki Page

https://github.com/USCiLab/cereal/wiki/Quick-Start

Unordered key deserialization support for JSON

The CEREAL_NVP system seems to produce nicely formatted output that can be interpreted by other deserializers in other languages, but if you change the order of the keys in the disk file the InputArchive processor dies.

For things like JSON and XML to be really useful, the InputArchive should be able to look up the names, not just depend on the sequence order. Is this achievable in the library, I'd be willing to assist in its development. Right now my only other alternative is to make sure my python script outputs JSON in exactly the same key order as the JSONOutputArchive does, but since that's not a requirement of the JSON document structure, I figured it'd be better to fix cereal.

This seems to be related to issues #13, #14, and #15 but it wasn't clear if they were fully implemented. If I'm just using the library wrong, I need some guidance.

What happen if I try to serialize raw pointer?

In the documentation, it said that raw pointer serialization is not supported because it get complicated tracking raw pointer. What happen if I try serialize raw pointer? Will it be compile error or it will still serialize the pointer without tracking?

If you don't mind to answer my ignorant question about the documented rationale, why tracking and not memory management is the main issue in serializing raw pointer since cereal already implemented object tracking for smart pointer?

I would like to try cereal to replace boost serialization in my project but not supporting raw pointer will reduce many of the use cases in my project.

Support Unordered Loading from XML

Users should be able to load data from XML archives that have arbitrary data so long as the correct NVPs are present to identify the data.

Deserializing from JSON fails

Code:

#include <iostream>
#include <string>
#include <sstream>
#include <cereal/types/string.hpp>
#include <cereal/archives/json.hpp>

struct stuff
{
    stuff () :
        a("hello"), b("world")
    {
    }

    template <typename Archive>
        void serialize ( Archive& ar )
        {
            ar( a, b );
        }

    std::string a;
    std::string b;
};

int main ()
{
    stuff collection1;

    std::ostringstream o_archive_stream;
    {
        cereal::JSONOutputArchive o_archive( o_archive_stream );
        o_archive( collection1 );
    }
    std::string data = o_archive_stream.str();

    std::cout << data << std::endl;

    stuff collection2;
    std::istringstream i_archive_stream( data );

    {
        cereal::JSONInputArchive i_archive( i_archive_stream );
        i_archive( collection2 );
    }
    std::cout << collection2.a << collection2.b << "\n";
}

Compile:

$ c++ --version
Apple LLVM version 5.0 (clang-500.2.75) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin12.5.0
Thread model: posix

$ c++ --std=c++11 --stdlib=libc++ -g test.cpp

Output:

{
    "value0": {
        "value0": "hello",
        "value1": "world"
    }
}
Assertion failed: (IsObject()), function MemberBegin, file /usr/local/include/cereal/external/rapidjson/document.h, line 245.

Program received signal SIGABRT, Aborted.
0x00007fff89303d46 in __kill ()

Backtrace:

(gdb) bt
#0  0x00007fff89303d46 in __kill ()
#1  0x00007fff84dd9f83 in abort ()
#2  0x00007fff84ddacb9 in __assert_rtn ()
#3  0x0000000100013c4c in rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >::MemberBegin (this=0x7fff5fbff340) at document.h:245
#4  0x0000000100012d0d in cereal::JSONInputArchive::JSONInputArchive (this=0x7fff5fbff2a0, stream=@0x7fff5fbff398) at json.hpp:289
#5  0x000000010000262d in cereal::JSONInputArchive::JSONInputArchive (this=0x7fff5fbff2a0, stream=@0x7fff5fbff398) at json.hpp:290
#6  0x00000001000018dc in main () at test.cpp:41

Unordered loading (XML and JSON) not working when types have a wrapper (e.g. ptr_wrapper)

I've resolved quite a few of the issues I was having, but I think I may have discovered an actual bug in this case. See the following pastebin for my textures.h header which has most of the relevant code.

http://pastebin.com/emW9HBSR

If you search for !ERROR! in the above pastebin you can see the location of the issue. Basically in my load_and_allocate method when I try to read from the archive it is looking directly inside "ptr_wrapper" at the "id" and "data" fields instead of inside the "ptr_wrapper"."data" field which contains the actual class values.

I know that it's looking at "id" and "data" because I modified "search" in json.hpp to look like this:

          //! Adjust our position such that we are at the node with the given name
          /*! @throws Exception if no such named node exists */
          inline void search( const char * searchName )//, GenericValue const & parent )
          {
            size_t index = 0;
            std::cout << "_____" << std::endl;
            for( auto it = itsMemberItBegin; it != itsMemberItEnd; ++it, ++index )
              if( std::strcmp( searchName, it->name.GetString() ) == 0 )
              {
                itsIndex = index;
                return;
              } else{
                std::cout << "!" << searchName << " == " << it->name.GetString() << std::endl;
              }

            throw Exception("JSON Parsing failed - provided NVP not found");
          }

This is the method that throws, and it throws just after printing:

!repeat == id
!repeat == data

Which means it is searching for "repeat" and compares against "id" and "data".

I'm trying to run this test function:

    void saveTest(){
        std::stringstream stream;
        {
            cereal::JSONOutputArchive archive(stream);
            auto testSave = MV::FileTextureDefinition::make("Assets/Images/dogfox.png");
            auto handleSave = testSave->makeHandle(MV::Point<int>(10, 10), MV::Size<int>(42, 42));
            archive(cereal::make_nvp("test", testSave));
        }
        std::cout << stream.str() << std::endl;
        //Up until here works!  Looks like the data is all there as expected.
        {
            cereal::JSONInputArchive archive(stream);
            std::shared_ptr<MV::FileTextureDefinition> loaded;
            archive(cereal::make_nvp("test", loaded));
            mainScene->get("arm")->setTexture(loaded->makeHandle());
            mainScene->get("arm")->setColor(MV::Color(1.0, 1.0, 1.0, 1.0));
        }
    }

Here's what my std::cout << stream.str() << std::endl; line spits out:

{
    "test": {
        "id": 1073741824,
        "ptr_wrapper": {
            "id": 2147483649,
            "data": {
                "repeat": false,
                "base": {
                    "name": "Assets/Images/dogfox.png",
                    "size": {
                        "width": 512,
                        "height": 512,
                        "depth": 0
                    },
                    "handles": [
                        {
                            "locked_ptr": {
                                "id": 1073741824,
                                "ptr_wrapper": {
                                    "id": 2147483650,
                                    "data": {
                                        "handleSize": {
                                            "width": 42,
                                            "height": 42,
                                            "depth": 0
                                        },
                                        "handlePosition": {
                                            "x": 10,
                                            "y": 10,
                                            "z": 0
                                        },
                                        "handlePercentSize": {
                                            "width": 0.08203125,
                                            "height": 0.08203125,
                                            "depth": 0
                                        },
                                        "handlePercentPosition": {
                                            "x": 0.01953125,
                                            "y": 0.01953125,
                                            "z": 0
                                        },
                                        "handlePercentTopLeft": {
                                            "x": 0.01953125,
                                            "y": 0.01953125,
                                            "z": 0
                                        },
                                        "handlePercentBottomRight": {
                                            "x": 0.1015625,
                                            "y": 0.1015625,
                                            "z": 0
                                        },
                                        "flipX": false,
                                        "flipY": false,
                                        "textureDefinition": {
                                            "polymorphic_id": 2147483649,
                                            "polymorphic_name": "MV::FileTextureDefinition",
                                            "ptr_wrapper": {
                                                "id": 1
                                            }
                                        }
                                    }
                                }
                            }
                        }
                    ]
                }
            }
        }
    }

Having difficulty with load_and_allocate and json archives searching through "id" and "data" instead of through the "data" members.

http://pastebin.com/emW9HBSR

I know that it's looking at "id" and "data" because I modified "search" in json.hpp to look like this:

          //! Adjust our position such that we are at the node with the given name
          /*! @throws Exception if no such named node exists */
          inline void search( const char * searchName )//, GenericValue const & parent )
          {
            size_t index = 0;
            std::cout << "_____" << std::endl;
            for( auto it = itsMemberItBegin; it != itsMemberItEnd; ++it, ++index )
              if( std::strcmp( searchName, it->name.GetString() ) == 0 )
              {
                itsIndex = index;
                return;
              } else{
                std::cout << "!" << searchName << " == " << it->name.GetString() << std::endl;
              }

            throw Exception("JSON Parsing failed - provided NVP not found");
          }

This is the method that throws, and it throws just after printing:

!repeat == id
!repeat == data

Which means it is searching for "repeat" and compares against "id" and "data".

I'm trying to run this test function:

    void saveTest(){
        std::stringstream stream;
        {
            cereal::JSONOutputArchive archive(stream);
            auto testSave = MV::FileTextureDefinition::make("Assets/Images/dogfox.png");
            auto handleSave = testSave->makeHandle(MV::Point<int>(10, 10), MV::Size<int>(42, 42));
            archive(cereal::make_nvp("test", testSave));
        }
        std::cout << stream.str() << std::endl;
        //Up until here works!  Looks like the data is all there as expected.
        {
            cereal::JSONInputArchive archive(stream);
            std::shared_ptr<MV::FileTextureDefinition> loaded;
            archive(cereal::make_nvp("test", loaded));
            mainScene->get("arm")->setTexture(loaded->makeHandle());
            mainScene->get("arm")->setColor(MV::Color(1.0, 1.0, 1.0, 1.0));
        }
    }

Here's what my std::cout << stream.str() << std::endl; line spits out:

{
    "test": {
        "id": 1073741824,
        "ptr_wrapper": {
            "id": 2147483649,
            "data": {
                "repeat": false,
                "base": {
                    "name": "Assets/Images/dogfox.png",
                    "size": {
                        "width": 512,
                        "height": 512,
                        "depth": 0
                    },
                    "handles": [
                        {
                            "locked_ptr": {
                                "id": 1073741824,
                                "ptr_wrapper": {
                                    "id": 2147483650,
                                    "data": {
                                        "handleSize": {
                                            "width": 42,
                                            "height": 42,
                                            "depth": 0
                                        },
                                        "handlePosition": {
                                            "x": 10,
                                            "y": 10,
                                            "z": 0
                                        },
                                        "handlePercentSize": {
                                            "width": 0.08203125,
                                            "height": 0.08203125,
                                            "depth": 0
                                        },
                                        "handlePercentPosition": {
                                            "x": 0.01953125,
                                            "y": 0.01953125,
                                            "z": 0
                                        },
                                        "handlePercentTopLeft": {
                                            "x": 0.01953125,
                                            "y": 0.01953125,
                                            "z": 0
                                        },
                                        "handlePercentBottomRight": {
                                            "x": 0.1015625,
                                            "y": 0.1015625,
                                            "z": 0
                                        },
                                        "flipX": false,
                                        "flipY": false,
                                        "textureDefinition": {
                                            "polymorphic_id": 2147483649,
                                            "polymorphic_name": "MV::FileTextureDefinition",
                                            "ptr_wrapper": {
                                                "id": 1
                                            }
                                        }
                                    }
                                }
                            }
                        }
                    ]
                }
            }
        }
    }

C style arrays serialized to JSON as objects instead of ARRAYS

Visual C++ 2013 support

It might be possible without a ton of work to support VSC++ 2012 w/ the november CTP (variadic templates), which would also mean support for 2013 RTM.

Main things to look at here are ever place we use constexpr, the name demangling, and the static instantiation code.

C style arrays not working in NVPs

Trying to wrap a C style array in an NVP results in a compilation error.

Cereal breaks with multiple definitions if you include a header with a registered type in different TUs

Cereal breaks with multiple definitions if you include a header with a registered type in different TUs.
The problem is CEREAL_BIND_TO_ARCHIVES which leads to multiple definitions in the object files.
A testcase can be seen here: https://gist.github.com/sithhell/5984469
It also contains a patch and a possible solution (uncomment the lines in b.cpp and c.cpp after you applied the patch).

Test portable_binary archives

Need to do testing on 32/64 bit and big/little endian machines.

Error With Nested Circular shared_ptr and weak_ptr References

I'm having issues with circular references. I'm not sure if this is another bug or if there's something I'm doing wrong. It seems like creating a nested pointer_wrapper to a node higher up the chain fails and simply returns 1. I'm getting an exception throwing on load because of this:

throw Exception("Error while trying to deserialize a smart pointer. Could not find id " + std::to_string(id));

Also, another oddity is that shared_ptr objects no matter if they are polymorphic or not seem to need to need CEREAL_REGISTER_TYPE which means I need to give them a virtual method as well. Trying to remove the CEREAL_REGISTER_TYPE and virtual destructor for ChildWithPointerToParent fails at compile time.

#include <iostream>
#include <sstream>
#include <string>
#include <map>

#include "cereal/cereal.hpp"
#include "cereal/types/map.hpp"
#include "cereal/types/vector.hpp"
#include "cereal/types/memory.hpp"
#include "cereal/types/string.hpp"
#include "cereal/types/base_class.hpp"

#include "cereal/archives/json.hpp"
#include <cereal/types/polymorphic.hpp>

class ChildWithPointerToParent;

class BaseClass : public std::enable_shared_from_this<BaseClass> {
public:
    virtual ~BaseClass(){}

    template <class Archive>
    void serialize(Archive & archive){
        archive(CEREAL_NVP(name), CEREAL_NVP(baseMember), CEREAL_NVP(child));
    }

    void addChild(std::shared_ptr<ChildWithPointerToParent> a_child){
        child = a_child;
    }
protected:
    BaseClass(const std::string &a_name):
        name(a_name){
    }

    std::weak_ptr<ChildWithPointerToParent> child;
    std::string name;
    int baseMember; //let this have random junk so we can see if it saves right.
};

class DerivedClass : public BaseClass {
    friend cereal::access;
public:
    static std::shared_ptr<DerivedClass> make(const std::string &a_name, int a_derivedMember){
        return std::shared_ptr<DerivedClass>(new DerivedClass(a_name, a_derivedMember));
    }

    template <class Archive>
    void serialize(Archive & archive){
        archive(CEREAL_NVP(derivedMember), cereal::make_nvp("base", cereal::base_class<BaseClass>(this)));
    }
private:
    DerivedClass(const std::string &a_name, int a_derivedMember):
        BaseClass(a_name),
        derivedMember(a_derivedMember){
    }

    template <class Archive>
    static DerivedClass * load_and_allocate(Archive &archive){
        return new DerivedClass("", 0); //values loaded in serialize (using work-around in memory.hpp)
    }

    int derivedMember;
};

class ChildWithPointerToParent {
    friend cereal::access;
public:
    virtual ~ChildWithPointerToParent(){} //Why do I have to do this?  I get an error if I don't.
    static std::shared_ptr<ChildWithPointerToParent> make(std::shared_ptr<BaseClass> parent){
        return std::shared_ptr<ChildWithPointerToParent>(new ChildWithPointerToParent(parent));
    }

    template <class Archive>
    void serialize(Archive & archive){
        archive(CEREAL_NVP(parent));
    }

private:
    ChildWithPointerToParent(std::shared_ptr<BaseClass> a_parent):
        parent(a_parent){
    }

    template <class Archive>
    static ChildWithPointerToParent * load_and_allocate(Archive &archive){
        return new ChildWithPointerToParent(nullptr); //values loaded in serialize (using work-around in memory.hpp)
    }
    std::shared_ptr<BaseClass> parent;
};

CEREAL_REGISTER_TYPE(DerivedClass);
CEREAL_REGISTER_TYPE(ChildWithPointerToParent); //Why do I have to do this?  I get an error if I don't.

void saveTest(){
    std::stringstream stream;
    {
        cereal::JSONOutputArchive archive(stream);
        std::shared_ptr<BaseClass> testSave = DerivedClass::make("TestName", 4);
        std::shared_ptr<ChildWithPointerToParent> child = ChildWithPointerToParent::make(testSave);
        testSave->addChild(child);
        archive(cereal::make_nvp("test", testSave));
    }
    std::cout << stream.str() << std::endl;
    std::shared_ptr<BaseClass> loaded;
    {
        cereal::JSONInputArchive archive(stream);
        archive(cereal::make_nvp("test", loaded));
    }
    std::stringstream stream2;
    {
        cereal::JSONOutputArchive archive(stream2);
        archive(cereal::make_nvp("test", loaded));
    }
    std::cout << stream2.str() << std::endl;
    std::cout << "TA-DA!" << std::endl;
}

int main(){
    saveTest
}

EXAMPLE OF REAL WORLD USE:

I have a class of type TextureDefinition. A TextureDefinition is always created via shared_ptr. TextureDefinitions are typically stored in a map keyed by their "name" so that multiple identical TextureDefinitions don't exist, but optionally a TextureDefinition might be owned by a specific rendering object like a text character which manages its own dynamic texture. It's also possible that I only care about the TextureDefinition when loading a scene and won't want to have it owned by a central node at all, but just let it be owned by its TextureHandles (more on this later). Basically I don't have one draconic TextureManager class that deals with their lifespan.

I have another class of type TextureHandle. A TextureHandle is created by a TextureDefinition always of type shared_ptr. When the first TextureHandle is created the TextureDefinition associated with it loads into memory. When the all TextureHandles are destroyed the TextureDefinition unloads, but if something still owns the TextureDefinition it remains possible to reload it by creating new handles.

TextureDefinition maintains a list of weak_ptr's to TextureHandles (because they do not manage a TextureHandle's lifespan, something like Shape::Rectangle would have a shared_ptr).

TextureHandle maintains a shared_ptr because it would be an error to have the TextureDefinition explode while there are any active TextureHandles.

In this way, we can plot the object composition like this:

----> is weak_ptr
====> is shared_ptr
TextureDefinition1|---->TextureHandle1===>TextureDefinition1
                  |---->TextureHandle2===>TextureDefinition1
                  |---->TextureHandle3===>TextureDefinition1

The objects owning these could be a different shape for each TextureHandle and a scene manager owning the TextureDefinition so it can spawn new handles. If we don't care about spawning new handles (let's say we've loaded the whole scene already) that scene manager owned handle is optional, but the objects owning TextureHandles must still keep the TextureDefinition alive.

Intel Compiler 14 - inheritance - multi libs

Hello,

I try to use cereal on Intel Compiler 14.
Theorically this compiler respect major feature of C++11:
http://software.intel.com/en-us/articles/c0x-features-supported-by-intel-c-compiler
But I can't use the official release (cxxabi.h <= is not a standard include).
So I use vs2013 version, that work better but I have some issue with derived classes.
Minimal code:
class Namable
{
public:
Namable();
Namable( Core::String const& sName );
virtual ~Namable();

void            SetName( Core::String const& sName );
Core::String    Name() const;

private:
Core::String m_sName;

friend class cereal::access;
template < class Archive >
inline
void serialize( Archive& oArchive )
{
    oArchive & ::cereal::make_nvp( "Name", m_sName );
}

};

That is compiled as lib file on ObjectSystem.lib (but that is the *.h file so not on the object file. And on real program (link with ObjectSystem.lib) the main.cpp (create an exe file):

{
ObjectsSystem::Namable oObj( _T( "DefaultName" ) );
OFileStream oFile( "./File.xml" );
cereal::XMLOutputArchive archive( oFile );
archive & ::cereal::make_nvp( "Namable", m_oObj );
}

This code work correctly and create a xml file of my object. I can Serialize and Unserialize it.

But when I try to do it for child class:
class Object : public Namable
{
public:
Object();
Object( Core::String const& sName );
virtual ~Object();

PlatformWord    ID();
void            AddReference();
void            RemoveReference();
emvBool     IsDeleted();

protected:
void SetIsDeleted( emvBool bIsDeleted );

PlatformWord    m_uID;
u32         m_uRefCount;
emvBool     m_bDeleted;

template < class Archive >
inline
void serialize( Archive& oArchive )
{
    oArchive & cereal::virtual_base_class< Namable >( this );
    oArchive & ::cereal::make_nvp( "ID", m_uID );
}

};

This is Object.h. And I have Object.cpp this file include Object.h (so contain all reference of inline file). This file is compiled on ObjectSystem.lib too.

On my main.cpp (for blabla.exe):
{
ObjectsSystem::Object oObj( "blabla" );
Core::OStringStream oss;
OFileStream oFile( "./File.xml" );
cereal::XMLOutputArchive archive( oFile );
archive & ::cereal::make_nvp( "Namable", oObj );
}
{
ObjectsSystem::Object oObj;
std::cout << oObj.Name() << std::endl;
Core::OStringStream oss;
IFileStream oFile( "./File.xml" );
cereal::XMLInputArchive archive( oFile );
archive >> oObj;
std::cout << oObj.Name() << std::endl;
}

But when I compile this one I have an error:
ObjectsSystem.lib(Object.obj) : error LNK2005: "public: static class std::unordered_map<unsigned __int64,unsigned int,struct std::hash,struct std::equal_to,class std::allocator<struct std::pair<unsigned __int64 const ,unsigned int> > > cereal::detail::Versions::mapping" (?mapping@Versions@detail@cereal@@2v?$unordered_map@_KIU?$hash@_K@std@@U?$equal_to@_K@2@V?$allocator@U?$pair@$$CB_KI@std@@@2@@std@@A) already defined in main.obj

I try to build with /MT or /MD.
What is the work around for this case?

Thanks,

Regards

chkone

“Use of undeclared identifier 'free' ” in util.hpp

File: /include/cereal/details/util.hpp

Everything compiles with #include which is missing "out the box".

danilabagroff$ clang --version
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix

stringstream and fstream seem to behave a bit differently (stringstream not working).

I am trying to serialize my objects to a string stream and getting errors with cereal on deserialization. After some research it turns out that these two streams were of different links despite serializing the same objects.

After further review I modified the following code in BinaryOutputArchive. (Keep in mind this code probably isn't error checking the most appropriate way.)

FROM:
size_t const writtenSize = itsStream.rdbuf()->sputn( reinterpret_cast<const char*>( data ), size );

if(writtenSize != size)
    throw Exception("Failed to write " + std::to_string(size) + " bytes to output stream! Wrote " + std::to_string(writtenSize));


TO:
if(itsStream.write(reinterpret_cast<const char*>( data ), size).bad()) {
    throw Exception("Failed to write " + std::to_string(size) + " bytes to output stream! Wrote ");
}

This seemed to fix the issue though and now both streams are behaving correctly. Does anyone have any idea why this would be the case?

What do I need to known on front-end(e.g. JavaScript) about remote(back-end) classes?

After a closer look I guess that before send something to the "cereal app" I have to have:
...
"polymorphic_id": 2147483649,
"polymorphic_name": "SomeClassName",
"ptr_wrapper": {"id": 2147483649,
...

Anything else?
Are polymorphic_id and ptr_wrapper constant only during the application run(due to auto polyId = itsCurrentPolymorphicTypeId++;)?

is_polymorpic

please consider the following code

struct A {
    int m;
};

struct A1 : A {};

struct B {
    virtual void foo();
};

struct B1 : B {};

CEREAL_REGISTER_TYPE( A1 )
CEREAL_REGISTER_TYPE( B1 )

int main( int argc, char* argv[] )
{ 
    std::cout << std::boolalpha;
    std::cout<< std::is_polymorphic<A>::value << '\n';
    std::cout<< std::is_polymorphic<A1>::value << '\n';
    std::cout<< std::is_polymorphic<B>::value << '\n';
    std::cout<< std::is_polymorphic<B1>::value << '\n';
}

registering A1 will cause a compiler error. why?
because, as running main shows, A1 is not considered 'polymorphic'

Solutions:

maybe there is a better trait to find out
maybe dont show the error
maybe you have to use CEREAL_REGISTER_DERIVED_TYPE( A, A1 )
then check if A1 is really derived from A - using std::is_base_of <A,A1>.
if not, its a compile time error

Full API compatibilty with Boost.Serialization

As far as I can see, the API of cereal only looks like Boost.Serialization.
We would really like to use cereal in our project but we would need it to be a full drop in replacement.

Things we currently miss:

versioning
operator& for archives
operator<< for output archives
operator>> for input archives

'cereal' name already used by another serialisation library

Did you guys know that there is already another project (that has been around since 2009) using the same name? https://github.com/GaloisInc/cereal (docs here: http://hackage.haskell.org/package/cereal).

When I first saw a mention of your project today, I thought it was referring to the older (and well known in the Haskell community at least) cereal, and I'm sure other people are likely to be confused by two different serialisation libraries having the same name.

Is this library going to be maintained?

I noticed the last commit is a month ago. Any plans to maintain this library?

free is undefined under cereal/details/util.hpp

Take the following code:

// for doing the actual serialization
#include <cereal/archives/xml.hpp>
#include <fstream>

class Stuff
{
public:
    Stuff() = default;

    void fillData()
    {
        data = 3;
    }

private:
    double data;

    friend class cereal::access;

    template <class Archive>
    void serialize( Archive & ar )
    {
        ar( CEREAL_NVP(data) );
    }
};

int main()
{
    std::fstream file("file.txt");
    cereal::XMLOutputArchive output(file); // stream to cout

    Stuff myStuff;
    myStuff.fillData();

    output( cereal::make_nvp("best data ever", myStuff) );
}

Fails compilation due to:

/usr/local/include/cereal/details/util.hpp:53:7: Use of undeclared identifier 'free'

Modify all uses of binary archives on fstream objects to use std::ios::binary

See the discussion at issue #50.

All binary archives operating on std::fstream derived objects need to open them using the std::ios::binary flag.

Also apply this to all documentation, relates #22.

uscilab / cereal Goto Github PK

cereal's People

Contributors

Stargazers

Watchers

Forkers

cereal's Issues

}

Recommend Projects

Recommend Topics

Recommend Org

Jobs