Comments (2)
What would your output look like for various types of nested objects? What about a simple C style array of objects which themselves have a mix of fields including other objects?
Writing this archive is easy if you can completely define how you want it to look and come up with a good way to parse it. The only real difficulty in NVP based archives is adding the ability to support out of order loading.
from cereal.
the following (incomplete) code should clarify the definition:
nesting is almost the same as in JSON
I have not the ressources to write a parser atm, but it should not be too difficult.
/*! \file nvp.hpp
\brief NVP input and output archives */
/*
Copyright (c) 2013, Randolph Voorhies, Shane Grant
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of cereal nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef CEREAL_ARCHIVES_NVP_HPP_
#define CEREAL_ARCHIVES_NVP_HPP_
#include <cereal/cereal.hpp>
#include <cereal/details/util.hpp>
#include <sstream>
#include <stack>
#include <vector>
#include <string>
namespace cereal
{
// ######################################################################
//! An output archive designed to save data to name = value pairs (NVP)
/*!
\ingroup Archives */
class NVPOutputArchive : public OutputArchive<NVPOutputArchive>
{
enum class NodeType { Default, StartObject, InObject, StartArray, InArray };
public:
//! Construct, outputting to the provided stream
/*! @param stream The stream to output to. Can be a stringstream, a file stream, or
even cout!
@param precision The precision for floating point output */
NVPOutputArchive( std::ostream & stream, int precision = 20 ) :
OutputArchive<NVPOutputArchive>( this ),
itsWriteStream( stream ),
itsNextName( nullptr ),
itsDepth( 0 )
{
itsNameCounter.push( 0 );
itsNodeStack.push( NodeType::Default ); // StartObject results in an unwanted { + newline for
}
//! Destructor, flushes the stream
~NVPOutputArchive()
{
itsWriteStream.flush();
}
//! saving values: just pass to stream
template<class T>
void saveValue( T t )
{
itsWriteStream << t;
}
//! saving chars as ints - otherwise they appear as ascii characters in text file
template<>
void saveValue( unsigned char c )
{
itsWriteStream << (uint)c;
}
template<>
void saveValue( char c )
{
itsWriteStream << (int)c;
}
//! saving strings: use quotes
template<>
void saveValue( std::string s )
{
itsWriteStream << "\"" << s << "\"";
}
void doIndent()
{
for ( int i = 0; i < itsDepth; ++i )
{
itsWriteStream << " ";
}
}
void saveName( const std::string& name )
{
doIndent();
itsWriteStream << name;
itsWriteStream << " = ";
}
void StartArray()
{
itsWriteStream << "[";
}
void EndArray()
{
itsWriteStream << "]";
itsNodeStack.pop();
itsNameCounter.pop();
}
void StartObject()
{
if ( true ) // style 1
{
itsWriteStream << "\n";
doIndent();
itsWriteStream << "{\n";
++itsDepth;
}
else // style 2
{
itsWriteStream << "{\n";
++itsDepth;
}
}
void EndObject()
{
--itsDepth;
if ( itsNodeStack.top() == NodeType::InObject )
{
itsWriteStream << "\n";
}
doIndent();
itsWriteStream << "}";
itsNodeStack.pop();
itsNameCounter.pop();
}
#ifdef _MSC_VER
// Visual Studio has problems disambiguating the above for unsigned long, so we provide an explicit
// overload for long and serialize it as its size necessitates
//
// When loading we don't need to do this specialization since we catch the types with
// templates according to their size
//! 32 bit long saving
template <class T> inline
typename std::enable_if<sizeof( T ) == sizeof( std::uint32_t ), void>::type
saveLong( T lu ) { saveValue( static_cast<std::uint32_t>( lu ) ); }
//! non 32 bit long saving
template <class T> inline
typename std::enable_if<sizeof( T ) != sizeof( std::uint32_t ), void>::type
saveLong( T lu ) { saveValue( static_cast<std::uint64_t>( lu ) ); }
//! MSVC only long overload
void saveValue( unsigned long lu ) { saveLong( lu ); };
#endif
//! Save exotic arithmetic types as binary
//template<class T>
//typename std::enable_if<std::is_arithmetic<T>::value &&
// ( sizeof( T ) >= sizeof( long double ) || sizeof( T ) >= sizeof( long long ) ), void>::type
// saveValue( T const & t )
//{
// auto base64string = base64::encode( reinterpret_cast<const unsigned char *>( &t ), sizeof( T ) );
// saveValue( base64string );
// }
//! Write the name of the upcoming node and prepare object/array state
/*! Since writeName is called for every value that is output, regardless of
whether it has a name or not, it is the place where we will do a deferred
check of our node state and decide whether we are in an array or an object. */
void writeName()
{
NodeType const & nodeType = itsNodeStack.top();
// Start up either an object or an array, depending on state
if ( nodeType == NodeType::InArray )
{
itsWriteStream << ", ";
}
else if ( nodeType == NodeType::InObject )
{
itsWriteStream << "\n"; // new names on new line
}
if ( nodeType == NodeType::StartArray )
{
StartArray();
itsNodeStack.top() = NodeType::InArray;
}
else if ( nodeType == NodeType::StartObject )
{
StartObject();
itsNodeStack.top() = NodeType::InObject;
}
// Array types do not output names
if ( nodeType == NodeType::InArray )
{
return;
}
if ( itsNextName == nullptr )
{
std::string name = "value" + std::to_string( itsNameCounter.top()++ ) + "\0";
doIndent();
itsWriteStream << name;
itsWriteStream << " = ";
}
else
{
doIndent();
itsWriteStream << itsNextName;
itsWriteStream << " = ";
itsNextName = nullptr;
}
}
//! Starts a new node in the NVP output
void startNode()
{
writeName();
itsNodeStack.push( NodeType::StartObject );
itsNameCounter.push( 0 );
}
//! Designates the most recently added node as finished
void finishNode()
{
const NodeType node_type = itsNodeStack.top();
// if we ended up serializing an empty object or array, writeName
// will never have been called - so start and then immediately end
// the object/array.
//
// We'll also end any object/arrays we happen to be in
switch ( node_type )
{
case NodeType::StartArray:
StartArray();
case NodeType::InArray:
EndArray();
break;
case NodeType::StartObject:
StartObject();
case NodeType::InObject:
EndObject();
break;
}
}
//! Designates that the current node should be output as an array, not an object
void makeArray()
{
itsNodeStack.top() = NodeType::StartArray;
}
//! Sets the name for the next node created with startNode
void setNextName( const char * name )
{
itsNextName = name;
}
//! Saves some binary data, encoded as a base64 string, with an optional name
/*! This will create a new node, optionally named, and insert a value that consists of
the data encoded as a base64 string */
//void saveBinaryValue( const void * data, size_t size, const char * name = nullptr )
//{
// setNextName( name );
// writeName();
// auto base64string = base64::encode( reinterpret_cast<const unsigned char *>( data ), size );
// saveValue( base64string );
//};
private:
std::ostream& itsWriteStream; //!< write stream
char const * itsNextName; //!< The next name
std::stack<uint32_t> itsNameCounter; //!< Counter for creating unique names for unnamed nodes
std::stack<NodeType> itsNodeStack;
int itsDepth;
}; // NVPOutputArchive
// ######################################################################
//! An input archive designed to load data from JSON
/*! This archive uses RapidJSON to read in a JSON archive.
Input JSON should have been produced by the NVPOutputArchive. Data can
only be added to dynamically sized containers (marked by JSON arrays) -
the input archive will determine their size by looking at the number of child nodes.
The order of the items in the JSON archive must match what is expected in the
serialization functions.
\ingroup Archives */
class NVPInputArchive : public InputArchive<NVPInputArchive>
{
// typedef rapidjson::GenericReadStream ReadStream;
// typedef rapidjson::GenericValue<rapidjson::UTF8<>> JSONValue;
// typedef JSONValue::ConstMemberIterator MemberIterator;
// typedef JSONValue::ConstValueIterator ValueIterator;
// typedef rapidjson::Document::GenericValue GenericValue;
// //! An internal iterator that handles both array and object types
// class Iterator
// {
// public:
// Iterator() : itsType( Null ) {}
// Iterator( MemberIterator it ) :
// itsMemberIt( it ), itsType( Member )
// {
// }
// Iterator( ValueIterator it ) :
// itsValueIt( it ), itsType( Value )
// {
// }
// Iterator & operator++( )
// {
// switch ( itsType )
// {
// case Value: ++itsValueIt; break;
// case Member: ++itsMemberIt; break;
// default: throw cereal::Exception( "Invalid Iterator Type!" );
// }
// return *this;
// }
// GenericValue const & value()
// {
// switch ( itsType )
// {
// case Value: return *itsValueIt;
// case Member: return itsMemberIt->value;
// default: throw cereal::Exception( "Invalid Iterator Type!" );
// }
// }
// private:
// MemberIterator itsMemberIt;
// ValueIterator itsValueIt;
// enum Type { Value, Member, Null } itsType;
// };
public:
//! Construct, outputting to the provided stream
/*! @param stream The stream to output to. Can be a stringstream, a file stream, or
even cout! */
NVPInputArchive( std::istream & stream ) :
InputArchive<NVPInputArchive>( this ),
itsReadStream( stream )
{
//itsDocument.ParseStream<0>( itsReadStream );
//itsValueStack.push_back( itsDocument.MemberBegin() );
}
//! Starts a new node, going into its proper iterator
void startNode()
{
//if ( itsValueStack.back().value().IsArray() )
// itsValueStack.push_back( itsValueStack.back().value().Begin() );
//else
// itsValueStack.push_back( itsValueStack.back().value().MemberBegin() );
}
//! Finishes the most recently started node
void finishNode()
{
//itsValueStack.pop_back();
//++itsValueStack.back();
}
// template<class T>
// typename std::enable_if < std::is_signed<T>::value && sizeof( T ) < sizeof( int64_t ), void>::type
// loadValue( T & val )
// {
// val = itsValueStack.back().value().GetInt();
// ++itsValueStack.back();
// }
// template<class T>
// typename std::enable_if < ( std::is_unsigned<T>::value && sizeof( T ) < sizeof( uint64_t ) ) &&
// !std::is_same<bool, T>::value, void>::type
// loadValue( T & val )
// {
// val = itsValueStack.back().value().GetUint();
// ++itsValueStack.back();
// }
// void loadValue( bool & val ) { val = itsValueStack.back().value().GetBool(); ++itsValueStack.back(); }
// void loadValue( int64_t & val ) { val = itsValueStack.back().value().GetInt64(); ++itsValueStack.back(); }
// void loadValue( uint64_t & val ) { val = itsValueStack.back().value().GetUint64(); ++itsValueStack.back(); }
// void loadValue( float & val ) { val = static_cast<float>( itsValueStack.back().value().GetDouble() ); ++itsValueStack.back(); }
// void loadValue( double & val ) { val = itsValueStack.back().value().GetDouble(); ++itsValueStack.back(); }
// void loadValue( std::string & val ) { val = itsValueStack.back().value().GetString(); ++itsValueStack.back(); }
// template<class T>
// typename std::enable_if<std::is_arithmetic<T>::value &&
// ( sizeof( T ) >= sizeof( long double ) || sizeof( T ) >= sizeof( long long ) ), void>::type
// loadValue( T & val )
// {
// std::string encoded;
// loadValue( encoded );
// auto decoded = base64::decode( encoded );
// if ( sizeof( T ) != decoded.size() )
// throw Exception( "Decoded binary data size does not match specified size" );
// std::memcpy( &val, decoded.data(), decoded.size() );
// }
// //! Loads some binary data, encoded as a base64 string
// void loadBinaryValue( void * data, size_t size )
// {
// std::string encoded;
// loadValue( encoded );
// auto decoded = base64::decode( encoded );
// if ( size != decoded.size() )
// throw Exception( "Decoded binary data size does not match specified size" );
// std::memcpy( data, decoded.data(), decoded.size() );
// };
// //! Loads the size for a SizeTag
// void loadSize( size_type & size )
// {
// size = ( itsValueStack.rbegin() + 1 )->value().Size();
// }
private:
std::istream& itsReadStream; //!< Rapidjson write stream
// std::vector<Iterator> itsValueStack; //!< Stack of values
// rapidjson::Document itsDocument; //!< Rapidjson document
};
// ######################################################################
// JSONArchive prologue and epilogue functions
// ######################################################################
// ######################################################################
//! Prologue for NVPs for JSON archives
/*! NVPs do not start or finish nodes - they just set up the names */
template <class T>
void prologue( NVPOutputArchive &, NameValuePair<T> const & )
{
}
//! Prologue for NVPs for JSON archives
template <class T>
void prologue( NVPInputArchive &, NameValuePair<T> const & )
{
}
// ######################################################################
//! Epilogue for NVPs for JSON archives
/*! NVPs do not start or finish nodes - they just set up the names */
template <class T>
void epilogue( NVPOutputArchive &, NameValuePair<T> const & )
{
}
//! Epilogue for NVPs for JSON archives
/*! NVPs do not start or finish nodes - they just set up the names */
template <class T>
void epilogue( NVPInputArchive &, NameValuePair<T> const & )
{
}
// ######################################################################
//! Prologue for SizeTags for JSON archives
/*! SizeTags are strictly ignored for JSON */
template <class T>
void prologue( NVPOutputArchive & ar, SizeTag<T> const & )
{
ar.makeArray();
}
//! Prologue for SizeTags for JSON archives
template <class T>
void prologue( NVPInputArchive &, SizeTag<T> const & )
{
}
// ######################################################################
//! Epilogue for SizeTags for JSON archives
/*! SizeTags are strictly ignored for JSON */
template <class T>
void epilogue( NVPOutputArchive &, SizeTag<T> const & )
{
}
//! Epilogue for SizeTags for JSON archives
template <class T>
void epilogue( NVPInputArchive &, SizeTag<T> const & )
{
}
// ######################################################################
//! Prologue for all other types for JSON archives
/*! Starts a new node, named either automatically or by some NVP,
that may be given data by the type about to be archived */
template <class T>
typename std::enable_if<!std::is_arithmetic<T>::value && !std::is_enum<T>::value && !cereal::is_primitive<T>::value&& !std::is_pointer<T>::value, void>::type
prologue( NVPOutputArchive & ar, T const & )
{
ar.startNode();
}
//! Prologue for all other types for JSON archives
template <class T>
typename std::enable_if<!std::is_arithmetic<T>::value && !std::is_enum<T>::value && !cereal::is_primitive<T>::value, void>::type
prologue( NVPInputArchive & ar, T const & )
{
ar.startNode();
}
// ######################################################################
//! Epilogue for all other types other for JSON archives
/*! Finishes the node created in the prologue */
template <class T>
typename std::enable_if<!std::is_arithmetic<T>::value && !std::is_enum<T>::value && !cereal::is_primitive<T>::value, void>::type
epilogue( NVPOutputArchive & ar, T const & )
{
ar.finishNode();
}
//! Epilogue for all other types other for JSON archives
template <class T>
typename std::enable_if<!std::is_arithmetic<T>::value && !std::is_enum<T>::value && !cereal::is_primitive<T>::value, void>::type
epilogue( NVPInputArchive & ar, T const & )
{
ar.finishNode();
}
// ######################################################################
//! Prologue for arithmetic types for JSON archives
template <class T>
typename std::enable_if<std::is_arithmetic<T>::value /*|| std::is_enum<T>::value || cereal::is_primitive<T>::value*/, void>::type
prologue( NVPOutputArchive & ar, T const & )
{
ar.writeName();
}
//! Prologue for arithmetic types for JSON archives
template <class T>
typename std::enable_if< std::is_arithmetic<T>::value /*|| std::is_enum<T>::value || cereal::is_primitive<T>::value*/, void >::type
prologue( NVPInputArchive &, T const & )
{
}
// ######################################################################
//! Epilogue for arithmetic types for JSON archives
template <class T>
typename std::enable_if<std::is_arithmetic<T>::value, void>::type
epilogue( NVPOutputArchive &, T const & )
{
}
//! Epilogue for arithmetic types for JSON archives
template <class T>
typename std::enable_if<std::is_arithmetic<T>::value, void>::type
epilogue( NVPInputArchive &, T const & )
{
}
// ######################################################################
//! Prologue for strings for JSON archives
template<class CharT, class Traits, class Alloc> inline
void prologue( NVPOutputArchive & ar, std::basic_string<CharT, Traits, Alloc> const & )
{
ar.writeName();
}
//! Prologue for strings for JSON archives
template<class CharT, class Traits, class Alloc> inline
void prologue( NVPInputArchive &, std::basic_string<CharT, Traits, Alloc> const & )
{
}
// ######################################################################
//! Epilogue for strings for JSON archives
template<class CharT, class Traits, class Alloc> inline
void epilogue( NVPOutputArchive &, std::basic_string<CharT, Traits, Alloc> const & )
{
}
//! Epilogue for strings for JSON archives
template<class CharT, class Traits, class Alloc> inline
void epilogue( NVPInputArchive &, std::basic_string<CharT, Traits, Alloc> const & )
{
}
// ######################################################################
// Common JSONArchive serialization functions
// ######################################################################
//! Serializing NVP types to JSON
template <class T> inline
void save( NVPOutputArchive & ar, NameValuePair<T> const & t )
{
ar.setNextName( t.name );
ar( t.value );
}
template <class T> inline
void load( NVPInputArchive & ar, NameValuePair<T> & t )
{
ar( t.value );
}
//! Saving for arithmetic to JSON
template<class T> inline
typename std::enable_if<std::is_arithmetic<T>::value, void>::type
save( NVPOutputArchive & ar, T const & t )
{
ar.saveValue( t );
}
//! Loading arithmetic from JSON
template<class T> inline
typename std::enable_if<std::is_arithmetic<T>::value, void>::type
load( NVPInputArchive & ar, T & t )
{
//ar.loadValue( t );
}
//! saving string to JSON
template<class CharT, class Traits, class Alloc> inline
void save( NVPOutputArchive & ar, std::basic_string<CharT, Traits, Alloc> const & str )
{
ar.saveValue( str );
}
//! loading string from JSON
template<class CharT, class Traits, class Alloc> inline
void load( NVPInputArchive & ar, std::basic_string<CharT, Traits, Alloc> & str )
{
//ar.loadValue( str );
}
// ######################################################################
//! Saving SizeTags to JSON
template <class T> inline
void save( NVPOutputArchive &, SizeTag<T> const & )
{
}
//! Loading SizeTags from JSON
template <class T> inline
void load( NVPInputArchive & ar, SizeTag<T> & st )
{
//ar.loadSize( st.size );
}
} // namespace cereal
// register archives for polymorphic support
CEREAL_REGISTER_ARCHIVE( cereal::NVPInputArchive );
CEREAL_REGISTER_ARCHIVE( cereal::NVPOutputArchive );
#endif // CEREAL_ARCHIVES_NVP_HPP_
from cereal.
Related Issues (20)
- c++20 std::span support
- Binary Serialization of String is still a readable string HOT 3
- Embedded RapidJSON can conflict with another RapidJSON used by the client (ODR violation!)
- Compilation error with struct as smat_pointer
- How to deserialize std::vector with XMLInputArchive
- Macos Build error: cannot bind lvalue of type unsigned long long to value of unrelated type unsigned long HOT 1
- I encountered a segmentation fault when processing XML file data with version 1.3.2 of Cereal. Could this be a security vulnerability
- pkg-config file
- std::aligned_storage is deprecated in C++23 HOT 1
- GCC 13.2 build error: possible dangling reference to a temporary HOT 2
- About the use of CEREAL_CLASS_VERSION HOT 1
- types/tuple.hpp does not compile on MSVC2019
- copy constructor is implicitly deleted because 'OutputArchiveBase' has a user-declared move constructor HOT 1
- Issue with closing braces in JSON using std::stringstream HOT 1
- Any plans for a new version?
- Is there a way to register polymorphic types during program execution? HOT 3
- How do I remove excess layers HOT 1
- RapidJson causes C5054 on MSVC
- JSONOutputArchive destructor is not really noexcept(true) despite it's declared noexcept(true)
- cereal unable to export xml comments
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cereal.