[serialization] deriving from xml_iarchive
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code: typedef boost::archive::xml_iarchive IutputArchive; I changed it to something like this: class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar ) Foo GetFoo() const { return m_bar; } private: Foo m_bar; }; The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
I think the best way to handle this is by making an archive helper. If you look into the implemenation of ..._oarchive, you'll see that it looks something like the following: class xml_oarchive : public naked_xml_oarchive, shared_ptr_helper {... This is to implement special facility for shared_ptr which would otherwise not be serializable. (note: this is the only case where the archive classes address issues related to the implemenation of a particular type). You could do something similar for your own "special" type. Look in the documentaion under "case studies" for a more complete explanation. Robert Ramey Kenny Riddile wrote:
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code:
typedef boost::archive::xml_iarchive IutputArchive;
I changed it to something like this:
class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar )
Foo GetFoo() const { return m_bar; }
private: Foo m_bar; };
The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
I've almost got it working, but now I'm getting some linker errors you might be able to help with. Here's my InputArchive class now: http://pastebin.com/m6d9ce587 GetRepository() and InitializeRepository() are defined in the cpp to avoid cyclical includes elsewhere. InputArchive is part of one of our static libs but we're currently linking against boost serialization dynamically. Here are my linker errors: http://pastebin.com/m72460420 Robert Ramey wrote:
I think the best way to handle this is by making an archive helper.
If you look into the implemenation of ..._oarchive, you'll see that it looks something like the following:
class xml_oarchive : public naked_xml_oarchive, shared_ptr_helper {...
This is to implement special facility for shared_ptr which would otherwise not be serializable. (note: this is the only case where the archive classes address issues related to the implemenation of a particular type).
You could do something similar for your own "special" type.
Look in the documentaion under "case studies" for a more complete explanation.
Robert Ramey
Kenny Riddile wrote:
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code:
typedef boost::archive::xml_iarchive IutputArchive;
I changed it to something like this:
class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar )
Foo GetFoo() const { return m_bar; }
private: Foo m_bar; };
The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
Same linker errors using a helper class too: http://pastebin.com/mf1c4c7 Robert Ramey wrote:
I think the best way to handle this is by making an archive helper.
If you look into the implemenation of ..._oarchive, you'll see that it looks something like the following:
class xml_oarchive : public naked_xml_oarchive, shared_ptr_helper {...
This is to implement special facility for shared_ptr which would otherwise not be serializable. (note: this is the only case where the archive classes address issues related to the implemenation of a particular type).
You could do something similar for your own "special" type.
Look in the documentaion under "case studies" for a more complete explanation.
Robert Ramey
Kenny Riddile wrote:
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code:
typedef boost::archive::xml_iarchive IutputArchive;
I changed it to something like this:
class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar )
Foo GetFoo() const { return m_bar; }
private: Foo m_bar; };
The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
Just found the blurb in the documentation about maybe having to include the corresponding .ipp, so I included xml_iarchive_impl.ipp in my Archive.hpp header and now I get tons of compilation errors like this: 1>C:\Documents and Settings\Administrator\My Documents\Development\development\third_party\boost\1_38_0\boost/archive/impl/xml_iarchive_impl.ipp(83) : error C2491: 'boost::archive::xml_iarchive_impl<Archive>::load' : definition of dllimport function not allowed Robert Ramey wrote:
I think the best way to handle this is by making an archive helper.
If you look into the implemenation of ..._oarchive, you'll see that it looks something like the following:
class xml_oarchive : public naked_xml_oarchive, shared_ptr_helper {...
This is to implement special facility for shared_ptr which would otherwise not be serializable. (note: this is the only case where the archive classes address issues related to the implemenation of a particular type).
You could do something similar for your own "special" type.
Look in the documentaion under "case studies" for a more complete explanation.
Robert Ramey
Kenny Riddile wrote:
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code:
typedef boost::archive::xml_iarchive IutputArchive;
I changed it to something like this:
class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar )
Foo GetFoo() const { return m_bar; }
private: Foo m_bar; };
The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
I'm still stuck on this. Here is my current code: http://pastebin.com/m24bcbe39 and here are the errors: http://pastebin.com/m6394ed72 Also, I think there's a small typo or bug around line 53 of: http://www.boost.org/doc/libs/1_38_0/libs/serialization/example/portable_bin... It doesn't assign the value of boost::archive::archive_exception::what() to msg, unless that line is cruft and the text "programmer error" is desired in the default case... Robert Ramey wrote:
I think the best way to handle this is by making an archive helper.
If you look into the implemenation of ..._oarchive, you'll see that it looks something like the following:
class xml_oarchive : public naked_xml_oarchive, shared_ptr_helper {...
This is to implement special facility for shared_ptr which would otherwise not be serializable. (note: this is the only case where the archive classes address issues related to the implemenation of a particular type).
You could do something similar for your own "special" type.
Look in the documentaion under "case studies" for a more complete explanation.
Robert Ramey
Kenny Riddile wrote:
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code:
typedef boost::archive::xml_iarchive IutputArchive;
I changed it to something like this:
class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar )
Foo GetFoo() const { return m_bar; }
private: Foo m_bar; };
The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
Both me and another dev have read all the relevant documentation/examples and still can't make this work. Would it be possible to get a simple example...maybe an input archive that behaves exactly like xml_iarchive and lets you pass an additional int to the constructor and retrieve it later via something like myArchive.GetStoredInt() ? Robert Ramey wrote:
I think the best way to handle this is by making an archive helper.
If you look into the implemenation of ..._oarchive, you'll see that it looks something like the following:
class xml_oarchive : public naked_xml_oarchive, shared_ptr_helper {...
This is to implement special facility for shared_ptr which would otherwise not be serializable. (note: this is the only case where the archive classes address issues related to the implemenation of a particular type).
You could do something similar for your own "special" type.
Look in the documentaion under "case studies" for a more complete explanation.
Robert Ramey
Kenny Riddile wrote:
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code:
typedef boost::archive::xml_iarchive IutputArchive;
I changed it to something like this:
class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar )
Foo GetFoo() const { return m_bar; }
private: Foo m_bar; };
The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
OK- here goes: a) make your own class my_helper and put it into a header file b) start with boost/archive/xml_iarchive.hpp c) where it says #include <boost/archive/shared_ptr_helper.hpp> add another line "#include "my_helper" d) where it says public detail::shared_ptr_helper add ", my_helper e) recompile xml_iarchive.cpp good luck, Robert Ramey Kenny Riddile wrote:
Both me and another dev have read all the relevant documentation/examples and still can't make this work. Would it be possible to get a simple example...maybe an input archive that behaves exactly like xml_iarchive and lets you pass an additional int to the constructor and retrieve it later via something like myArchive.GetStoredInt() ?
Robert Ramey wrote:
I think the best way to handle this is by making an archive helper.
If you look into the implemenation of ..._oarchive, you'll see that it looks something like the following:
class xml_oarchive : public naked_xml_oarchive, shared_ptr_helper {... This is to implement special facility for shared_ptr which would otherwise not be serializable. (note: this is the only case where the archive classes address issues related to the implemenation of a particular type).
You could do something similar for your own "special" type.
Look in the documentaion under "case studies" for a more complete explanation.
Robert Ramey
Kenny Riddile wrote:
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code:
typedef boost::archive::xml_iarchive IutputArchive;
I changed it to something like this:
class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar )
Foo GetFoo() const { return m_bar; }
private: Foo m_bar; };
The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
That's similar to what I've tried, but I am not allowed to directly modify boost headers or source files... Robert Ramey wrote:
OK- here goes:
a) make your own class my_helper and put it into a header file b) start with boost/archive/xml_iarchive.hpp c) where it says #include <boost/archive/shared_ptr_helper.hpp> add another line "#include "my_helper" d) where it says public detail::shared_ptr_helper add ", my_helper e) recompile xml_iarchive.cpp
good luck,
Robert Ramey
Kenny Riddile wrote:
Both me and another dev have read all the relevant documentation/examples and still can't make this work. Would it be possible to get a simple example...maybe an input archive that behaves exactly like xml_iarchive and lets you pass an additional int to the constructor and retrieve it later via something like myArchive.GetStoredInt() ?
Robert Ramey wrote:
I think the best way to handle this is by making an archive helper.
If you look into the implemenation of ..._oarchive, you'll see that it looks something like the following:
class xml_oarchive : public naked_xml_oarchive, shared_ptr_helper {... This is to implement special facility for shared_ptr which would otherwise not be serializable. (note: this is the only case where the archive classes address issues related to the implemenation of a particular type).
You could do something similar for your own "special" type.
Look in the documentaion under "case studies" for a more complete explanation.
Robert Ramey
Kenny Riddile wrote:
I have a situation where I want an additional piece of data to be owned by the archive type we're using. Previously, I had this typedef that's used throughout our code:
typedef boost::archive::xml_iarchive IutputArchive;
I changed it to something like this:
class InputArchive : public boost::archive::xml_iarchive { public: InputArchive( std::istream& is, Foo bar ) : boost::archive::xml_iarchive( is ) , m_bar( bar )
Foo GetFoo() const { return m_bar; }
private: Foo m_bar; };
The problem I'm seeing is that whenever a serializable class's serialize() method is invoked via an InputArchive object, the template parameter to serialize() is instantiated with boost::archive::xml_iarchive, not InputArchive, which I need. Is it possible to get the behavior I want through public derivation like this, or is there a better way?
Hi, two or three weeks ago I reported a problem concerning very slow access to thread specific storage pointers on Mac OS in multithreaded environments. When using eight threads performance was reduced by about 80%. I have to say that I use more than a million calls to the thread specific storage per second, so you won't notice that problem with only a few calls. In the meantime I have not found a clean solution, but I found the error, so the maintainer(s) or anybody more smart than me can think about a clean solution. The problem is caused in pthread/once.cpp in the method get_once_per_thread_epoch() boost::uintmax_t& get_once_per_thread_epoch() { BOOST_VERIFY(! pthread_once(&epoch_tss_key_flag,create_epoch_tss_key)); void* data=pthread_getspecific(epoch_tss_key); if(!data) { data=malloc(sizeof(boost::uintmax_t)); BOOST_VERIFY(!pthread_setspecific(epoch_tss_key,data)); *static_cast<boost::uintmax_t*>(data)=UINTMAX_C(~0); } return *static_cast<boost::uintmax_t*>(data); } On Mac OS X the first BOOST_VERIFY causes a fully executed call to pthread_once each time, which in turn uses mutexes to lock something. This is however not the case on Windows and Linux, where the performance is as expected. My "solution" to this problem was to simply comment the line out. As far as I understand the usage of BOOST_VERIFY it is only an assertion and not required to run the code properly. This then gaves me identical performance on all three platforms. I also tried to used different compilers, as I was told Apple gcc 4.0.1 had a problem with statics... but results were the same with the Intel compiler. As far as I understand the Boost license, I am allowed to patch boost and distribute the compiled dynamic link library with my own software. I further understand, that I do not need to also distribute the patched source code. Is that correct? Best regards, Oliver
AMDG Oliver Abert wrote:
two or three weeks ago I reported a problem concerning very slow access to thread specific storage pointers on Mac OS in multithreaded environments. When using eight threads performance was reduced by about 80%. I have to say that I use more than a million calls to the thread specific storage per second, so you won't notice that problem with only a few calls.
In the meantime I have not found a clean solution, but I found the error, so the maintainer(s) or anybody more smart than me can think about a clean solution. The problem is caused in pthread/once.cpp in the method get_once_per_thread_epoch()
boost::uintmax_t& get_once_per_thread_epoch() {
BOOST_VERIFY(!pthread_once(&epoch_tss_key_flag,create_epoch_tss_key)); void* data=pthread_getspecific(epoch_tss_key); if(!data) { data=malloc(sizeof(boost::uintmax_t)); BOOST_VERIFY(!pthread_setspecific(epoch_tss_key,data)); *static_cast<boost::uintmax_t*>(data)=UINTMAX_C(~0); } return *static_cast<boost::uintmax_t*>(data); }
On Mac OS X the first BOOST_VERIFY causes a fully executed call to pthread_once each time, which in turn uses mutexes to lock something. This is however not the case on Windows and Linux, where the performance is as expected. My "solution" to this problem was to simply comment the line out. As far as I understand the usage of BOOST_VERIFY it is only an assertion and not required to run the code properly. This then gaves me identical performance on all three platforms. I also tried to used different compilers, as I was told Apple gcc 4.0.1 had a problem with statics... but results were the same with the Intel compiler.
The pthread_once call is necessary for the code to function correctly. BOOST_VERIFY differs from BOOST_ASSERT in that it always evaluates its argument.
As far as I understand the Boost license, I am allowed to patch boost and distribute the compiled dynamic link library with my own software. I further understand, that I do not need to also distribute the patched source code. Is that correct?
Yes. In Christ, Steven Watanabe
On 20.04.2009, at 17:34, Steven Watanabe wrote:
AMDG
Oliver Abert wrote:
two or three weeks ago I reported a problem concerning very slow access to thread specific storage pointers on Mac OS in multithreaded environments. When using eight threads performance was reduced by about 80%. I have to say that I use more than a million calls to the thread specific storage per second, so you won't notice that problem with only a few calls.
In the meantime I have not found a clean solution, but I found the error, so the maintainer(s) or anybody more smart than me can think about a clean solution. The problem is caused in pthread/once.cpp in the method get_once_per_thread_epoch()
boost::uintmax_t& get_once_per_thread_epoch() { BOOST_VERIFY(! pthread_once(&epoch_tss_key_flag,create_epoch_tss_key)); void* data=pthread_getspecific(epoch_tss_key); if(!data) { data=malloc(sizeof(boost::uintmax_t)); BOOST_VERIFY(! pthread_setspecific(epoch_tss_key,data)); *static_cast<boost::uintmax_t*>(data)=UINTMAX_C(~0); } return *static_cast<boost::uintmax_t*>(data); }
On Mac OS X the first BOOST_VERIFY causes a fully executed call to pthread_once each time, which in turn uses mutexes to lock something. This is however not the case on Windows and Linux, where the performance is as expected. My "solution" to this problem was to simply comment the line out. As far as I understand the usage of BOOST_VERIFY it is only an assertion and not required to run the code properly. This then gaves me identical performance on all three platforms. I also tried to used different compilers, as I was told Apple gcc 4.0.1 had a problem with statics... but results were the same with the Intel compiler.
The pthread_once call is necessary for the code to function correctly. BOOST_VERIFY differs from BOOST_ASSERT in that it always evaluates its argument.
okay, I understand - However for whatever reasons my code runs just fine without it. And very stable as well. Then the question is, why does pthread_once(&epoch_tss_key_flag,create_epoch_tss_key) takes so much time while it does not on windows and linux. It seems that it uses locks, since one thread has native performance, two threads are only 1.6 faster, while eight threads are less than half as fast as a single thread.
As far as I understand the Boost license, I am allowed to patch boost and distribute the compiled dynamic link library with my own software. I further understand, that I do not need to also distribute the patched source code. Is that correct?
Yes.
Thanks for the information. Best regards, Oliver --------------- Dipl.-Inform. Oliver Abert Email: abert@uni-koblenz.de Institut für Computervisualistik Fon : +49 261 287-2770 Universität Koblenz Fax : +49 261 287-2735 Postfach 20 16 02 Raum : B213, Gebäude B 56070 Koblenz
participants (4)
-
Kenny Riddile -
Oliver Abert -
Robert Ramey -
Steven Watanabe