Boost serialization (bug?) issue with MSVC, with vector<shared_ptr> in several libraries
We use boost serialization in ours projects, both on linux (gcc) and Windows. We encounter a problem only on Windows (for "every" versions of Windows (7 and 10) and of MSVC: Visual 2010 and 2015, ie. MSVC++ 10.0 _MSC_VER == 1600 or MSVC++ 14.0 _MSC_VER == 1900). *Here is the problem summarized*, but I need to give more details after: When I serialize a class - which have base class - and have 2 vector of shared_ptr of ClassB, with the same contents, and B inherits from the same base class the serialization works, but the deserialization doesn't ("input stream error"). The following image is here : https://i.stack.imgur.com/osoiF.png (if it's not displayed) [image: enter image description here] <https://i.stack.imgur.com/osoiF.png> Yes, it's very strange: - if I remove the base class, it works. - if I inline the serialization code (empty function) of the base class, it works. - if the 2 vectors don't have the same content, it works !! - if I register the ClassBase in the serialize method of ClassTest, it works. I.e.: // in ClassTest:template <class Archive>void serialize(Archive & ar, const unsigned int version) { ar.template register_type<ClassBase>(); ...} *Some details* *When it works* on linux, and when it works on Windows with ClassBase type registration in the archive, we have the same xml output. It's worth noting that: - boost_serialization is version 17 on Windows and 9 on linux (I didn't tried to update boost on linux yet). - On BaseClass xml tag, the tracking_level="1", so we have object_id. *And when it doesn't work* (once again: the serialization of ClassTest is OK, but cannot deserialized "input stream error"): - The main difference is in the serialization of ClassTest: the ClassBase part has no tracking_level. - However it's not the only cause, because if the vectors are differents (same 2 first items, and a third item in one vector), tracking_level of ClassBase in ClassTest part is "0". Last thing, if it can give you some clues: while we debugged our application, we noticed that the deserialization framework tries to deserialize a class into another. It makes me think the serialization framework is confused about the classe types identifications. Could it be a matter of dll linking? Of global static shared storage ? I've posted a visual studio solution here <https://1drv.ms/u/s!Aoy4ax0S3ZS5gkxv1UIvRUd32oVO?e=ACVi4c>. Thanks a lot for your help. And if any clarification is needed, I'm available of course !
On 11/21/19 8:51 AM, Nicolas Guillot via Boost wrote:
Thanks a lot for your help. And if any clarification is needed, I'm available of course !
Please do the following: a) double check closed issues on git on the serialization library. I recently fixed a bug which might be related. It might be helpful to look at this and see if it matches your case. b) Open up a github issue with all the information you've included in this post. c) I'll take a look at it. Robert Ramey
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Thanks a lot for your help. And if any clarification is needed, I'm available of course !
Please do the following:
a) double check closed issues on git on the serialization library. I recently fixed a bug which might be related. It might be helpful to look at this and see if it matches your case.
b) Open up a github issue with all the information you've included in this post.
c) I'll take a look at it.
Robert Ramey
Hi Robert and thanks for your answer. I didn't know boost serialization project was on git hub. I only looked for a similar bug fix at boost website. And no release note since boost 1.58, wahoo! no bug ! Please can you advice me a little bit: I guess the issue related to my problem is the issue 154 ( https://github.com/boostorg/serialization/issues/154). What do you advise: take the code from git, or only the bug fix? Is the code source from git compatible with last version of boost (1.71.0)? If you advice to take only the commit that fix this issue, which is it? Now it's 0:00 in my part of the world, but tomorrow for sure I'll do the test and keep you inform. Thanks for your help, and congratulation for this library. On Thu, Nov 21, 2019 at 5:51 PM Nicolas Guillot <nicolas.guillot@gmail.com> wrote:
We use boost serialization in ours projects, both on linux (gcc) and Windows.
We encounter a problem only on Windows (for "every" versions of Windows (7 and 10) and of MSVC: Visual 2010 and 2015, ie. MSVC++ 10.0 _MSC_VER == 1600 or MSVC++ 14.0 _MSC_VER == 1900).
*Here is the problem summarized*, but I need to give more details after: When I serialize a class
- which have base class - and have 2 vector of shared_ptr of ClassB, with the same contents, and B inherits from the same base class
the serialization works, but the deserialization doesn't ("input stream error").
The following image is here : https://i.stack.imgur.com/osoiF.png (if it's not displayed)
[image: enter image description here] <https://i.stack.imgur.com/osoiF.png>
Yes, it's very strange:
- if I remove the base class, it works. - if I inline the serialization code (empty function) of the base class, it works. - if the 2 vectors don't have the same content, it works !! -
if I register the ClassBase in the serialize method of ClassTest, it works. I.e.:
// in ClassTest:template <class Archive>void serialize(Archive & ar, const unsigned int version) { ar.template register_type<ClassBase>(); ...}
*Some details*
*When it works* on linux, and when it works on Windows with ClassBase type registration in the archive, we have the same xml output. It's worth noting that:
- boost_serialization is version 17 on Windows and 9 on linux (I didn't tried to update boost on linux yet). - On BaseClass xml tag, the tracking_level="1", so we have object_id.
*And when it doesn't work* (once again: the serialization of ClassTest is OK, but cannot deserialized "input stream error"):
- The main difference is in the serialization of ClassTest: the ClassBase part has no tracking_level. - However it's not the only cause, because if the vectors are differents (same 2 first items, and a third item in one vector), tracking_level of ClassBase in ClassTest part is "0".
Last thing, if it can give you some clues: while we debugged our application, we noticed that the deserialization framework tries to deserialize a class into another. It makes me think the serialization framework is confused about the classe types identifications. Could it be a matter of dll linking? Of global static shared storage ?
I've posted a visual studio solution here <https://1drv.ms/u/s!Aoy4ax0S3ZS5gkxv1UIvRUd32oVO?e=ACVi4c>.
Thanks a lot for your help. And if any clarification is needed, I'm available of course !
On 11/21/19 3:06 PM, Nicolas Guillot via Boost wrote:
Thanks a lot for your help. And if any clarification is needed, I'm available of course !
Please do the following:
a) double check closed issues on git on the serialization library. I recently fixed a bug which might be related. It might be helpful to look at this and see if it matches your case.
b) Open up a github issue with all the information you've included in this post.
c) I'll take a look at it.
Robert Ramey
Hi Robert and thanks for your answer.
I didn't know boost serialization project was on git hub. I only looked for a similar bug fix at boost website. And no release note since boost 1.58,
I fix bugs as they are found. I make other small enhancements all the time. I'm remiss in not updating the release notes.
wahoo! no bug !
Please can you advice me a little bit: I guess the issue related to my problem is the issue 154 ( https://github.com/boostorg/serialization/issues/154).
That's a good possibility
What do you advise: take the code from git, or only the bug fix? Is the code source from git compatible with last version of boost (1.71.0)?
I advise downloading the whole library - latest version and re-building it from scratch. For reasons I don't want to into now - this is harder than it should be. But it's the only way that you know that you have the latest package. I also recommend that you run the test suite on your local system(s). This is an incredibly exhaustive series of tests which almost guarentees that you have a bug free combination of compiler, linker, other libraries etc. This seems like it would be a lot of work - it is. But it's better than playing whack-a-mole trying to understand problems via experimentation. These days any bug still existing is very, very, very, obscure and almost impossible to find. It's very hard even to make a useful test case. And if you can reproduce it, it's almost impossible to fix. So I recommend downloading the whole of boost 1.71 and building everything from scratch.
If you advice to take only the commit that fix this issue, which is it?
I don't advise this, but if you want to do it this way, you'll just have to read the comments as I don't remember.
Now it's 0:00 in my part of the world, but tomorrow for sure I'll do the test and keep you inform.
Instead, how about doing what I suggest and informing me if it doesn't work.
Thanks for your help, and congratulation for this library.
You're welcome, I'm happy to be of service. Robert Ramey
On Thu, Nov 21, 2019 at 5:51 PM Nicolas Guillot <nicolas.guillot@gmail.com> wrote:
We use boost serialization in ours projects, both on linux (gcc) and Windows.
We encounter a problem only on Windows (for "every" versions of Windows (7 and 10) and of MSVC: Visual 2010 and 2015, ie. MSVC++ 10.0 _MSC_VER == 1600 or MSVC++ 14.0 _MSC_VER == 1900).
*Here is the problem summarized*, but I need to give more details after: When I serialize a class
- which have base class - and have 2 vector of shared_ptr of ClassB, with the same contents, and B inherits from the same base class
the serialization works, but the deserialization doesn't ("input stream error").
The following image is here : https://i.stack.imgur.com/osoiF.png (if it's not displayed)
[image: enter image description here] <https://i.stack.imgur.com/osoiF.png>
Yes, it's very strange:
- if I remove the base class, it works. - if I inline the serialization code (empty function) of the base class, it works. - if the 2 vectors don't have the same content, it works !! -
if I register the ClassBase in the serialize method of ClassTest, it works. I.e.:
// in ClassTest:template <class Archive>void serialize(Archive & ar, const unsigned int version) { ar.template register_type<ClassBase>(); ...}
*Some details*
*When it works* on linux, and when it works on Windows with ClassBase type registration in the archive, we have the same xml output. It's worth noting that:
- boost_serialization is version 17 on Windows and 9 on linux (I didn't tried to update boost on linux yet). - On BaseClass xml tag, the tracking_level="1", so we have object_id.
*And when it doesn't work* (once again: the serialization of ClassTest is OK, but cannot deserialized "input stream error"):
- The main difference is in the serialization of ClassTest: the ClassBase part has no tracking_level. - However it's not the only cause, because if the vectors are differents (same 2 first items, and a third item in one vector), tracking_level of ClassBase in ClassTest part is "0".
Last thing, if it can give you some clues: while we debugged our application, we noticed that the deserialization framework tries to deserialize a class into another. It makes me think the serialization framework is confused about the classe types identifications. Could it be a matter of dll linking? Of global static shared storage ?
I've posted a visual studio solution here <https://1drv.ms/u/s!Aoy4ax0S3ZS5gkxv1UIvRUd32oVO?e=ACVi4c>.
Thanks a lot for your help. And if any clarification is needed, I'm available of course !
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
participants (2)
-
Nicolas Guillot -
Robert Ramey