- Boost - lists.boost.org

Re: Re:[boost] Formal review: serialization
by Robert Ramey 21 Apr '04

21 Apr '04

I just realized I skipped something Vladimir Prus wrote: > > ** Custom archive creation ** > > I've tried to create a custom archive class (in fact, > polymorphic_oarchive), > and got some comments. > > - Documentation says the derived class must provide operator<< and method > 'load'. It also says there common_iarchive::operator<<. I wonder why the > derived class should provide operator<<. Can't base operator call 'load' > directly? > common_iarchive has no operator<< 'load' is called by the library to load primitives - its generally not called by the << operator unless one wants to bypass the serialization system entirely. Perhaps, my usage of load_override was confusion. I use load_override in order to work around the lack of partial template function ordering. Were it not for this, load_override would not exist. This is an implementation artifact. Load on the other hand is the function used to load primitives. > - Even if not, why derived operator<< should have > > return * This(); > > not plain "return *this"? After all, we're in most derived object > No we are not. The class hierarchy is common_iarchive base_text_oarchive text_oarchive_impl text_oarchive in my archives, the operator<< is a member of base_text_oarchive. If I want the << operator to return a reference to a text_oarchive NOT basae_text_oarchive. The archives included are models of the trivial_archive. However, the desire to factor things out in a convenient way makes them less transparent than I would prefer. Robert Ramey

1 0

Re: [boost] Re: Using Serialization for binary marshalling
by Robert Ramey 21 Apr '04

21 Apr '04

Matthew Vogt wrote: >Robert Ramey <ramey <at> rrsd.com> writes: >> Correct. That's exactly what the current implementations of STL classes >> do. So my question is why override vector for you xdr/cdr archives? > Primarily, because the implementation you have chosen for default > serialization of STL containers is just an implementation detail from your > point of view; you could quite reasonably change it in the next version, > if someone with alternate requirements requested that, and it didn't > conflict with your own purposes. OTOH, XDR and CDR mandate how sequences > should appear in a stream, so it is necessary to ensure that the default > implementation is not relied upon for conformance. OK, but then why not just delay the effort until the underlying implementation actually changes? > Another reason, is that other formats will vary the implementation in > minor ways - by using a preceding length field in 8-bit signed format, for > example. I believe this could be addressed without too much trouble - perhaps by making a special type (a strong type) to hold collection count and tweaking collection implementation to use it. So it could then be implemented to taste on each kind of archive. >> I would like to see your xdr/cdr archives finished off at the same >> level of generality as the other archives. Any special purpose >> marshalling would leverage off that. > I don' think this is possible, since these formats do not specify > Sufficient types to do so. The library maps an arbitrary C++ data structure to a sequence of C++ primitives and back again. Any format that can handle all C++ primitives can therefore be used to serialize any C++ data structure. I'm sure that XDR/CDR can represent any C++ primitive data type. Hence, XDR/CDR can be used to serialize any C++ data structures with this library. > It would be possible to allow type promotion > to cover some missing types - for example, in XDR, it would be possible to > allow all C++ integral types less than 32-bits to be promoted to 32-bits > during serialization, but I think this would be misleading. Misleading? > More problematically, there is no concept of objects in this situation. Of course at the archive level you only see a string of numbers. It's the serialization that recovers C++ objects from all of this. > It wouldn't make sense to transmit objects via pointers, since 'the other > side' will never reconstitute objects. Perhaps this is a source of confusion. It never occurred to me that the other side wouldn't have the mirror serialization code. That is of you're using xdr_oarchive to make the stream I assumed you would use xdr_iarchive on the other side to recover the data. Even if you don't anticipate the need for doing this yourself, other users will and they will want the universal coverage that the current system provides. I wager that if you tell users on this this list - "Oh the XDR archive is special, it's just for marshalling, you can't do objects with XDR" you'll get an earful. And its not necessary to subject oneself to this kind of grief. It's much less work to provide for every C++ primitive data type and know that any serialization is always going to work than it is to try to anticipate ahead of time for just those combinations of features users are going to actually use. If you want a "restricted" or special purpose xdr archive that prohibits pointers or something else in order to conform to some other program not built with this library, you can just derive from the simple general one and overload the prohibited operations with compile-time assertions. Robert Ramey

3 2

a question about "typeof"
by Arkadiy Vertleyb 21 Apr '04

21 Apr '04

Hi all, A couple of questions regarding typeof: 1) Is it going to be in the next C++ standard? 2) Is there any estimate when it is going to happen? I assume that quite a few Boost libraries could largely benefit from this operator. Spirit documentation even contains a section on typeof and how the syntax could be improved with it. I recently had a need to determine a type of a Lambda functor (but got away with Boost.Function with some performance expences). Also Joaquin recently raised a similar question. Our RTL syntax would also benefit a lot if this facility was available. About one and a half year ago Steve Dewhurst published three articles on this subject in CUJ. The idea was to encode the type as an integer using bitwise operations. This integer could be than "returned" from the function, this crossing the boundaries between a type and a value, and then decoded into the original type. I wonder if this or any other technique is currently being considered by anyone to be added to Boost or we are just waiting until the real typeof is available? I have been experimenting with this, and came up with a very simple trick that allows to use a static list of integers, like mpl::vector_c, instead of bitwise operations, to encode the type, and pass this list across the function boundaries. I think, if nothing else, this is much simpler than bitwise operations. Everything else is pretty much like in Steve's approach. Based on this it seems to be pretty simple to encode/decode types to implement typeof-like facilities for some specific (or maybe not so specific) cases. Has anybody already done something similar, and/or is this of any interest? If there is an interest I am ready to provide explanation, code, etc. Regards, Arkadiy

5 5

gcc-3.4 release imminent
by Neal D. Becker 21 Apr '04

21 Apr '04

gcc-3.4 is now frozen. I wonder if anyone is testing boost compat? I have tried compiling some of my code using gcc version 3.4.0 20040327 (Red Hat Linux 3.4.0-0.9). There are usually a LOT of errors. gcc-3.4 doesn't seem to like boost iterator adaptor stuff, in particular. Also lambda. Many errors are of this nature: src/c_to_r.hpp:83: error: there are no arguments to `base_reference' that depend on a template parameter, so a declaration of `base_reference' must be available

5 7

Re: [boost] Re: serialization: review
by Robert Ramey 21 Apr '04

21 Apr '04

Thorsten Ottosen wrote: > Robert Ramey wrote:(> |) > | I don't understand the need for this. Given that mammal is an abstract > | base class with no state variables of its own, it needs no serialization > | at all. > Ok, but could I have implemented it by having just one function in the > abstract base class which seriaized by calling virtual functions, eg, > name() and age()? I believe you could. In fact I believe that in some cases not yet encountered ( runtime plug-ins ) it will be necessary. This is still an unresolved area > | Just declare it as abstract using BOOST_IS_ABSTRACT(mammal). Only the > | serialization of the derived classes need be specified. > Can BOOST_IS_ABSTRACT be avoided? Maybe some type-traits can figure this > out? Is_abstract is in the new type traits. It fails on lots of compilers. Also it was in a state of flux at the time of the release of 1.31. I expect that a) is_abstract get fixed up best it can b) and have an implementation that is benign for compilers that don't support it c) and that we'll continue to specialize it for serialization code that needs to be compiled on less-conforming compilers. (this is all BOOST_IS_ABSTRACT does.) In > struct whale : public mammal > { > friend class boost::serialization::access; > whale(); > whale( const std::string& name, unsigned age ); > virtual const std::string& name() const; > virtual unsigned age() const; > private: > template<class Archive> > void serialize(Archive &ar, const unsigned int version){ > ar & boost::serialization::base_object<mammal>(*this); > ar & name_ & age_ ; > } > std::string name_; > unsigned age_; > }; > can the line "ar & boost::serialization::base_object<mammal>( *this )" > be put in a base-class like this > mammal::serialize( Archive& ar, unsigned version ) { > ar & boost::serialization::base_object<mammal>(*this); > do_serialize( ar, version ); > // virtual function that does the rest > } > to remove some redundancy? The intention of base_object is really to cast to the base class (and register the base/derived relationship). So it its out of place here. The real issue is do_serialize. Is that supposed to be a virtual function? With a templated argument? Maybe you want to consider this question a little bit more. > | I would love to hear more about this. I would hope that it would easy > | to be totally successful. > it is now given you showed me how :-) In effect, once the container > supports serialization, a class can be supported in four lines. If you > have not got an example of class hierarchies, then maybe it should be > added. The tutorial contains an example essentially identical to your mammals, whales and apes example. Under case studies there's a discussion of the implementation of serialization for shared_ptr which is almost exactly analogous to your vector_ptr case. Also, there some discussion in the documentation regarding serializaition of templates. > | > I would like to see the focus shift from vector<T*> towards vector< > | > shared_ptr<T> > and ptr_vector<T>. using T* array[sz] or vector<T*> > | > is inherently unsafe if the pointers are heap-allocated. > | > | I believe that enforcing this is outside my role as a library creator. > ok, but I think you at least should point out very clearly the problems in > it. Some non-experienced programmer might do what he sees in > documentation; therefore examples should encourage good style. Well, here is one programmer that needs to be better sold on this. Honestly, I don't think the serialization tutorial is the place to do it. > | 3. const members uses a const-cast to re-init them. AFAICT, this is > | undefined behavior (cmp 7.1.5.1). I guess the need for serialization > | is just another reason for not having const members. Hmmm - what does re-init mean in this context? > | > this has come up from time to time. I've tried to promote the view > | > that const members shouldn't be serialized but I couldn't convince > | > everyone. > | > I consider it outside my role as a library developer to try to > | > enforce this. > my point was that the examples given give undefined behavior so that > solution cannot be used. Which example gives undefined behavior? > no, but maybe with this > | stl::archive_input_seq<Archive, std::vector>, > | stl::reserve_imp<std::vector> > by arranging the order of the template parameters. > | Isn't a little bit of redundancy OK giving the alternative of having > | to keep the details of deducibility in one's brain? > but you already know the types that will be deduced from the outer > function. OK, could we agree to disagree on this and forget about it? > | > 18. implementation of save_collection(). you cannot use unsigned int > | > count portably. > | > | I don't see this. > | > | > Instead you should use the new collection traits (reviewed after > | > this review) as follows: > | > | > typename boost::size_type_of< Collection >::type count = size( s ); > | > | > then the code will also work for iterator views and arrays. Similar > | > comments apply to load_collection_impl.hpp > | > | I wanted to make the library buildable under 1.31 . > ok, then do > typename Collection::size_type count = s.size(); > but don't assume Collection::size_type will be unsigned int. I'm not assuming that. I'm assuming that it will be convertible to an unsigned int and that an unsigned int will be large enough to hold any number of elements in an collection. Of course THAT might be rash assumption. BTW where is the namespace Collection defined in 1.31 ? Robert Ramey

2 1

RE: [boost] Re: date-generators (was Date-Time)
by Hurd, Matthew 20 Apr '04

20 Apr '04

> Jeff Garland > > > Cyrille Dupuydauby wrote > > Totally agree with that, business related apps need > holidays and business > > days. But holidays determination should be easily localizable. > > It isn't just by locale, the application needs to determine > holidays. For > example, while there are some holidays in the US that > everyone agrees on, but > lots of companies use a couple floating holidays every year. > They typically > will tack one say a Friday after a July 4th on a Thursday to > give employees a > 4 day weekend. So any calculation of business days needs to > be capable of > handling these application specific cases. Yep. And just to emphasise the point you just made... Some jurisdications might have a business day with respect to some activity but not others. For example, an exchange might have a trading day but not a settlement day. A foreign exchange contract might be written in NY effective on a Sydney holiday but the settlement date might depend on the union of holidays between Sydney and NY. All this indicates is that a calendar can be completely arbitrary and that different calendars can apply to the same business at the same point in time for the same jurisdiction for different parts of the same transaction. Like me, the world has warts ;-) Regards, Matt Hurd. IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

1 0

RE: [boost] Re: Date-Time
by Powell, Gary 20 Apr '04

20 Apr '04

>Ummm, OK, you made it work, but why? Why would you want to be >able to call add_month() on a date and have it always represent >the day before the last day of the month? date d = August 30; // add six months. add_month(d); // September 30; add_month(d); // October 31? add_month(d); // November 30 add_month(d); // December 31? add_month(d); // January 30 add_month(d); // February 28/29 add_month(d); // March 28? 29? 30? 31? Just looking this there is no way to predict which day of the month the last call is going to be. I'd find it surprising that (d + year) != (12 * months) + d; I think you are looking for a simplistic function for a complex problem. IMO its a trap for the unwary and should be called out as such. I can see the need for LAST, but my contention is that its not enough information to do the least surprising thing. The easiest way to resolve this is to either write a FAQ, and or have a dumb rounding month adding function that can screw up the dates for those times when "meet you next month" is the right answer. Yours, -Gary-

4 9

Re: numeric_cast
by Dill, John 20 Apr '04

20 Apr '04

"Rob Stewart" <stewart(a)sig.com> wrote in message news:<200404201905.i3KJ55e08992(a)lawrencewelk.systems.susq.com>... > From: "Dill, John" <john-dill(a)uiowa.edu> > > > > I am wondering about the use-case of numeric_cast in this sample. > > > > unsigned char uchar_max = std::numeric_limits<unsigned char>::max(); > > char schar_value = 0; > > > > try { > > schar_value = boost::numeric_cast<char>( uchar_max ); > > } > > catch ( boost::bad_numeric_cast ) > > { std::cout << "Overflow occurred..." << std::endl; } > > > > When I execute this sample, I don't get the exception. What's > > the background on this behavior? I'm using gcc 3.3.1. > > Your char is unsigned. I'd like to know why conceptually the exception isn't thrown. Why is it beneficial to have unsigned T to signed T overflows not be detected? It is not just char, but short, int, and long as well. If this was the intended behavior, then what is the reasoning behind it? Thanks, John

2 1

Re:Re: Re: Re: [boost] Re: [serialization] + [boost.python serialization of python object
by Robert Ramey 20 Apr '04

20 Apr '04

Ralf W. Grosse-Kunstleve wrote: > FWIW: For the serialization of C++ arrays wrapped as Python objects (via > Python's pickle) I implemented a small "library" (it really is just one > header file) for converting integers and floating-point numbers to a > pseudo text format. In principle it works just like the conversion to > base-10 numbers, but uses base-256 instead. I.e. the result looks like a > binary format, but it is as machine-independent as a text format. The > serialized strings are smaller than regular text format, but larger than > raw binary format. > Integers are serialized like this: > NXX...X ... This is interesting. I made an example as part of the library called:demo_portable_binary or which attempted to handle issues such as endianness and size. I bailed at floating point because of the concern for handling things like conversion between 80 bit IEEE float and 64 proprietary floats. And then I was just making as sketch to inspire someone to implement XDR which I presumed had some sort of way of handling this. (I still don't know how other formats handle this) I like your method. I may steal it yet. Robert Ramey

1 0

Re: Re: Re: [boost] Re: [serialization] + [boost.python serialization of python object
by Robert Ramey 20 Apr '04

20 Apr '04

> Ralf W. Grosse-Kunstleve wrote: > Interesting. If I serialize and deserialize std::vector<double> as a text > archive but on the same machine, will I always get back exactly the same > bit patterns for the double values? The text archive uses a stream manipulator to set the precision of the output to capture all the precision in the double. It uses the numeric<limits> to determine this. So I can't say it will be exactly the same bit stream but it will be close to the original number. If you want to guarantee the exact representation you can either use the included native binary archive or serialize the data element (double) as a (non-portable) binary object. Robert Ramey

2 1