Re: [boost] [serialization] Doc problems

From: Robert Ramey (ramey_at_[hidden])
Date: 2005-09-04 13:21:28

David Abrahams wrote:

> 1. The menu control doesn't seem to keep the browser's displayed URL
>   in synch with the location being browsed (FireFox on Windows XP).
>   Wasn't that a major coup of Jonathan's menu control?

The table of contents doesn't use Jonathons's control.  I looked into it but 
it required that I mess around with allt the html (IIRC) and I was thinking 
that an eventual conversion to boost book would take care of this as a side 
effect.  I've checked back on boost book from time to time and it doesn't 
seem to include this "for free".  In fact, a cursory examination of boost 
book makes it look to me that could be another time consumer to set up and 
get running on my windows/xp developement system.  Also it seems that I have 
to install and keep up to date another bunch of stuff that I really don't 
have the time to keep up with (DOxygen?).  It may not be as bad as I think 
its going to be - but why risk it as I had/have other stuff that consumes my 
attention.

> 2. Deserialization of derived classes through base class pointers is
>   an incredibly important subject -- even for many first-time users
>   -- but the instructions for handling it are buried deep in
>   Reference > Serializable Concept > Pointers > Pointers to Objects
>   of Derived Classes
>
> (http://www.boost.org/libs/serialization/doc/serialization.html#derivedpointers)
>
>   This material _needs_ to be in the tutorial!  In fact, the material
>   in the reference section is narrative and tutorial in nature, which
>   seems inappropriate for a reference manual.  Maybe you should just
>   move it?

Maybe.  I'll consider it.  But importance isn't really a good reason for 
including it in the tutorial.  My decision of what to include in the 
tutorial is based on my own experience in trying to use other people's 
software.  I need something I can skim to see if the thing is useful for 
what I want to do.  I'm not really interested in understanding the 
intricacies of the package at this point.  That's for later.  I really want 
to know if it does enough in the way I need to justify investing more time 
in.  Understanding of the issues related to derived pointers is a lot deeper 
than I think anyone want's to go at this point.

It might be tempting to tweak the tutorial example to demostrate 
serialization through a base class pointer - but then one has to explain 
registration or export an instantiation of code not explicitly referred to.

All this make me quite skeptical of the idea.

Of course if I were writing a tutorial with David Abrahams in mind as the 
audience it would be quite different - actually I don't think we'd even need 
it!

> 3. In that reference section, '&' is used repeatedly where only '>>'
>   is appropriate.  For example:
>
>     main(){
>        ...
>        base *b;
>        ar & b;
>     }
>
>   That can only be confusing.

I'll look at this.

> 4. The documentation says that you can write your freestanding
>   serialize() function in namespace boost::serialization, with the
>   strong implication that it will work even on compilers that support
>   ADL.  But it won't work unless boost::serialization is an
>   associated namespace of one of the arguments, as demonstrated by
>   the following program:
>
>
>        namespace me
>        {
>          class X {};
>        }
>
>        namespace boost { namespace serialization
>        {
>
>          template <class T>
>          int call_serialize(T const& x)
>          {
>             serialize(x);
>             return 0;
>          }
>
>          void serialize(me::X);
>
>        }}
>
>        int y = boost::serialization::call_serialize(me::X());
>
>   As far as I can tell, there's no requirement that any of the
>   arguments to serialize have boost::serialization as an associated
>   namespace.

There isn't.  And I don't think it's necessary.  serialize(me::X) is only 
called from within the namespace boost::serialization never from anywhere 
out side this namespace.  Hence, the serialize function is found according 
to the rules of ordinary lookup.

Note that all the stl serializations (eg. boost/serialization/list.hpp ) all 
work on all platforms regardless of the namespace that the templated 
arguments are found it.

If one's compiler supports ADL, then he can use a free function in 
namespaces associated with the type being serialized.  But it's not a 
requirement

I concede I've struggled with two-phase lookup and ADL so I'm willing to be 
shown to be wrong about this.

> 5. Archive Concept Requirements are specified in a completely new,
>   unprecedented way.

Well that was certainly not my intent.  I studied the SGI documentation and 
the explanation of why it was written the way it was and tried to conform to 
it in substance if not exactly in form.  I also looked at the documentation 
for the new iterators to understand how to do this.

>   can see why you want to write them this way,
>   but the result is that it isn't completely clear which elements of
>   the interface are required and which are optional.  For example,
>   there's a colon after the class name.  Does the class need to be
>   derived from something?

The intention is to describe the requirements that an archive class must 
fulfill to be used with the serialization class that conform to their 
requirements.  It describes what an archive has to be able to do in order to 
work.  This is to be independent of any particular implementation of the 
archive concept and apply to any one.

>   What about the names of function and
>   member template parameters? I know the answer to that one, but a
>   novice might not.

Hmm, I'm extremely doubtful that a novice understand documentation which 
describes library templates in a formal way.  I suppose it depends who is 
considered an novice.

>  What about the default arguments to member
>   functions?  Is it okay to write overloads?

As I read this, I don't understand the question but I'll review the relevant 
section.

I will concede I struggled with formal documentation.  I never really 
understood the nomenclature and format of  "formal library documentation" in 
any detailed way.  The SGI website has been very helpful to me in this 
regard.  The boost page on how to write documentation by William Kempf has 
also been  helpful in various ways.

I resolved to make the reference part of the documentation conform to the 
"formal" standard and used the references I described above.  I found that I 
did have most of the required information in the documentation but that it 
was sort of jumbled about so I moved it about best I could to conform to the 
the "formal" standard.  Still there wasn't a good place for some things so 
they ended up in places like "Special Considerations".  layout

>  also, there's an extra pair of namespace closers in the second
>   archive concept example.

Ah, at last, an easy one to fix.

>
> 6. In
>
>
> http://www.boost.org/libs/serialization/doc/archive_reference.html#implementation
> it says:
>
>        All input archives should be derived from the following
>        template:

This would be incorrect.

This section describes the archive implementations included in this library 
and the common features that they have.   So the above should say: "All 
archives included in this library are derived from the following template"

>          template<class Archive>
>          detail::common_iarchive;
>
>   but that's nowhere to be found in the archive concept requirements.

As I said, I see this implementation section as A means to fullfilling the 
basic requirement not as new requirements.  For example.  The archive 
concept in no way requires that archive be implemented in terms of streams. 
However, all the archive classes in the library and all those derived from 
base classes in the library do in fact depend on streams.

The intention is that this is useful to those that might want to use these 
implementations as base classes.  But it is not required that these classes 
be used.  Any classes which fullfill the requirements of the Archive concept 
would be acceptable.

>   Which is it?  Also, that "detail::" is actually nested in the
>   boost::archive namespace, which is not at all clear from the text
>   there.

I've put them in detail as they are features of the implementation of the 
archive classes included - not interfaces that any users would be expected 
to have to know about.

>  It's not clear to me why archives should live in a
>   namespace other than serialization;

Ahhh.  This is every interesting.  And its clear that a key feature (to me) 
of the design of the library hasn't been made sufficiently obvious.

The library is two parts:

a) Serialization Concept.  This describes what a type has to support in 
order to be considered a serializable type.  All the support for this 
concept is in the namespace "boost::serialization".    Its concievable that 
someday someone (not me) might want to expand this to something more 
grandios such as Reflection concept.  There is already a baby step in that 
direction wiht the notion of serializable traits, assignment of an 
exportable name, assignmetn of a printable name for the variable which the 
type is applied to(name-value pairs).  There is to be no notion of the 
archive concept here.

b) Archive Concept.  This describes what an type has to support in order to 
be used an archive.  Basically these requirements boil down to the archive 
supporting the operations ar << and ar >> for any type which models the 
Serialization Concept.

Thus

a) any class fullfilling the requirements of the Archive Concept, can be 
used to serialize any types which model the Serializable concept.
b) in creating a new data type, it is sufficient to know that it fullfills 
the Serialization Concept to know that it can be serialized with any archive 
class.

This is THE key design goal of the serialization libary.

In fact, along with two requests for more formal documentation (one from 
you), one of my main motivations for investing effort in improving the 
documentation was to make this design much more obvious.  Hmmm - looks like 
it failed in this regard.  Oh well.

My intention has been to organize source code modules in subdirectories and 
matching namespaces to reflect this design.  This is described in a special 
section of the manual.  Also there is a VC7.1 ide *.sln file which contains 
all the library, tests and examples.  The library section is divided in to 
groups of files which reflect the considerations above.  If you have VC 7.1 
you can open up this *.sln file.  It is my key tool to keeping all these 
concepts straight and avoid confusion on my own part.  This is a huge job.

Although I've been pretty successful in maintaining this organization, it 
hasn't been easy and a couple of times I'm had to hack through a namespace 
that shouldn't be there.  But mostly its turned out OK.

The maintainence of the key distinction between serialization and archive 
concepts - and the fact that perhaps its not as clear in the documentation 
as it might be has a couple of other repercussions that have manifested 
themselves on this list.

a) Any header from the "serialization" part can be included in a header 
without triggering requiremetn for the library even in the presense of 
auto-link.  This would not have been possible without a library design based 
on this concept.  Actually supporting this was a some effort - mainly to fix 
places where I accidently violated this design feature.

b) My imposition of the "rule" that serialiation headers precede archive 
headers.  This came about as an afterthought while fixeing the auto-link 
issue described above.  Remember I always have in mind that serialization is 
an independent concept an archives are dependent upon seerialization.  So it 
never occured to me that one would ever mix and archive and serialization 
header in another header.   So I saw no issue in imposing this "rule".  As 
it turned out, this "rule" wasn't really the the right way to fix the 
auto-link issue above and now I'm manage to fix it so the rule isn't 
necessary anymore.  But I'm struck by the fact that lots of people never saw 
my point of view.

c) I sometimes get code which overides the serialization for a specific 
archive.  I can see where sometimes it might be interesting (eg. a special 
archive type for debug or checkpointing) but in the cases I've seen its just 
done without considering that one is now breaking the orthogonality between 
archves and serializations in a way that doesn't add anything other than 
future job security.  Its not a big thing - its just slightly irksome - like 
sitting on a small pebble.

Clearly there's an aspect of the library that I see as fundamental that 
either I'm wrong about or have failed to communicate.  Oh well.

>   if they were in  boost::serialization *and* users were required to 
> derive from one
>   of the archive types, that would solve problem 4.

I believe that problem 4 is not a  problem.  Feel free to prove me wrong 
here.

I very much appreciate you're looking at my documentation with your 
incredibly acute vision.  This is the first time anyone has really done this 
and find it very helpful.   I have spent more time on this than it might 
seem from looking at it and I do want to improve it - but it ain't as easy 
as it looks.

Robert Ramey