Re: [boost] [serialization] Doc problems

From: David Abrahams (dave_at_[hidden])
Date: 2005-09-04 17:35:23

"Robert Ramey" <ramey_at_[hidden]> writes:

> David Abrahams wrote:
>
>> 1. The menu control doesn't seem to keep the browser's displayed URL
>>   in synch with the location being browsed (FireFox on Windows XP).
>>   Wasn't that a major coup of Jonathan's menu control?
>
> The table of contents doesn't use Jonathons's control.  

Aw, shucks.

> I looked into it but it required that I mess around with allt the
> html (IIRC) and I was thinking that an eventual conversion to boost
> book would take care of this as a side effect.  I've checked back on
> boost book from time to time and it doesn't seem to include this
> "for free".  

Not yet.

> In fact, a cursory examination of boost book makes it
> look to me that could be another time consumer to set up and get
> running on my windows/xp developement system.  

It's fast to get going if you just use Rene's installer.  Even so,
there are many of us who'd be happy to hold your hand.

> Also it seems that I have to install and keep up to date another
> bunch of stuff that I really don't have the time to keep up with
> (DOxygen?).  

Nope; if you want to use doxygen you have to use an old version and
not update it ;-)

> It may not be as bad as I think its going to be - but why risk it as
> I had/have other stuff that consumes my attention.

Anyway, this has nothing to do with #1, really.

>> 2. Deserialization of derived classes through base class pointers is
>>   an incredibly important subject -- even for many first-time users
>>   -- but the instructions for handling it are buried deep in
>>   Reference > Serializable Concept > Pointers > Pointers to Objects
>>   of Derived Classes
>>
>> (http://www.boost.org/libs/serialization/doc/serialization.html#derivedpointers)
>>
>>   This material _needs_ to be in the tutorial!  In fact, the material
>>   in the reference section is narrative and tutorial in nature, which
>>   seems inappropriate for a reference manual.  Maybe you should just
>>   move it?
>
> Maybe.  I'll consider it.  But importance isn't really a good reason for 
> including it in the tutorial.  My decision of what to include in the 
> tutorial is based on my own experience in trying to use other people's 
> software.  I need something I can skim to see if the thing is useful for 
> what I want to do.  I'm not really interested in understanding the 
> intricacies of the package at this point.  That's for later.  I really want 
> to know if it does enough in the way I need to justify investing more time 
> in.  Understanding of the issues related to derived pointers is a lot deeper 
> than I think anyone want's to go at this point.

a. You might be skimming this introduction to see if it handles
   base/derived class serialization correctly, especially if you've
   been disappointed by other serialization frameworks.

b. I've seen this phenomenon before.  In considering the Boost.Build
   docs, even after a simple Hello, World tutorial intro, it's clear
   that users need to be guided onward with more tutorial-style
   writing.  In other words, users need a tutorial, extended user
   guide, and formal/technical reference.  I think the same applies to
   Boost.Python.  Neither of these follow through optimally on the
   need for a user guide, yet.  On a smaller scale, I think we did
   pretty well with the parameter library, even though the tutorial
   isn't really divided from the user guide.

> It might be tempting to tweak the tutorial example to demostrate
> serialization through a base class pointer - but then one has to
> explain registration or export an instantiation of code not
> explicitly referred to.

Not necessarily.  In the tutorial section you can present it as a
"magic boilerplate incantation" that will be explained later.

> All this make me quite skeptical of the idea.

Proof is in the pudding.  My clients who have tried to use
serialization were unable to get it right, because they never found
that part of the doc, or if they saw it, they didn't recognize its
importance.  Serializing containers of pointers to polymorphic objects
is a pretty basic need.  You haven't had this question from users
before?

> Of course if I were writing a tutorial with David Abrahams in mind
> as the audience it would be quite different - actually I don't think
> we'd even need it!

?? I would *definitely* have needed it.

>
>> 4. The documentation says that you can write your freestanding
>>   serialize() function in namespace boost::serialization, with the
>>   strong implication that it will work even on compilers that support
>>   ADL.  But it won't work unless boost::serialization is an
>>   associated namespace of one of the arguments, as demonstrated by
>>   the following program:
>>
>>
>>        namespace me
>>        {
>>          class X {};
>>        }
>>
>>        namespace boost { namespace serialization
>>        {
>>
>>          template <class T>
>>          int call_serialize(T const& x)
>>          {
>>             serialize(x);
>>             return 0;
>>          }
>>
>>          void serialize(me::X);
>>
>>        }}
>>
>>        int y = boost::serialization::call_serialize(me::X());

In case it wasn't 100% obvious, the above fails to compile on a
conforming compiler.

>>   As far as I can tell, there's no requirement that any of the
>>   arguments to serialize have boost::serialization as an associated
>>   namespace.
>
> There isn't.  And I don't think it's necessary.  serialize(me::X) is only 
> called from within the namespace boost::serialization never from anywhere 
> out side this namespace.  Hence, the serialize function is found according 
> to the rules of ordinary lookup.

No, not in the example above it isn't, because serialize follows the
definition of call_serialize.  The set of candidates that can be found
by ordinary lookup is fixed at the template's point of definition.

You have introduced yet another header order dependency here.  You
don't need to experience the wailing and gnashing of teeth associated
with that problem again, do you?

> Note that all the stl serializations
> (eg. boost/serialization/list.hpp ) all work on all platforms
> regardless of the namespace that the templated arguments are found
> it.

I don't see what that has to do with anything.

> If one's compiler supports ADL, then he can use a free function in
> namespaces associated with the type being serialized.  But it's not
> a requirement

I beg to differ.  And the docs make it sound as though putting the
functions in boost::serialization is the more portable of the two
options.

> I concede I've struggled with two-phase lookup and ADL so I'm
> willing to be shown to be wrong about this.

The example above demonstrates.

>> 5. Archive Concept Requirements are specified in a completely new,
>>   unprecedented way.
>
> Well that was certainly not my intent.  I studied the SGI documentation and 
> the explanation of why it was written the way it was and tried to conform to 
> it in substance if not exactly in form.  I also looked at the documentation 
> for the new iterators to understand how to do this.
>
>>   can see why you want to write them this way,
>>   but the result is that it isn't completely clear which elements of
>>   the interface are required and which are optional.  For example,
>>   there's a colon after the class name.  Does the class need to be
>>   derived from something?
>
> The intention is to describe the requirements that an archive class
> must fulfill to be used with the serialization class that conform to
> their requirements.  It describes what an archive has to be able to
> do in order to work.  This is to be independent of any particular
> implementation of the archive concept and apply to any one.

Yes, I understand what the intent is.  Actually, there is precedent
for what you're trying to do, sorta.  The Indiana proposal for
concepts in C++0x
(http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1849.pdf)
uses something called a "pseudo-signature" that looks very much like
what you've written.  However, there are special rules for reading and
understanding pseudo-signatures.  One way to make your specification
formally complete would be to make sure that, when expressed according
to the rules in that proposal, it says what you mean it to say.  Then
you can reference the proposal in the docs in case someone wants to
understand the requirements on a formal level.

>>   What about the names of function and member template parameters?
>>   I know the answer to that one, but a novice might not.
>
> Hmm, I'm extremely doubtful that a novice understand documentation
> which describes library templates in a formal way.  I suppose it
> depends who is considered an novice.

Maybe.  It's certainly not complete enough for someone like me to be
sure of what it really means.

>>  What about the default arguments to member functions?  Is it okay
>>  to write overloads?
>
> As I read this, I don't understand the question but I'll review the
> relevant section.
>
> I will concede I struggled with formal documentation.  I never
> really understood the nomenclature and format of "formal library
> documentation" in any detailed way.  The SGI website has been very
> helpful to me in this regard.  

:) I'm using it on some slides now and discovering just how many
holes, anachronisms, and inaccuracies there are in it.  Still, the
basic approach is a good one.

> The boost page on how to write documentation by William Kempf has
> also been helpful in various ways.
>
> I resolved to make the reference part of the documentation conform
> to the "formal" standard and used the references I described above.
> I found that I did have most of the required information in the
> documentation but that it was sort of jumbled about so I moved it
> about best I could to conform to the the "formal" standard.  Still
> there wasn't a good place for some things so they ended up in places
> like "Special Considerations".  layout

I don't actually think Bill Kempf's format results in particularly
good docs.  I used that approach for Boost.Python's reference and now
I wish I hadn't.  I think the Parameter library (admittedly much
smaller) is much better.

>> 6. In
>>
>>
>> http://www.boost.org/libs/serialization/doc/archive_reference.html#implementation
>> it says:
>>
>>        All input archives should be derived from the following
>>        template:
>
> This would be incorrect.
>
> This section describes the archive implementations included in this library 
> and the common features that they have.   So the above should say: "All 
> archives included in this library are derived from the following template"
>
>>          template<class Archive>
>>          detail::common_iarchive;
>>
>>   but that's nowhere to be found in the archive concept requirements.
>
> As I said, I see this implementation section as A means to
> fullfilling the basic requirement not as new requirements.  

You said that?  Where?

> For example.  The archive concept in no way requires that archive be
> implemented in terms of streams.  However, all the archive classes
> in the library and all those derived from base classes in the
> library do in fact depend on streams.
>
> The intention is that this is useful to those that might want to use
> these implementations as base classes.  But it is not required that
> these classes be used.  Any classes which fullfill the requirements
> of the Archive concept would be acceptable.
>
>>   Which is it?  Also, that "detail::" is actually nested in the
>>   boost::archive namespace, which is not at all clear from the text
>>   there.
>
> I've put them in detail as they are features of the implementation of the 
> archive classes included - not interfaces that any users would be expected 
> to have to know about.

I don't care about your rationale---at least not yet.  I care about
the fact that the doc is unclear/confusing.

>>  It's not clear to me why archives should live in a
>>   namespace other than serialization;

I can't answer the rest tonight, sorry.  Gotta sleep.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com