Re: [boost] [serialization docs] Ping?

From: Robert Ramey (ramey_at_[hidden])
Date: 2005-09-21 17:41:14

David Abrahams wrote:

Note that points not explicitly responded to have just been fixed in 
accordance
with your suggestion.

> It's an improvement, but either you missed or intentionally discarded
> some of my more important remarks.  The most important question was:
>
> **** Isn't something in an Archive required to call "serialize?" ****

Strictly speaking no.  Of course, I would expect all archive implementations
to call "serialize" but maybe someone comes up with an archive 
implementation
that just checks syntax or something like that.  Someone might want to make
an archive which only stored/loaded primitive types.  Or someone might make
an archive that was implemented solely in terms of save/load_binary.  These
would never call serialize. The point is that if one writes a program in 
accordance with the
requirements stated here, the program will be a legal program and will 
compile.

> If not, how does an Archive work with Serializable types?

That is defined by the particular archive implementation.  There are 5 
included
and maybe more on the way.  There may be one on the way that is only
for output - perhaps useful for debug logging.  These are not meant
to be prohibited by these requirements.

>> Common Type Definitions in all Archives
>
> This should be a section for the "Archive" concept.  These are not
> type definitions but valid expressions.

>> A::is_saving
>>
>>     An integral constant of type boost::mpl::bool_.
>
> I'm sorry to be so blunt, but that's nonsense.  A::is_saving is a
> type, not an integral constant, and boost::mpl::bool_ is a class
> template, not a type.  If you mean to refer to the MPL Integral
> Constant concept, the correct thing to say would be:
>
>         A boolean MPL Integral Constant
>
> with a link to the documentation for that concept.
>
>>     Value is boost::mpl::bool_<true>

OK what I meant to say is

A boolean MPL Integral Constant  for which the following expression is true

boost::mpl::is_equal<
    A::is_loading,
    boost::mpl::bool_<true>
>

<snip>

How would you suggest I phrase this?

>> Common Type Definitions in all Archives

>> Common Member Functions in all Archives
>
> Again, these are not member functions, but valid expressions.
> Presumably "ar & x" can be implemented by a free function and
> "ar.template register_type<T>()" invokes a member function template.

I have to confess I plagerized this exact phrasing from my copy of
STL Tutorial and Reference Guide - page 263.  I felt it was a very
close analogy and a good model.   I did note that in this reference
operators that could or would be implemented as free functions
were included under the "wrong" heading but I presume this was
done this way on purpose in order to improve clarity and reflect
the fact that most common implmentations would be member functions.

I could change the heading to "Common Operations for all Archives"
or .. your suggestion here.

>
>> ar & x
>>
>>     Returns a reference to ar. Appends the value of x along with
>>     other information to ar.
>
> I think you should swap the two sentences; emphasis should be on
> effects and only secondarily on return value.

Again I Plagerized this form the same source as above page 264.
I'm not sure this is a big issue

> But wait.  Surely this expression, with the specified semantics,
> doesn't have to be valid for loading archives?  Are you saying that
> all archives have to be able to save?
>
>> ar.template register_type<T>();

To deal with derived types not otherwise explicitly referred to,  any
invocation of this function must appear in both the saving AND loading 
archives
in the same place.  This is the mechanism by which saving and loading is
kept in syncronization.

> You should leave out "template."  This is a little tricky, but that
> expression will never be valid in a context where A is not a dependent
> type.  I realize you do need it when A is a dependent type, but the
> convention is to use the simpler notation.

Hmm - all Archives included with the library are templates.  So, removing
this will be including a "valid expression" which will fail to compile on
all conforming compilers.   I user who looks here to see what the
valid syntax is for invoking this operation, and uses it to get a bug will
be writing to tell me the manual is wrong.  I stll could use 
ar.register_type(& t)
- but that's confusing as well.  So what's a good solution here?

> The author of an
> archive isn't going to be calling register_type anyway (especially not
> in a dependent context), so it won't help prevent him from making
> errors.

This is extremely curious to me.  This whole section started as the
"Interface for users of the serialization library".  I would expect that 
this
is one section that would be of interest to users.   register_type is
essential to library users. So it has to be explained correctly right here.

>>   . count is an integer that can be converted to std::size_t.
>                  ^^^^^^^
> no, it's an instance of any type that can be so converted.
OK
> Your "strong typedef" isn't an integral type.

I don't see how this is related.

<snip repetition of the above but for loading archives>

> The following sentence belongs under "Archive Models."
>
>> There are archives based on text, binary and XML file formats but
>> all have the above interface.
>
>
>> Given that all archives present the
>> same public interface, specifcation of serialization is exactly
>> the same for all archives.

I think that should be pretty clear

> Are you telling me, after all this, that there are additional
> requirements on archives not documented in the section on Archive
> concepts?

No.

>>Archive classes have other members not
>> mentioned here.

For example, in order to use a text_oarchive, a user must invoke
the appropriate constructor with the appropriate arguments.

>> However they are related to the internal
>> functioning of the library and are not meant to be called by
>> users of an archive.

Hmm - now that I read this I'm not sure what I really meant.  I'll think
about this.

>I find this rather alarming.  Haven't I made it abundantly
> clear that the concept documentation has to give at least a minimal
> and complete documentation of what's required of an Archive?

I don't believe anything so far conflicts with that.

> Okay, I don't know what to say at this point.  I'm about to give up,
> because I don't have the time to keep going over this.  It sure seems
> like you want to do this your own way, rather than by following the
> established practices and norms.
> If you were just "creative" but
> still rigorous, it would be one thing, but it doesn't even seem like
> you're paying close attention to what you yourself are writing.

Actually that's not true.

I think the record on this list and CVS shows
multiple versions of trying to get this right. Other changes you have 
suggested
have been incorporated once the issues have been clarified.  I have based
my various versions on several known "good" models.  You have said
yourself that the latest version is an improvement.

> What should I do?

You're doing just great Dave, just hang in there.

>> Implementation of new archives is discussed in New Archives -
>> Implementation.
>
> Which then goes on to describe some "input archive" idea that hasn't
> been defined, etc...
>
> <snip>

Assuming that "Archive Concept" is more or less resloved, the question 
arises
regarding the rest of the documentation.  Basically there are two "large"
sections.

Serializable Concept.

Without having looked at it recently, I'm can assume this section doesn't 
meet
your definition of formal documentation.  I would guess this could be 
addressed
along the lines of Archive Concept, but I do have a couple of reservations 
even
before I look at it.  Much of the documentation there explains things in 
terms of
implementation behavior (how serialization of pointers works, etc) which 
doesn't
seem to fit all what well with the definition of formal semantics.

The whole question of how much semantics should be in a formal reference is
murky to me.  It seems that the idea of formal specification is to emulate
something like the specification of and algebra - not a bad place to start. 
But
specification of the rules for an algebra makes no reference at all to its
semantics - hence its wide applicability.

In the case of library specification we want to role library semantics into 
this.
It seems to me that that might obscure the whole idea in the first place. 
Your
question - when does the "serialize" funcition get called? and my response
"what does it matter" illustrate differing understandings about this.  I'm
starting to wonder - if its really a formal definition - how can it have 
sematics?
And if doesn't have semantics, how can it be useful?

This is the reason I was so intrigued with Joaquin's post.  It touches upon
what to me is the crux of the issue.  I'm not sure it resolves it - but it
does address the proper place of semantics in the discussion.

So, I'm still thinking about what needs to be done (if anything) regarding 
the "Serializable" type
concept section.

Don't feel obligated to coment on the above.

Achive Implementation

Here is the part of the documentation that I have been most concerned about.

I changed "Archive User Interface" to "Archive Concept" without thinking too
much about it.  I don't remember any user having any question about this. 
So
I wasn't too concerned.

This part of the library really describes a "tool kit" for implementing the 
Achive Concepts.
To make an analogy to your iterator library there are two things going.

Definition of iterator (archive) concept - A user of an iterator instance 
need only look
at this part of the documentation to use an iterator which models the 
concept.

Definintion of the iterator (archive) CRTP classes for deriving one's own 
iterators (archive)
This is a lot harder to explain and use.  I know because I did use it.  I 
should say that I was
very pleased with the results as once I got things to compile, I had very 
few problems.

So I've been planning to re-visit "Archive Implementation" in any case.  But 
I don't think there
is any quick fix.  I just have to spend more time on it.  And there are a 
couple of other issues.
I had to add to the interface a tiny bit in order to support shared_ptr 
serialization.  Its not a big
thing but I didn't want to document it because I'm thinking its not quite 
right (though its good
enough for shared_ptr) and I don't want anyone else to use it.  There might 
be a couple
of other things that might or might not need to be made public such as
stack_construct<T>(Archive &ar).

I'm still thinking about how to handle things BOOST_SERIALIZATION_NVP.  This
is optional for most archives but required for xml_archives.  So does this 
mean an
refinement on the concepts already described.  It would look  like it.

So - I would like to:

a) resolve the final issues regarding "Archive Concept" I don't think we're 
at all far apart in this.
b) ship RC_1_33_0 (again).  This is important to me as I've fixed the header 
sequence issue
as well as a nasty bug.  So I would like to see the new version shipped 
ASAP.
c) Give me some time to go back to the things I mentioned above.

I believe that the serialization library has proved quite useful to some 
people inspite of
the lack of formality in its documentation and shipping of a corrected 
version shouldn't be
held up for this.

Lastly,  I know I'm driving you crazy.  I'm sorry about that.  Please 
believe me that
its not intentional on my part - its just comes naturally.  I do sometimes 
think you
underestimate the depth of my thinking on some of these subjects.  Perhaps 
I'm
confused about some of these things because I think about them too much.
I don't know if its any consolation, but its stressful for me as well.  But 
your input
has been very helpful and added a lot to quality of the library and its
documentation and that's why I put up with it.

I sure hope you can laugh about this.

Robert Ramey