Re: [boost] [serialization docs] Ping?

From: Robert Ramey (ramey_at_[hidden])
Date: 2005-09-22 00:35:11

David Abrahams wrote:

>>  The point is that if one writes a program in
>> accordance with the requirements stated here, the program will be a
>> legal program and will compile.
>
> But that's not enough.  I'm not interested in making an archive that
> just compiles; I want it to have some sensible semantics.  What
> requirements do I have to fulfill in order to make the archive
> actually *work*?
>
> If you look at the iterator requirements, for example, you'll see that
> there are semantic requirements in place, not just syntactic ones.
> These requirements are needed so that the algorithms can say something
> about what they do.  The sort() algorithm wouldn't be able to make any
> claims about its result if writing into a random access iterator were
> allowed to be a no-op.

I can see that, but then one has an ostream_iterator which is a forward
iterator.  It does something quite different from other forward iterators
such as one associated with a std::list.  I'm sure I could dig up other
examples.  I really did think about what one should say about the
sematics.  And when I considered possibilities like an archive used
for data logging, it seemed that there was less and less I could
say about the semantics without saying something incorrect.   So
I moved to saying as little as possible.

> As I've said before, you will need eventually to describe the
> relationship between compatible loading and saving archives.  It will
> be something like
>
>   T x, y;
>   // arbitrary operations on x to set its state
>   sar & x;
>   lar & y;
>
>   Postcondition: y is equivalent to x
>
> [Someone else suggested a complex way to describe this idea without
> mentioning the word "equivalent" but I've no idea whether it hangs
> together, and at least this is good enough for the standard]
>
> You can't say anything like that unless loading and saving are
> required to do _something_.

<snip>

I really couldn't give a good response in a reasonably short time.

>> Of course, I would expect all archive implementations to call
>> "serialize" but maybe someone comes up with an archive
>> implementation that just checks syntax or something like that.
>> Someone might want to make an archive which only stored/loaded
>> primitive types.  Or someone might make an archive that was
>> implemented solely in terms of save/load_binary.  These would never
>> call serialize.
>
> Do not build in generality that you're not even sure is needed,
> especially if it costs you your ability to write concepts that tell
> people how to get sensible results.  You can always introduce
> less-refined concepts that allow these kinds of things and provide
> weaker semantic guarantees.

My thought was not to build in any requirement that wasn't necessary.

>>>> Common Type Definitions in all Archives
>>>
>>> This should be a section for the "Archive" concept.

That was my intention.  I didn't make a subtitle "Archive Concept" because
I already had the chapter titled "Archive Concept"

>> OK what I meant to say is
>>
>> A boolean MPL Integral Constant for which the following expression is
>> true
>>
>> boost::mpl::is_equal< A::is_loading, boost::mpl::bool_<true>
>>>
>>
>> <snip>
>>
>> How would you suggest I phrase this?
>
> I believe that requirement is too strong, and it would be sufficient
> to say
>
>   An MPL Integral Constant with a nonzero ::value member
>

So if somewhere in the implementation I use:

boost::mpl::is_equal< A::is_loading, boost::mpl::bool_<true>

but the archive has A::is_loading defined as boost::mpl::int_<4> it will 
still work?
If you say its true, I don't doubt it.  But I would have hoped for a compile 
time
error here.

> register_type is not a member function but a member function
> template.  A template is not a function.
>
>> I could change the heading to "Common Operations for all Archives" or
>> .. your suggestion here.
>
> The heading should be "Archive Concept Requirements" with a note that
> each entry describes an expression that must be valid.

How about just "Requirements" as its already part of a section titled
"Saving Archive (Concept)"?

>      Surely the expression "ar & x" , with the semantics, "appends
>      the value of x along with other information to ar", doesn't have
>      to be valid for loading archives?
>

whoops - OK I see that now.

> This is the sort of thing that makes me think you're not really paying
> attention.

Well I confess the symetry and similarity of the text between the passages
makes it easy to overlook this kind of stuff.  This is expecially so when
one has written it himself.

> Also, I just noticed "along with other information."  If I'm writing
> an archive, how am I supposed to fulfill that requirement?  Can I
> write any arbitrary "other information" I want?  Suppose I write a
> zero-valued char?  Suppose I don't write _any_ "other information?"

This "other information" is an implementation detail of the particular
archive class.  The user of any archive implementing the concept
doesn't have to know what other information is in the archive.

I do believe I'm begining to see what the origin of this problem is.
I'll address it below after a few details are out of the way.
>
>>> You should leave out "template."  This is a little tricky, but that
>>> expression will never be valid in a context where A is not a
>>> dependent type.  I realize you do need it when A is a dependent
>>> type, but the convention is to use the simpler notation.
>>
>> Hmm - all Archives included with the library are templates.
>
> Irrelevant.
>
>> So, removing this will be including a "valid expression" which will
>> fail to compile on all conforming compilers.
>
> No.
>
>        some_archive_template<Whatever> ar;
>        ar.template register_type<T>();
>
> is ill-formed.
>
>        some_archive_template<Whatever> ar;
>        ar.register_type<T>();
>
> is well-formed if T is a type and Whatever is a valid template
> argument to some_archive_template.

I don't see this.  I am looking at Appendix C.13.6 of  The C++ Programming
Language by Stroustrup page 858.  It seems that he has our exact
example there:

<begin quote>
class Memory {
public:
    template<class T> T* get_new();
    ...
};

template<class Allocator> void f(Allocator & m){
    int * p1 = m.get_new<int>();    // syntax error: int after less-than 
operator
    int * p2 = m.template get_new<int>(); // explicit qualification
    ...
}

Explicit qualification of get_new() is necesary because its template
parameter cannot be deduced.  In the case the template prefix
must be used to inform the compiler (and the human reader) that
get_new is a member template so taht explicit qualification with
the desired type is possible.  Without qualification with template,
we would get a syntax error because < would be assumed
to be the less-than operator
<end quote>

Some compilers don't accept the ar.template synatax so that's where
I came to be where we are.

>> I user who looks here to see what the valid syntax is for invoking
>> this operation, and uses it to get a bug will be writing to tell me
>> the manual is wrong.  I stll could use ar.register_type(& t) - but
>> that's confusing as well.  So what's a good solution here?
>
> Do it the way I'm suggesting.  If you like, add a footnote for the
> uninitiated explaining that when A is a dependent type, the "template"
> keyword may be required.
>
>>> The author of an archive isn't going to be calling register_type
>>> anyway (especially not in a dependent context), so it won't help
>>> prevent him from making errors.
>>
>> This is extremely curious to me.  This whole section started as the
>> "Interface for users of the serialization library".  I would expect
>> that this is one section that would be of interest to users.
>
> If you want this to be directed only at users, take the "Concept"
> label off it and put that elsewhere.  Go back to doing whatever you
> want here to informally explain archives to users, and in some other
> section, give me real, complete concept requirements that tell me
> what I need to fulfill in order to create a working archive.

OK !!

> The explanation I've suggested _is_ correct.

>>> Are you telling me, after all this, that there are additional
>>> requirements on archives not documented in the section on Archive
>>> concepts?
>>
>> No.
>
> Maybe I've misinterpreted the contents of
> http://www.boost.org/libs/serialization/doc/archive_reference.html#implementation
>
> Is that not describing the _real_ requirements for a working archive?
>
>>>> Archive classes have other members not mentioned here.
>>
>> For example, in order to use a text_oarchive, a user must invoke the
>> appropriate constructor with the appropriate arguments.
>
> Okay, that's reasonable, but doesn't merit a mention here; it's just
> confusing to do so and adds nothing.
>
>>>> However they are related to the internal functioning of the library
>>>> and are not meant to be called by users of an archive.
>>
>> Hmm - now that I read this I'm not sure what I really meant.  I'll
>> think about this.
>
> OK.
>
>>> I find this rather alarming.  Haven't I made it abundantly clear
>>> that the concept documentation has to give at least a minimal and
>>> complete documentation of what's required of an Archive?
>>
>> I don't believe anything so far conflicts with that.
>
> Maybe I've been too hasty, but it seemed to me, from the lack of any
> requirement that the archive do something that interacts with the
> Serializable concept (and produce sensible semantics) and your
> reference to a separate section describing how to implement an
> archive, that you had been trying to make the "Archive Concept"
> section a "user-only" document, and had put the real requirements
> elsewhere.

correct. But I didn't think of them as "real requirements",  I thought
of them as "implementation details" from the perspective of the user.

> True, you have responded, but the record also shows that I've needed
> to ping you several times to keep from being ignored -- or so it
> seemed -- which has been discouraging.

Its hard to respond as quickly as you would like.  A lot of question
require that I reflect on why I originally did something a certain way.
I didn't necessarily write it down because I can't know ahead of time
which things are going have to be given this kind of treatment.  On the
other hand for the poster its easy to say "Why don't you do X" but to
respond can take a lot of time thinking through all the implications
and re-discovering why one didn't do  it in the first place.  In the
recent case handling two-phase lookup I did that in february and in this
case I think it was May.  And I do have other things going on besides
the serialization library.

> Maybe they're leading you astray.  If you stop and think about some of
> the things I mentioned in this email I think you'll see that some of
> them just don't make sense except under the loosest, most informal
> interpretations.
>
> No, that's not true.  An algebra's semantic rules describe invariants.
> The meanings of statements in an algebra is constrained their
> invariant transformations.
>
>          a = b   ==>   b = a
>
> is semantic.

As much as I'd like to respond to that it would lead us too far astray

>> In the case of library specification we want to role library
>> semantics into this.  It seems to me that that might obscure the
>> whole idea in the first place.  Your question - when does the
>> "serialize" funcition get called? and my response "what does it
>> matter" illustrate differing understandings about this.
>
> Clearly.
>
> It's all about practical concerns.

I'll go with that

>
>> This is the reason I was so intrigued with Joaquin's post.  It
>> touches upon what to me is the crux of the issue.  I'm not sure it
>> resolves it - but it does address the proper place of semantics in
>> the discussion.
>
> First plug the dyke and then, if you have time,
> think about a rewrite.

see below

>> I changed "Archive User Interface" to "Archive Concept" without
>> thinking too much about it.
>
> When was that?  Maybe that was the root of the problem.

That was several months ago.  No one seemed to notice - or care
until now.

> It's not formality that's the problem: it's vagueness, inconsistency,
> and lack of a practical document that describes what an archive
> implementor needs to do.

Maybe - But I haven't yet heard from anyone who wants to do this
but failed because of this.

Anyway I see now that my view of the role of different sections of
the documentation is way out of sync with what you expect.

To summarize my thiniking was

a) "Archive Concept" defines what a user has to know in order to
make a type a serializable type. That is what expressions are valid
on an Archive.  This started as basically a class interface in the
traditional C++ documentation model.  It seemed a natural
generalization and formalization to move it to a "Concept"
as a list of valid expressions.  This fixed the problem that it really
wasn't a class.

b) Given a), "Serializable Concept" defines what a type must have
(requiremetns) in order to be serializable.  This is not stated in
a formal table.  In fact its somewhat different in that there are
several disjoint sets of requirements that a type can fullfill and
still be "Serializable"

c) "Archive Implementation" would describe how a) was implemented
and how to leverage on the included code.  This is a very informal
narrative.   This also started out as basically notes on a class
implementation.  This kind of broke down as things got factored
into interface and implementation from which template classes
are derived using CRTP.  This may well lend itself
to a formal treatment but its nowhere near that now.  I never
saw documentation for any thing like this.  I always found that
disconcerting.  But now I feel better about it.

So I think I made a big mistake when you first raised the issue.
(this is why I hate to be pressured to respond too soon!)

I think what I should have done and what we should do now is:

a) change the title of the section "Archive Concept" back to "Archive User 
Interface"
or "Archive Interface". and restore the original content which is based
of function prototypes.
b) change the title of the section titled "Serializable Concept" back to 
"Serializable types".
c) The question of formal documentation for the serialization library
can be considered at leisure.

Robert Ramey