Re: [boost] [serialization] Header Placement

From: David Abrahams (dave_at_[hidden])
Date: 2006-05-22 15:35:45

"Robert Ramey" <ramey_at_[hidden]> writes:

> Hmm - if I knew everything I know when I started, I might have
> done a lot of things differently.  I do remember the motivation
> for some of decisions which I can state here which might or
> might not be helpful.
>
> David Abrahams wrote:
>> The serialization library seems to blatantly violate Boost header
>> placement conventions.
>>
>> 1. Normally detail namespaces are for implementation details that
>>   should never be touched by a user of the library, but Serialization
>>   puts several framework classes designed to be used by archive
>>   authors inside of detail subnamespaces.
>
> I made a distinction between archive authors - a small group and
> archive users a much larger group.  

Nothing wrong with that, but in associating that distinction with a
namespace called detail, you violated the Boost convention for the use
of that name.  

> I put those things of interest only to archive users in detail
> namespace.  Even so, there are only a couple of things there - abi
> stuff, and common implementations I would expect archive authors to
> use without having to alter or even look at.

If you want to organize things that way, call the namespace something
else.  You can even leave a namespace alias around for backward
compatibility.

  namespace boost { namespace archive { 

  namespace construction 
  {
     ...
  }

  namespace detail = construction; // backward compatibility.

  }}

>> 2. Also, the top-level boost/ directory is supposed to be reserved for
>>   non-implementation details, i.e. files with documented purposes
>>   that expose the interfaces of accepted libraries, yet
>>   boost/pfto.hpp is clearly an implementation detail of the
>>   serialization library.  It is not a header I would bet any other
>>   Boost library would use, but if I'm wrong about that, it should at
>>   least be pushed into boost/detail, where common implementation
>>   details live.
>
> When I did that, I saw pfto.hpp as a general solution to addressing
> the problem of compilers which fail to properly implement
> partial function template ordering.  I had no way to forsee whether
> or not others would find this useful, but I certainly didn't see it
> as anything specific to serialization. 

You are not hearing me.  The contents of boost/pfto.hpp are not part
of the public interface of any Boost library, so they belong in a
detail (sub)namespace.

I have *lots* of files in detail subnamespaces of my libraries.  When
I or someone else finds them useful in another library, we move the
files to boost/detail.  If we want these interfaces to go public, we
create a new library with its own documentation, and get it into the
review queue.

The same "I had no way to forsee..." argument could have applied
equally well to most of these files.  If everybody did what you did,
the boost/ directory would be littered with hundreds of files that are
not public library interfaces, most of which were used by only a
single library.  Boost established these conventions for a reason, and
blatantly violating them because the file _might_ be useful to more
than one library is not only premature generalization, but it
undermines the efforts of your Boost colleagues to maintain a sane
directory structure.

> I don't know if anyother boost library uses this to address the
> problem, I did find it necessary to implement "dataflow iterators"
> in a portable way.  So its useful in other contexts aside from
> serialization.  I suppose I should have documented it as a separate
> thing in "misc".

No, you should have left it in boost/detail.

>> The conundrum of item 2 is partly the from a mistaken organization.
>> boost/archive/ and boost/serialization/ are segments of the same
>> library, and they should have been organized as such.  An obvious
>> organization would've been boost/serialization/archive and, e.g.,
>> boost/serialization/streaming.  Then boost/pfto.hpp could've been
>> boost/serialization/detail/pfto.hpp
>
> We've been here before.  I went a lot of effort to maintain the
> distinction between archive and serialization - "concepts" (uh oh)
> and having that reflected in the namespaces used.

That's fine.  There's no reason to presume that the right parent
namespace/directory for these nicely-separated groups of functionality
is the top level boost namespace/directory.  Grouping them under a
common subnamespace/subdirectory would be more appropriate.

> I realize that not everyone sees the value in the separation, but
> I'm convinced that it has been invaluable in making the library
> easier to build and maintain.

I absolutely agree.  

> I could have make another layer
>
> boost::
>     ???::
>         serialization
>         archive
>
> and sprinkled ???:: all over the place 

That is *exactly* what I was suggesting (though I used different names).

> with no gain in clarity

Wrong.  What you did had a cost to the overall clarity of the Boost
codebase.  While the above scheme wouldn't have made your personal
code any clearer by itself, it would have avoided de-clarifying Boost
as a whole.

> The directory organization mirrors the namespaces which
> seems quite natural to me.

Directory organization mirroring namespaces is a well-established
Boost practice.  Of course.

> It also wasn't clear to me that "serialization" wouldn't be be
> eventually absorbed in something like a future system implementing
> "reflection" or that "serializable" might find its way into some
> sort of type_traits thing.  I wanted the concept of a serializable
> type to stand independent of the archive.  These things havn't come
> to pass, but they were present in my mind at the time.

Yes, anything could happen.  Cross that bridge when you come to it.
Again, this is a case of premature generalization that undermines the
convention many of us have worked hard to establish.

>> These unconventional moves are at best confusing for users and other
>> maintainers.  The Boost source base is hard enough to control without
>> library authors inventing their own new rules for organizing things.
>> I realize that these problems can't be repaired all at once, but they
>> should be fixed.
>
> I think you're blowing this way out of proportion.

Of course you think that.  You didn't think our conventions were worth
respecting in the first place.  Some of us have worked for many years
to figure this stuff out before the Serialization library came along;
maybe it's natural that you don't appreciate the effort that has gone
into shaping the development of Boost before you came on the scene.

>> I'd start by pushing boost/pfto.hpp into boost/detail, for example.
>
> I can see the apeal of that - 

This is not about "appeal."  It's not a matter of my personal
aesthetic.  It's a matter of being a good citizen in this organism we
call "Boost."

> feel free.

Aren't you willing to take at least a tiny bit of responsibility for
these problems?  After all, I didn't cause them.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com