Re: [boost] [async] messages threads and networks

Subject: Re: [boost] [async] messages threads and networks
From: Scott Woods (scott_at_[hidden])
Date: 2010-04-26 04:16:05

Hi,

>> I made the same point about a Pact version of a similar operation. It
>> was never about whether users of Boost.Serialization could add the
>> code you have added. It was about the fact that there is no
>> Boost.ArchiveFactory.
>
> I don't seen the need for any archive factory. In nine years
> no one has asked for such a thing either.  Either because no
> one needs one, or making one is such trivial task.
>
>> At the important levels they are apples and oranges. When someone
>> chooses a serialization solution out of Protobuf, Boost.Serialization and
>> s11n they are choosing much more than a function that converts an
>> application type to a sequence of bytes.
>
> Hmm - I would say just the opposite.  A serialization library derives its
> utility from it's clearly defined narrow purpose.  It is for this reason
> that it can be used in a variety of contexts.  It has been used for
> persistence, marshalling (e.g. see MPI library) and probably others.
> It can be used with any stream type.  Stream types can use any
> encoding and (almost) any protocol.

I respect the noble calling. For some of the solutions mentioned I think
that approach has been to the expense of initiatives involving combinations
of components.

For example, with a certain approach to the decoding phase of serialization
there is no need for the framing machinery that has been a subject on
several support forums including Boost. That may be as part of network
messaging or an attempt to store a sequence of mixed-type data items
in a file.

You mention the MPI library. From the section "Separating structure
from the content";

<quote>
When communicating data types over MPI that are not fundamental to MPI (such 
as strings, lists, and user-defined data types), Boost.MPI must first 
serialize these data types into a buffer and then communicate them; the 
receiver then copies the results into a buffer before deserializing into an 
object on the other end. For some data types, this overhead can be 
eliminated by using is_mpi_datatype. However, variable-length data types 
such as strings and lists cannot be MPI data types.
</quote>

Why is this special handling so common around network messaging and
files of data sequences? Isn't there merit in trying to fix this at the root 
cause?

<excerpts>
> I think your documentation should focus more on what your library does 
> rather
> than what you believe other libraries can't do.

> I think that your proposal suffers from a common failing that is shares 
> with
> many proposals received here.

> This turns out to be the exact wrong approach.

> For an example of a good library - look at iterators:
> all the others.  That's power!!!

> After 30 years
> grep is probably the single most frequently invoked program
> on the planet !!!

> have take euclid pages to describe.  That's Power!!!

> That is what we should strive for.  Those should be our
> inspiration.  I realize this thinking is out of style and that
> I sort of got carried away.  Sorry, but I feel better now.
</excerpts>

I should hope so! :-)

Love the passion. Still there after nine years of Boost.Serialization?

My own passions are actually parsing technologies and network protocols. 
Because
of years in the telephony/networking field I was forced (yeah; kicking and 
biting)
to adopt SDL. In the end I had to concede it was a good thing. For a certain 
flavour
of work it's now the only hammer I keep in the toolbox.

Pact is an implementation of SDL (with a dusting of Active Objects). It's 
been
the vehicle for indulging my own coding passions for several years. It's 
only recently
achieved the completeness that is hopefully apparent from documentation and
samples. It is a minimal implementation of an international standard, not a 
behemoth
staggering under the weight of a bloated feature set.

SDL is not an iterator or a grep or a VM supporting a half-dozen algebraic 
opcodes.

As far as my failure to incorporate Boost components goes; that is both a
little sad and a little embarrassing. In some cases potential components did
not yet exist and in other cases I provided feedback hoping to move designs
in directions indicated by SDL. In the case of Spirit I was deeply motivated
to deploy. But along with it's blindingly clever use of overloading it also 
accepts
a constrained set of grammars and has an input sytem not amenable to change.
Authors of Spirit should not take insult from this as eventually Bison did 
not
allow the necessary changes either.

The point of that last paragraph is that if I had committed to implementing
Pact with only Boost components I would still be discussing the proper
feature set of Boost serialization (imagine that, for the last 9 years). And 
the
feature sets of other components. And there would be no Pact.

Genuine thanks,
Scott