Re: [boost] [Review] Shmem, Serialization ad I/O streams (was: Shmem)

From: Reece Dunn (msclrhd_at_[hidden])
Date: 2006-02-12 06:37:26

Andy Little wrote:

NOTE: I haven't had time to look at this library yet, only read a few of the 
reviews.

>The restrictions placed on the types of objects that can be placed in 
>shared
>memory are understandable, but heavy and presumably cant be checked?. Now 
>there
>Must be a special definition of an shareable-object outside C++ class eg 
>like a
>COM interface. In fact some evidence that this is occurring with classes 
>such as
>basic_named_shared_object. However should these objects not be designed for
>distributed environment as well as local one? If shmem doesnt provide 
>protocols
>for building objects that can be passed over a network and in a scaleable 
>way
>then objects using its mechanisms direct will be of limited use, so larger
>picture should be taken into account Now.

The shmem library will provide a common data block between process A and 
process B. From the shmem library PoV, this is just a block of data who's 
contents is user defined. That is OK from the PoV of this library as it 
should just be concerned with the memory sharing and synchronisation 
thereof.

If process A is writing data to that block, process B should be aware of 
that and not start reading until process A signals that all the data is 
written (especially if process A is writing the data incrementally). 
Similarly, process A should not be allowed to write to that memory while 
process B is reading from it.

With this in place, transfering C++ objects across processes becomes easy if 
you know the type of object being passed. Boost already has a mechanism to 
do this (serialization), so you could make use of the Boost.Serialization 
library to persist the objects to a shmem memory block rather than a file. 
Therefore, the only thing that the shmem needs to provide is a serialization 
archive that works for its shared memory resources.

What would be nice is if the I/O part of the serialization could be seperate 
from the archive format. (I am not sure this is possible with the current 
serialization library). That way, you could do:

   object --> binary archive --> shmem | shmem --> binary archive --> object

or

   object --> xml archive --> shmem | shmem --> xml archive --> object

where shmem provides what is needed by the different archives in terms of 
I/O.

Does the shmem library provide an I/O stream library sink and source? If it 
did, then it would be easy to serialize data using exising C++ streams.

>**Construction ***
>
>On construction issue , only because its been brought to light recently.
>
>     named_shared_object segment;
>     //Create shared memory
>     if(!segment.open_or_create(shMemName, memsize)){
>     std::cout << "error creating shared memory\n";
>     return -1;
>
>could (presumably?) be replaced with
>
>     try{
>         //Create shared memory
>         named_shared_object segment(shMemName, memsize);
>     }
>     catch( shmem_exception & e){
>         std::cout << "error creating shared memory\n";
>         return -1;
>     }
>
>which I would prefer.

what happens if you have a class called shared_data that uses shmem, and use 
it like this:

shared_data data;

int main()
{
   return 0;
}

If this throws an exception, your application will crash and it could be 
difficult to determine why. Doing something like that, you should at least 
provide an error policy, so you could bring up a sensible error and exit 
gracefully. For example:

struct shmem_exit_policy
{
   static void creation_error()
   {
      std::cout << "The application failed to initialize." << std::endl;
      assert( false );
      std::exit( -1 );
   }
};

>***Should it be a boost library**
>
>You betcha. Yes indeed It should definitely be a boost library. C++ needs 
>this
>type of library badly. It is all very low level however and I would like to 
>see
>a higher level abstraction for designing objects that can be passed about 
>over a
>network too though and shmem objects should then try to conform to that, 
>but
>that shmems role would be maybe as a building block for that. Needs to be
>discussed. If it is in Docs.. Sorry I missed it!

If shmem provides Boost.Serialization support and iostreams support then it 
would be easy to leverage those libraries to provide the high-level passing 
of objects you describe.

- Reece