Re: [boost] extensions/plugins library, conclusion

From: Robert Ramey (ramey_at_[hidden])
Date: 2006-10-10 00:29:45

Jeremy Pack wrote:
> Janek and Robert (and others),
>
> First of all, the library that I'm working on does use the
> Serialization library, but not to the level that Robert is suggesting.
>
> Two points:
>
> 1 - The exporting of classes from a dll or whatever is not a terribly
> difficult part to implement - if this were all that I needed, I would
> not have started my own library. However, I agree that this part
> SHOULD use boost::serialization (not reusing already proven code
> would be a mistake in this case). As long as the way that has been
> suggested will meet a few criteria, I think work should immediately
> begin on doing it using Robert's suggestion. I will check through the
> extended_type_info documentation to see if it will be able to provide
> each of the requirements that I feel necessary. It sounds like it does.

Hmm - hopefully it does.  But it probably doesn't make them
available in the most convenient way for general usage.  Remember
I just made it to solve a specific problem - serialization of
derived pointers through a base class.  So it will have to
be looked at in a new light.

> 2 - The primary requirement that is important to me is tracking
> dependencies between loadable objects. I'm working on a relatively
> flexible way of handling that. I think it is a vital part of making
> this library more useful than just a classloader. It's also, I think,
> the hardest and most subjective part.

And this is something not addressed at all -

> I will still attempt to get my code posted, keeping in mind that most
> of the object loading will probably be completely rewritten to use
> extended_type_info.
>
> Robert, if you have the time, I believe your experience creating the
> boost::serialization library would make you the most qualified to
> head up the effort to create a plugin library for Boost. Even if you don't
> have that kind of time, your continuing input would be very
> appreciated.

All I know about the subject is included in the code and documentation
for extended_type_info.  I'm happy about offering that as a starting
point - and answer any questions about why I used X to fullfill
requierment Y (if I can remember) but that's the best I can do.
Your on your own from now on. Of course, as a "veteran booster" I reserve
the right to complain about your solution along with everyone else
when its done.

Also I'm not altogether sure that what I've done is useful to you.
I just want to make sure you look at to see if we can factor out
some commonality.  If you take an honest look at it an conclude
it doesn't work for you, I'm fine with that.  I just want to make
sure that an opportunity to make things smaller (and off load
part of the library to someone elses library) isn't missed.

> One important question: should we avoid requiring RTTI? Should
> boost::serialization make it unnecessary? I'm currently using a
> dynamic cast or two, mostly just for type safety. I know that certain
> older compilers have a problem with that (Borland at one time, perhaps 
> still, cannot
> handle RTTI in DLLs correctly for instance), and it might annoy
> people to have to remember to enable it on those that do not.

Here is how the non-RTTI facility came to be and how it fits in.

The first review of the serialization was subjected to whithering
critique during the boost review process and failed to be accepted.

Lots of issues were raised by different people.  One reviewer (G Rosenthal)
however, distinguished himself by making an exhaustive
review of all its flaws and the problems he saw in the implementation.
This list had about 12-15 items on it.

At first I was pretty discouraged.  Then I started picking at the library
using this list.  A couple of things weren't that hard to fix.  Hmmm
as time went on I went fixing each thing.  And each thing was harder
to fix than the previous thing.  But after a while, the whole project
had me by the throat again.  (I'm a very determined person).
Finally there was only one thing left on the list - make the usage of
RTTI optional.  I was planning to blow this off as only one person
had even mentioned it in the review - and it wasn't obvious how to
implement it.  But I said to myself - I've come this far, and following
the review critique actually has resulted better than I thought - maybe
this won't be so bad - maybe not as hard as it looks.  I don't remember
how hard it was.   I also had to make those extended_type_info based
on RTTI to  interoperate with those which use any other system.
This is what the "test_no_rtti" tests.

But in the course of all this it became clear that
extended_type_info could be almost factored out as a separate
library - thereby permiting consideration of these almost orthogonal 
libraries
to time share my attention and permit the full power of the steadily
shrinking surface area of my aging brain to be focused extended_type_info
independent of serialization.

So I had an almost independent library - extended_type_info.  But
although it kept the serialization simpler, it wasn't clear how much
value - if any it would have outside the library.

Then something really surprising happened.  People started using
defining classes and thier serialization in DLLS.  And the whole
system continued to function (after a couple of very minor tweaks).

This worked for both DLLS imported at pre-runtime via the usage
of and import library as well as DLLs loaded and unloaded dynamically
at runtime.  It turns out that loading a DLL constructs the
extended_type_info object when the DLL is loaded.  The extended_type_info
constructor addes the objects to a global table.  Unloading does the 
opposite.
(I had to make the destructor more elaborate remove destroyed
objects from the global table).

So the whole managment of keeping track of which types can be
dynamically created via GUID is totally automatic and dynamic !!!
(Of courses can still hang themselves by unloading a DLL before
a required destructor is called - but I can't help them there)

This came for free - just be trying to fix things that occurred when
some ambitious users went where no man had gone  before.

So that's how we came to where we are today.  I don't know
if all this is interesting - but there it is.  I just wanted to get it off 
my chest.

Robert Ramey