Re: [boost] [extension] Concatenating factory_map objects

From: Robert Dailey (rcdailey_at_[hidden])
Date: 2008-07-08 16:57:34

On Mon, Jul 7, 2008 at 9:42 PM, Jeremy Pack <rostovpack_at_[hidden]> wrote:
> Robert,
>
>> I'm developing a plugin system and I will be loading several DLLs.
>> Basically, as far as I know, this means having one factory_map per DLL
>> (on Windows).
>
> This depends on how you want to use the factory_maps. Do you want to close
> the shared libraries that you aren't using? If not, just use the same
> factory_map for every shared library. Unless you have a LOT of separate
> shared libraries, this is usually a good solution (if you have a lot, you
> may not want the unused libraries taking up RAM).
>
> If you do want to close unused shared libraries, it's a bit trickier. I had
> some code that did this, but have let it languish because it didn't cover a
> number of special cases. Here is more or less how it worked:
>
>   - Create a counted_factory_map instead of a factory_map. Internally, a
>   counted_factory_map contains a separate map and shared_library object for
>   each shared library.
>   - Whenever you ask it to add a constructor for a class T, it instead adds
>   a constructor for a class counted<T> (which is a derived class of T), which
>   increments and decrements a counter whenever it is created or destroyed.
>   - There is a function in the shared library that can be called to see how
>   many objects are currently live from the given shared library.
>   - You can call a function on the counted_factory_map to do garbage
>   collection - ie, close shared libraries with no live objects.
>
>
> The reasons this code is not currently used in Extension:
>
>
>   - It requires a duplicate of almost every class in Extension - a counted
>   version and a non-counted version.
>   - The counted versions have very high coupling - whereas the uncounted
>   versions of shared_library and factory_map are very loosely coupled. It also
>   makes it impossible to use parameter_map in its current form.
>   - It requires, in the common case, the use of mutexes to protect the
>   counters. This requires linking in the Thread library.
>   - It only works if all you are doing in a shared library is constructing
>   objects using counted_factory. If you take a reference to anything else in a
>   shared library, you risk having a dangling reference when all of the
>   counted<T> objects are destroyed. Note that this can be alleviated by using
>   a separate shared_library instance if you want to do anything besides
>   populate a counted_factory_map, since a shared library is only closed when
>   all shared_library instances referencing it are destroyed (it is also
>   possible to force a shared library to remain open until program termination
>   - by passing false as the second parameter to the shared_library
>   constructor).
>
>
> There were a few other special cases that broke it as well - but I can't
> remember what they were right now.
>
> I can update that code if people are interested. I certainly wouldn't mind
> suggestions about how to avoid some of the problems that I listed, or a
> different way of handling this sort of "garbage collection" of unused shared
> libraries.

Thanks for responding Jeremy.

I'm a little confused about your response, but perhaps that's because
I don't know as much about shared libraries as I should. Basically,
I'm assuming that the shared libraries must remain open in order to
continue to use the factories. I'm not sure what you mean by closing
the shared libraries I'm not using. I'm technically using the shared
libraries every time I call factory::create(). However, once I
concatenate the factory_map for each shared library, I do not use the
factory_map any longer. I simply keep them stored in a
boost::ptr_vector to maintain lifetime, and when I no longer need
access to my concatenated list of factories, I simply destroy the
ptr_vector of factory_map objects.

Right now this is what I have as members in a wrapper class of mine:

typedef std::map<unsigned int, boost::extensions::factory<Aspect> > MapType;
boost::ptr_vector<boost::extensions::factory_map> m_factories;
MapType m_aspectMap;

Say, for example, I load 3 shared libraries. As I add more shared
libraries, I am capable of creating more types, since each shared
library provides a way for me to extend the capabilities of my factory
(lets it create more types). In the case of having 3 shared libraries,
the m_factories member would simply be 3 in size. I do not use the
m_factories member anywhere, it simply serves the purpose of
maintaining the lifetime of the shared libraries. m_aspectMap is the
concatenation of the 3 maps returned by factory_map::get(). I
concatenate them so that instead of doing an O[log(n)] three times, I
only do it once, which is slightly more efficient.

The main issue I have is that the overhead of the 3 factory_map
objects must remain in memory, because I can't keep the 'factory'
objects alive without them, according to my research. I would simply
like to eliminate the m_factories member of my class and keep the
'factory' objects by themselves, and have the shared libraries they're
each associated with remain alive until all associated factory objects
have been destroyed, almost like the shared library is reference
counted (Just an initial idea).

I was looking for your opinion on the best way to approach this issue.
I figured that I perhaps was making this more complicated than it
needed to be, due to a feature I missed or something. I hope I've
properly emphasized my situation. I do apologize for any lack of
information in my original post.

If you could explain in more detail what you mean by using the same
shared_library for multiple libraries, I would appreciate it. I
figured there was a 1-to-1 mapping between factory_map and DLLs (on
windows).