Pool Review

From: Jeff Garland (jeff_at_[hidden])
Date: 2000-12-03 16:13:25

I recommend Pool be accepted into boost.

Here are some thoughts:

Overall:
You should add Bulka and Mayhew (Efficient C++, ISBN 0-201-37950-3) to your
concept references.  If you haven't looked at this reference, you should.
They devote 2 chapters to the topic of C++ memory pools.  Chapter 6: Single
Threaded Memory Pooling looks at the performance of fixed and variable sized
pools.  Chapter 7: Multithreaded Memory Pooling builds a multithreaded
memory pool class.

The good...
1) Thanks for this important contribution!  This is a good start on an
essential library for many types of high performance applications.
2) Initial performance measurements look good.
In a simplistic case (allocating and deallocating a single object) pool
allocation showed an 8.7 times speed up.  See below for details.
3) Nice documentation of the library design internals.

The bad....
1) User examples are needed.  Please, some usage examples somewhere in the
docs

2) The pool interface. I'm not fond of the pool interface.  The idea of
  pool.construct(...some_parms...)
just isn't very clean.  One common way to do this is to a class specific
storage pool overriding new/delete for the class.  This keeps client code
closed to the details of how things are allocated.  I have uploaded a
modified version of the pool template called type_specific_pool which
implements the helper routines (based on pool) which enable a class to
provide class specific storage using.  A class wishing to use this mechanism
must create a singleton instance of the type_specific_pool template and
implement new and delete using the helper functions.  It looks something
like this:

class simple {
public:
  //..the usual stuff
  void* operator new(size_t size)
    {return pool()->domalloc();};
  void* operator new(size_t size, simple* rhs)
    {return pool()->domalloc();};
  void operator delete(void* target, size_t size)
    {pool()->dofree(target);};
private:
  //create class specific storage type
  typedef boost::type_specific_pool<simple> pool_type;
  static pool_type* pool() {
    if (!simplePool) {
      simplePool = new pool_type();
    }
    return simplePool;
  }
  static pool_type* simplePool;
}
//allocate the storage...
simple::pool_type* simple::simplePool;

You can see the details in the code I uploaded.

2) Ok, so what's the advantage?  As a user I want to seem some example (even
trivial) of the advantages of taking the trouble to use a memory pool.  It
seems to me it is critically important to have documentation of performances
gains.

I have performed a simple test which allocates and deallocates an object 10
million times.  It compares the allocation performance of a class that uses
a custom new/delete based on the library and one that doesn't.  The bottom
line was a respectable 8.7 times speedup.  This measurement was taken using
gcc2.95.2 mingw32 on Windows98. I took 20 runs to ensure accurate timings.

Obviously this just scratches the surface.  Some other tests I would like to
see include MT versus no MT, slowdown due to pool reallocations, allocator
tests, more complex objects, etc....

3) Threading.  I think the pool interface should be parameterized by locking
strategy.  If I know I am in a single threaded environment (or using thread
specific storage) I want to maximize speed and hence I want to compile out
all threading code.  Since I might not have the same threading decisions for
all objects in an application, what I really want is to parameterize the
threading decisions on a pool by pool basis instead of "the whole library".
In addition, on some platforms I may want custom locking primitives which
are faster.

I'll refer you again to Efficient C++ (page 104).  Their fixed size single
threaded memory pool achieves a 23 times speedup over global new and delete.
The MT version only gets a 1.6 times speedup (and that's with custom locking
primitives for the platform).  They use a null lock to implement the non-MT
version of their pool.  It looks like:
  template <class POOLTYPE, class Lock> ...

4) Running destructors.  Another useful feature would be to not run
destructors when deallocating.  For non-resource utilizing objects this is
an appropriate policy.  I can't say how much it speeds things up, but ILOG
Solver, which provides a C++ constraint propagation engine used in
performance critical domains, uses an internal heap for it's objects that
never runs destructors.  I implemented this as a policy option for
type_specific_pool.

So now you can say
  boost::type_specific_pool<mytype,32,false>
and the pool will never destroy a mytype.

The release method on the simple class and the pool code runs the destructor
based on the selected policy.  Normal delete is still going to run
destructors, and if the template parameter is set to true then destructors
will be run by release or on pool destruction.

5) As boost users gain experience using these classes, there should also be
documentation on performance tuning.

The ugly...
1) The dependency diagram and documentation map in index.html renders
strangely.  Thus I am left wondering what you mean.  Check out
pool_dependencies.jpg in the uploaded files and see if it represents what
you are trying to show.
2) Copyright in the html should be at the bottom and in a readable font.
3) Garbage collection.  I would expunge the word garbage collection from the
documentation.  Pool is not a traditional garbage collector in the sense of
something which periodically runs to recollect memory. As you say it
"provides automatic destruction".  I would add "upon destruction of the
pool".

Thanks again for the excellent library!

Jeff