Re: [Boost-users] [general question] on threading and double checked locking pattern

From: Gottlob Frege (gottlobfrege_at_[hidden])
Date: 2007-01-22 17:45:34

On 1/22/07, Ovanes Markarian <om_boost_at_[hidden]> wrote:
>
> I might be wrong here, but I would like to hear some thoughts of C++
> professionals.

not sure I qualify,  except in the technical definition of 'professional'
(ie I do get paid to write code :-).  But I'll give it a try - I have spent
lots of time dealing with these issues.

I read in lots of sources about DCLP and how it is broken in conjunction
> with Singleton pattern
> and my question is of a very simple nature. Is it possible to solve this
> problem by introducing
> function calls via function pointers? Or is there still a way, that these
> calls might be inlined
> and if yes then how could this look like? This example is not about
> freeing the singleton, it is
> just about creating it.

I see 1 or 2 problems here, neither of them inlining.

class Singleton;
> typedef Singleton* (*singleton_getter)();
>
> class Singleton : boost::noncopyable
> {

...

};
>
> //implementation
> Singleton*       Singleton::pInstance = NULL;
> singleton_getter Singleton::getter    = &creating_singleton_getter;

problem 1.)  can you really guarantee that getter is set before it is ever
called?  I think it is OK, as it is not a dynamic constructor, just a
initialization that can be done at compile time.  Or maybe, more accurately,
link time.  Which raises questions about whether this will work OK inside
DLLs, etc.  But it probably is OK.  And then again, if you throw threads
into the mix, the standard says nothing.  Even things like 'global variables
are set before functions in the file are ever called' (paraphrasing,
obviously) don't apply when threads are involved, at least when it comes to
dynamic construction.  But, as I said, this compile/link time *might* be
better.

Singleton* Singleton::creating_singleton_getter()
> {
>         //aquire mutex here
>         if(Singleton::pInstance==NULL)
>                 Singleton::pInstance = new Singleton;

*** problem 2.1: need a write barrier here, between writing Singleton and
writing getter ***

        Singleton::getter = &non_creating_singleton_getter;
>         return Singleton::pInstance;

*** problem 2.2.a: the following mutex unlock is just a write barrier, it
probably isn't a read barrier on some other processor ***

} //mutex unlocked

Problem 2.)  when considering DCLP stuff, you need to look at memory
barriers (full), and partial ('acquires' and 'releases').  In particular,
note that in this case:
   2.1) you write out the new Singleton and then set the getter, without a
barrier in between - so to another processor, they may appear to have
happened out of order.  So another thread can come in and see 'getter'
BEFORE seeing the bytes of Singleton properly written out.  You need a write
barrier there after constructing Singleton (because you don't enforce a
mutex/barrier on the get).

Overall, nowadays, you need to think of memory just like a harddrive - when
writing data onto a harddrive, the harddrive controller gets a big queue of
things to write out.  It looks at the queue and decides to be smart - it
will reorder the queue *in sector order* so that it can minimize seeks -
effectively changing the *temporal* order in which the bytes were written.

Similarly, when reading from disk, a bunch of read-requests are queued, and
the controller reorders them to minimize seeks, so the temporal order of the
reads is changed.

Now, for memory, it isn't exactly minimizing 'seeks' but almost the same -
it is trying to read in full cache lines or contiguous memory, maintain its
cache(s) etc. The result, for us programmers, is the same - everything is
screwed up.  We just didn't notice it before, because we didn't often read
and write files in ways that depended on temporal order.

Singleton* Singleton::non_creating_singleton_getter()
> {
>         //no mutex anymore

*** problem 2.2.b: need a  read barrier here ***

        return Singleton::pInstance;
> }

   2.2) the unlocking of the mutex in creating_singleton_getter() (2.2.a) is
just a 'release' or 'write barrier'.  When you later use/read 'getter', even
if they were written in the right order (ie solve 2.1), you might still not
*read* them in the right order, without a read barrier - ie even if
Singleton was written to memory before getter, you might still read getter
before Singleton.  Which really seems unintuitive and I even doubt myself as
to whether I unrderstand it and can easily explain it, but I'm pretty sure
it is a problem.

Singleton* Singleton::instance()
> {
>         return (Singleton::getter)();
> }
>
>
> With Kind Regards,
>
> Ovanes Markarian

Tony