Re: [boost] Gauging interest in a thread-safe containers library

Subject: Re: [boost] Gauging interest in a thread-safe containers library
From: vicente.botet (vicente.botet_at_[hidden])
Date: 2009-02-10 13:27:50

Hi Eric,
----- Original Message ----- 
From: "Eric Greveson" <eric_at_[hidden]>
To: <boost_at_[hidden]>
Sent: Wednesday, January 14, 2009 12:34 AM
Subject: [boost] Gauging interest in a thread-safe containers library

> 
> Hi all,
> 
> I've recently been doing some work that required thread-safe
> containers (queue, channel) and I found that in addition to these, a
> locking_ptr with similar semantics to boost::shared_ptr (except being
> thread-safe too)

locking_ptr is not thread-safe, but it allows you to make the pointee data thread-safe.

> allowed me to perform all my threaded operations
> without needing to worry about synchronisation outside the
> container/locking_ptr code, and without a major impact on performance
> compared to coding locks in a slightly less generic (== paranoid) way.
> These have been re-used with some success by others, and a couple of
> containers (specifically the queue and channel) seem to be used
> commonly enough to be worthy of an inclusion in Boost.Thread or
> similar.
> 
> This discussion has been had in the past, on this thread:
> 
> http://lists.boost.org/Archives/boost/2004/09/71454.php
> 

Sorry but I have not yet started to see these atomics operations, so I can not help on tis.

> Someone else mentioned it later, but there doesn't seem to be much
> else said here:
> 
> http://lists.boost.org/Archives/boost/2005/11/96991.php

The queue from Olivier could be greate,but offen the user has already other locks and so what we need is external locking. My Synchro library provides several ways to get this.

> I was wondering if there is more interest now, a few years on. Some of
> the pros I can think of are:
> 
> * Writing correct thread-safe code can be tricky (I messed up the
> first time I wrote my queue).
> * Use of multithreading is perhaps more prevalent now, with many-core
> CPUs, than it was 4 years ago

I agree.

> * Queues / channels in particular are very useful for building
> threaded pipelines etc, and more people might be tempted to use such
> programming techniques if a safe option was available "out-of-the-box"

Could you be more explicit about threaded pipelines? Could the ThreadPool library help you?

> * A thread-safe shared pointer provides a lazy but effective way of
> synchronising data in many scenarios (my implementation supports the
> usual boost::shared_ptr stuff except for get(), -> and *:
> dereferencing is provided by calling an acquire() function which
> returns a scoped_lock object on which you can call * and ->.The
> destructor of the scoped_lock releases the lock.)

Ye this is a realy useful and easy to use class. I have also my own.

> * The thread-safe shared pointer means you could use a pop-copy queue
> without significant overhead (depending on use)

Could you be more explicit?

> * The code length for these class templates on a single platform is
> actually quite short (a hundred lines of code or so)

Well the relative small code size is not what makes these classes more usable and maintenable. We need to take care of most of user needs.
  
> Some cons:
> 
> * Need to support all the platforms Boost works on

This should not be a big problem for most of the features (except lock-free if asm is needed.)

> * Not much interest for these containers last time

This can change.

> * More code to maintain

I agree. But also to test, document, ...

> * Questions about how re-usable a synchronised queue is due to
> performance worries (I believe this not to be a major problem for a
> major class of problems - e.g. pipelining fairly long operations, such
> as in image and video processing)

Have you take a look to the Dataflow library?
 
> I'm sure I've missed some things, but I'd be interesting in hearing
> whether there's a positive or negative reaction to this. I'd be happy
> to contribute code for this - it wouldn't be the fastest possible
> code, but I believe that robust, lightweight, maintainable code is
> better than nothing in this case, even if it's not the very best
> lock-free implementation.

You can contribute to my Synchro library with suggestions, code&tests, documentation. See https://svn.boost.org/svn/boost/sandbox/synchro for the moment.

Best regards,
Vicente