Re: [Boost-users] circular buffer in a multithreaded program

Subject: Re: [Boost-users] circular buffer in a multithreaded program
From: Sebastian Gesemann (s.gesemann_at_[hidden])
Date: 2014-04-22 17:52:48

Am 22.04.2014 20:02, schrieb Antony Polukhin:
> 2014-04-22 15:47 GMT+04:00 Sebastian Gesemann <s.gesemann_at_[hidden]>:
> <...>
> 
>>   - locking a mutex
>>   - calling cb.erase_first(n);
>>   - calling cb.resize(cb.capacity());
>>   - notify writer that there is new free space to write stuff into
>>   - unlocking the mutex
>>
> 
> This is what was happening in the reader thread. What was happening in the
> writer thread?

Let me back up a little ...

I wrapped the circular_buffer into another object that offers this
interface:

  range allocate(int n); // for writer
  void commit(int n);    // for writer

  const_range read(int n); // for reader
  void consume(int n);     // for reader

Apart from the container that backs up the storage (which used to be a
circular_buffer), it also contains

  - a mutex
  - two condition variables:
    - enough_writable (which allocate blocks on)
    - enough_readable (which read blocks on)

All of these four actions include locking the mutex. The first function
of a pair returns a range object (think of it as a pair of iterators)
with which a thread actually reads or overwrites the data _without_
owning a lock. The second function of a pair updates the state to
indicate that either new data is available for a reader or that new free
space is available for the writer.

The only interesting bit is the consume function which I tried to
describe in my first email. Other than that, the only things happening
with circular_buffer was invoking begin() and/or end() and doing some
"iterator arithmetic" in allocate/read while having a lock.

So, essentially, begin/end/erase_first/resize was only executed when
having a lock where the last ones did not invalidate any iterators that
were in use somewhere. Deref'ing of the iterators was not locked.

My guess is that "working" with an iterator while concurrently doing an
erase_first/resize creates a data race -- possibly because an iterator
tries to read some of circular_buffer's state which is modified via
erase_first and/or resize. But that's just a guess. I did not check the
circular_buffer implementation too closely.

>> But the program did not work. The first thing that came to my
>> attention was that the checked iterators of circular_buffer don't
>> support this kind of multithreading as they register and unregister
>> themselves into a linked list without any synchronization.
> 
> I had exactly the same thoughts...

This was definitely an issue. But it's not the only one apparently.

Cheers!
sg