Re: [Performance of circular_buffer vs Queue][Multithreading may be broken]

From: Gavin Lambert (boost_at_[hidden])
Date: 2020-04-02 23:16:10

On 3/04/2020 08:41, Jan Hafer wrote:
> On 02.04.20 20:59, Mathias Gaunard wrote:
>> So you're saying that circular_buffer is slower on a given thread when
>> other threads are accessing their own circular_buffer in parallel?
>> That sounds unlikely to be circular buffer's fault.
> 
> Yes and I dont know quite the reason for it.
> My Threads know their id to access a file-global data structure
> containing their queue/circular buffer. They start another after in a
> thread-safe way and exit on emptying the queue/circular buffer.

That sounds like you're allocating a single array of circular_buffers 
and then accessing them from different threads.

That's basically the worst possible thing to do; as Mathias was saying, 
that will end up sharing cache lines between different cores and your 
performance will tank.

At minimum, you should embed the circular_buffer into another struct 
that has sizeof() >= std::hardware_destructive_interference_size, and 
make an array of that.

But better still, embed the circular_buffer into your processing classes 
and don't have arrays of them at all.

(If you're pre-C++17 and don't have 
std::hardware_destructive_interference_size, then using 64 works for 
most modern platforms.)

Ideally, the circular_buffer implementation itself should also separate 
all internal producer-thread members and consumer-thread members by 
std::hardware_destructive_interference_size and try very hard to not 
cross over.  (Here the main thing that matters is write accesses.)

If you want to try using a more modern circular buffer that gets this 
correct, have a look at Boost.Lockfree's spsc_queue.

https://www.boost.org/doc/libs/1_72_0/doc/html/boost/lockfree/spsc_queue.html

(While you're there, there's also an MPMC queue as well.  Note that a 
lockfree queue will tend to be slower than std::queue in uncontended 
benchmarks, but the avoidance of locks can be superior in highly 
contended or other specialised scenarios.)