Subject: Re: [boost] interest in structure of arrays container?
From: Andreas Schäfer (gentryx_at_[hidden])
Date: 2016-10-16 06:59:26
On 09:59 Sun 16 Oct , degski wrote:
> In your example, which I think is flawed as an example, you create in
> memory a 20+ GB data structure (hope I got the maths right).
300M * 4 * 4B = 2.4GB. IMHO not too large. Even 20GB isn't that much
> This data structure optimizes one operation (calculating the average) at
> the cost of pessimising almost any other operation (relocation if the
> vector needs to grow, insert, delete, push_back etc are all done 4 times in
> your example)
Keep in mind that this is a sketch of what he would like to use, not a
polished solution. The point is: with a Struct-of-Arrays (SoA) memory
layout you can implement kernels operating on that data that can be
vectorized. Depending on your CPU, vectorization can yield you a
speedup of 2x - 8x. If implemented properly then insert, delete,
push_back etc. can also be efficient for SoA.
> while iterating over the records will land you right into
Depends. If your access pattern is very random and your algorithm
needs to access every member of an object, then a std::vector might be
faster. But if your algorithm has some (piecewise) continuity, then
the SoA structure will benefit just as well from the CPU's caches.
-- ========================================================== Andreas Schäfer HPC and Supercomputing Institute for Multiscale Simulation Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany +49 9131 85-20866 PGP/GPG key via keyserver http://www.libgeodecomp.org ========================================================== (\___/) (+'.'+) (")_(") This is Bunny. Copy and paste Bunny into your signature to help him gain world domination!
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk