Re: [boost] interest in structure of arrays container?

Subject: Re: [boost] interest in structure of arrays container?
From: Oswin Krause (Oswin.Krause_at_[hidden])
Date: 2016-10-15 06:15:16

On 2016-10-15 10:14, Michael Marcin wrote:
> On 10/14/2016 11:32 PM, degski wrote:
>> On 15 October 2016 at 05:43, Michael Marcin <mike.marcin_at_[hidden]> 
>> wrote:
>> 
>>> Does this already exist somewhere?
>>> 
>> 
>> Take a look at Boost.DoubleEnded 1 <http://erenon.hu/double_ended/> 
>> and see
>> whether boost::double_ended::batch_deque
>> <http://erenon.hu/double_ended/double_ended/design_rationale.html#double_ended.design_rationale.batch_deque>
>> fills all of your requirements.
>> 
> 
> 
> Sorry I think I wasn't clear.
> 
> Let me back up a bit and give some examples.
> 
> Arrays of Structures is the normal programming model.
> 
> Let's take a toy example, say I'm looking at some census data and have
> a structure that looks like:
> 
> struct citizen_t {
>     string first_name;
>     string last_name;
>     int salary;
>     int age;
> };
> 
> Then the Array of Structures container would be:
> vector<citizen_t> aos_citizens;
> 
> The Structure of Arrays version of the same data would look like:
> 
> struct soa_citizens_t {
>     vector<string> first_names;
>     vector<string> last_names;
>     vector<int> salary;
>     vector<int> age;
> };
> 
> soa_citizens_t soa_citizens;

Hi,

We implemented something similar for our software. Our usecase was: 
"when we have a batch of inputs to process, what is the most efficient 
way to store them such that we can process them with a fast batch 
algorithm". For example a batch of vectors is a matrix with vectors 
stored row-wise (and sparse vectors are a sparse matrix). Similarly, if 
we have pairs of inputs and outputs a batch of those pairs are pairs of 
batches, tuples of batches, etc.

The problem starts when we look at "how do i get an element from a 
batch?". A matrix-row is not the same as a vector. Moreover, it is 
unnatural to see a matrix as "batch of rows" (e.g. why not "batch of 
columns"?) and thus we likely do not have a way to get a row using op[] 
or iterate over its rows. It becomes only more complicated when looking 
at a pair of vectors example (a single element of that would be a pair 
of two matrix-rows) etc.

Even after some working on this, we have not really found a stable and 
"good" solution that does not involve massive amounts of preprocessor 
boilerplate and massive use of traits classes.

I would like to see your take on this, maybe this could give us some way 
to improve our work!

Best,
Oswin