[multi] A successor to Boost.MultiArray

From: Alfredo Correa (alfredo.correa_at_[hidden])
Date: 2024-09-26 05:28:06

On Wed, Sep 25, 2024 at 9:37 PM Andrzej Krzemienski <akrzemi1_at_[hidden]>
wrote:

>
>
> czw., 26 wrz 2024 o 04:25 Alfredo Correa <alfredo.correa_at_[hidden]>
> napisał(a):
>
>> Hi Andrzej,
>>
>> Thanks for the continued conversation.
>>
>>
>>
>>> From: Andrzej Krzemienski <akrzemi1_at_[hidden]>
>>> Subject: [boost] [multi] A successor to Boost.MultiArray
>>>
>>
>> Don't get me wrong, BMA was a tour-de-force at the time.
>> It even spearheaded C++ concept checking in C++98, which I have delayed
>> integrating into the library until C++20 Concepts.
>>
>
> C++98 compatibility may be a good reason. Other than that, the lack of
> responsiveness could be an indication that a library needs a new
> maintainer, and then you could replace the implementation. But it would
> surely not be C++98-compliant.
>

There are many precedents of Boost libraries that provide new
implementations of libraries with similar.
Boost.variant2 and Boost.Parser are recent examples, there might be others.

>
>>
>> * The same questions, "is it useful at all", apply also to
>>> Boost.MultiArray.
>>>
>>
>> BMA is useful in principle and was a great leap forward then.
>> In practice, it is unfortunately almost unusable.
>> You can tour Stackoverflow questions on #boost-multi-array and see all
>> its pain points.
>>
>
> I am suggesting this still in the context of documenting a solid
> motivation for why Multi is useful. If the users of BMA have their story to
> tell, you could use it as well for your motivation.
> But if you are saying BMA is unusable, and maybe unused, then there may
> not be much to take. This could also be an indication that Artyom is
> correct, and that the need for a container alone may be scarce.
>

Yes, as I said, if a container is not enough I am okay with that.
I claim that this is not just a container; it is also a generator of views
(part of what functions in ranges like drop, take, subrange, chunk_by, etc.
provide), but all seamlessly integrated in the same underlying data
structure.
And it is a whole suite of features that worm together to 1) implement
algorithms, 2) interface efficiently and nicely with tens of existing C
libraries that work with this datastructure.

I think this is valuable but others might disagree strongly; I accept that.

My *dream* is that a well designed interface for this classic data
structure will initiate a golden age of multidimensional algorithms that
are a generalization of STL (which BTW is also criticized for being
one-dimensional centric).

Of course, this is a monumental task; that eventually I could integrate it
into Multi.
Putting a few selected exemplars of such multidimensional algorithms is as
much as I can hope for since I am working by myself.
(In place of that I decided to provide the adaptors.)

I also accept that generic programming might not be fashionable in this
decade.
In which case the product wouldn't be as valuable.

>>
>> I don't know how you conclude this is for small-sized data?
>> Do you mean that vector<vector<...>> would be ok for large-size data in
>> contrast?
>> I don't think so.
>>
>
> No. Sorry, I brought up two separate points. One point is that if you need
> a multidimensional array Multi is always superior to vector<vector<...>>.
> So if you happen to use a nested vector in your program, you can optimize
> and improve your program cheaply by employing Multi.
>

Sure, that can be used for teaching of course.
Vector<vector is what most beginners reach for after learning vector and
STL and finding out that multidimensional arrays are not available in the
language, and that there is a wild jungle of libraries for that, and none
of which resemble vector or STL at all.

> The other, separate, point is in the context of a critique that a
> general-purpose multidimensional array container cannot compete with
> tailored frameworks for huge dataset processing that require domain-based
> optimizations.
>

yes, that is not the intention of Multi.
I don't want to compete with frameworks.
I feel that, together with STL algorithms and a few adaptors, Multi is
already very powerful.

Even if true, Multi has its use for small-sized data. The user experience
> would be: the following. I have an idea to solve a problem using a
> multidimensional array. First I prototype a solution, and for this I use
> Multi.
>

I won't discourage this kind of use.
I am convinced that the library can do prototyping and beyond.

Then the prototype demonstrates that my solution will work, I need to
> implement a production ready, super-optimized version, and for this I use a
> framework, like OpenCV.
>

That can be an option, another option would be to add an ad-hoc OpenCV
adaptor (following the examples in the adaptors directory) so that one
doesn't lose the nice things of Multi.

Here Multi still plays an important role in enabling prototyping.
>

sure.

Boost.Spirit has such an advertising technique ("use this library for small
> and medium parsers").
>
>>
I would agree if you mean “small” in the sense of simple, consice code.
I don't think there is a penalty on using Multi for storing large data.
It is not that arrays become “slower” when they are big. (well that depends
on caches and access patterns but that is a different story)
The data structure is equally simple for large and small arrays.

>>
>>> As a side note, when comparing Multi with std::mdspan. The latter
>>> requires
>>> C++23, while the former, only C++17. So, when someone has C++17, mdspan
>>> is
>>> not an option.
>>
>>
>> yes
>>
>>
>>> BTW, why do you require C++17?
>>>
>>
>> - The main reason was for ease of development:
>> - implementing allocator propagation without if-constexpr is a nightmare.
>> - c++17 provides polymorphic allocators, and I wanted to be compatible
>> with them (at the time, I didn't know about __has_include, it was much
>> simpler to just use C++17)
>> - If I remember correctly, I could use some constexpr algorithms and make
>> the arrays fully constexpr.
>>
>
>> These are not hard reasons to require C++17.
>> I can port it to C++14 as soon as any user asks for it, but it hasn't
>> happened yet.
>> In fact, Vinnie Falco promised a gazillion dollars from Microsoft to port
>> it back to C++14. ;P (
>> https://cpplang.slack.com/archives/C27KZLB0X/p1682786897070269)
>>
>
> This is not a request. Just an observation that it would make the library
> even more competitive.
>

yes, so far the question has been hypothetical, the library can be ported
back to C++14 when someone (or Bill Gates) ask for it.

Thanks
Alfredo

> Regards,
> &rzej;
>
> Thank you,
>> Alfredo
>>
>>
>>