Re: [Boost-users] question about multi_array use
On Sat, Aug 8, 2009 at 4:51 PM, Alle Meije Wink<a.m.wink@gmail.com> wrote:
You could always just wrap up each of the possible dimensions in a struct that can handle the (de)serialization and everything too, perhaps use crtp to put identical functionality in a non-virtual base class.
Overmind, I guess compared to what I had in mind (define the data container of the image [template?] class as a multi_array with a dimensionality given as a size_t parameter), the CRTP struct you are talking about specifies all of the possible types. Is that correct?
If I understand it correctly then this does what I want, but does that mean that in the worst-case scenario, I have to write out the functionality for all different dimensionalities? If the parameterised template class idea is possible, that would save some typing!
I could see using fusion (mostly the "apply" construct) would make it generic to any number of dimensions. Boost.Fusion is amazing for such work, think MPL, but it works at runtime too.
I like the look of boost::Fusion in terms of advanced generic C++ programming, but in terms of finding a concise description of my image template class / class / data type, this may be overshooting the aim a bit. Plus I will need to read up on template metaprogramming! Coming back to the original idea: my images are stored as files which have a (on-disc 1-dimensional) sequence of intensities, and a header containing a number dims (dimensionality) and an array dim[8] (maximum of 8 dimensions). The problem that images can have different (but all built-in) data types can be circumvented with template programming, which is great compared to the old C-style casting to double. But talking about C-style, dealing with dimensions was possible in a fast way (in terms of both coding and runnning) with malloc and pointers (to pointers (to pointers)). Previously this is how I used to implement operations on images: casting everything to double, using pointers to get to image elements. I was hoping that by using templates to abstract from element type, and multi_array to cover different dimensionalities, it would be possible code image operations only once, for all types and dimensionalities. From what I understand, the curious recursion way matches most closely what I was looking for (and is still as close to generic programming as possible?). The scopira narray class http://www.scopira.org/api/narray_8h-source.html seems to do something similar. If there is another way to do this then I'm still very interested! Thanks, Alle Meije
On Aug 10, 8:30 am, Alle Meije Wink <a.m.w...@gmail.com> wrote:
On Sat, Aug 8, 2009 at 4:51 PM, Alle Meije Wink<a.m.w...@gmail.com> wrote: I like the look of boost::Fusion in terms of advanced generic C++ programming, but in terms of finding a concise description of my image template class / class / data type, this may be overshooting the aim a bit. Plus I will need to read up on template metaprogramming!
As someone who started using fusion recently, it really is remarkable. But it is a lot of work if you have never done metaprogramming before. The "C++ Template Metaprogramming" book is invaluable, even though it doesn't address fusion directly. One other (sub-optimal) method might be to fix a maximum number of dimensions that can be dealt with in the class and use it to initialize the multi_array: * When you construct an instance of the class with your "dimensions" loaded from the file, it could fill in the extents with the proper extents for the loaded data, and then put in a "1" for the other extents up to the maximum number of dimensions. * If you do this in the right order (i.e. put the single extents at the end rather than the beginning... or vice-versa depending on your storage ordering...) you might not have much of a performance hit since the strides might run through your real dimensions directly. * You can always address the multi_array indirectly by passing in a container of std::size_t indices, which you could pad with the appropriate number of "1"s depending on how many dimensions you are dealing with. * Or last, you could start recursively getting iterators from the multi_array to drop the dimension as many times as you have useless dimensions. * This all sounds like a pain, but it might not be that bad.
On Thu, Aug 13, 2009 at 6:36 AM, Alle Meije Wink<a.m.wink@gmail.com> wrote:
Dear OverMind
Hope I'm not making this thread too long, I've tried to search the archives but did not get further than browsing.
After looking at different options, your idea of wrapping the different dimensions/dimensionalities in a struct sounds like the best idea, and closest to the type of data and problems I'm dealing with.
Would you have a few more clues (maybe code, maybe URLs) as to how to shape this? I had a look at http://en.wikipedia.org/wiki/Curiously_recurring_template_pattern but that is a very broad description. As I understand it, the struct that encapsulates the possible dimensionalities and the CRTP class refer to the same solution? Or are they different things?
Thanks! Alle Meije
Date: Fri, 7 Aug 2009 18:09:11 -0600 From: OvermindDL1 <overminddl1@gmail.com> Subject: Re: [Boost-users] question about multi_array use
You could always just wrap up each of the possible dimensions in a struct that can handle the (de)serialization and everything too, perhaps use crtp to put identical functionality in a non-virtual base class.
On Fri, Aug 7, 2009 at 2:04 PM, Anton Daneyko<self@mezhaka.com> wrote: Hello,
I might be wrong, but it seems the answer is "no you can't do that", because the dimensionality is a template parameter multi_array<int, 2> and multi_array<int, 3> are different types. However I would like to hear some else' opinion as well.
Regards, Anton Daneyko.
On Fri, Aug 7, 2009 at 3:48 PM, Alle Meije Wink <a.m.wink@gmail.com> wrote:
Hello,
I would like to use the multi_array container to handle imaging data. The image data can be anything up to 8-dimensional (usually up to 4) and dimension information is found in records that accompany the binary data.
Would it be possible to create an image class that takes the dimensionality and extents list as parameters, so that images of all dimensionalities can be treated as one class?
For example, could the dimensionality be passed as a size_t argument?
What I understand from the multi_array paper is that the extents list would only be required whan the array is initialised.
Bit busy currently so no real-code to test with, but I might have pseudo-code. Hmm, I just (saved out first and) cut a large paragraph I was writing because I just came up with another idea. Why not use Boost.Variant stuffed with all possible types of matrices? A simple templated visitor could handle them all for any operations you want, and it optimizes out quite well. If you are worried about memory use you could wrap each matrix in the variant in a boost::recursive_variant so that it is dynamically allocated as necessary as well. Perhaps something like this: typedef boost::variant <boost::recursive_variant< multi_array<int, 2> > ,boost::recursive_variant< multi_array<int, 3> > ,boost::recursive_variant< multi_array<int, 4> > ,boost::recursive_variant< multi_array<int, 5> > ,boost::recursive_variant< multi_array<int, 6> > ,boost::recursive_variant< multi_array<int, 7> > ,boost::recursive_variant< multi_array<int, 8> > > multi_array_variant_2_to_8_t; And of course you could template it so you could specify the int type in a template parameter as well. Then make a variant for each of your operations, something along the lines of this for an equal test for example: class multi_array_variant_dotproduct_compare_equality : public boost::static_visitor<bool> { public: template <typename T, typename U> bool operator()( const T &, const U & ) const { return false; // different dimensions will always test unequal } template <typename T> bool operator()( const T & lhs, const T & rhs ) const { return lhs==rhs; } } And you could probably template it all (maybe using fusion) so most of of your tests could be generated for simple things like == and so forth. Then probably wrap it all up in a pretty class for ease of use, like this for a beginning shell: class multi_array_2to8 { protected: typedef boost::variant <multi_array<int, 2> ,boost::recursive_variant< multi_array<int, 3> > ,boost::recursive_variant< multi_array<int, 4> > ,boost::recursive_variant< multi_array<int, 5> > ,boost::recursive_variant< multi_array<int, 6> > ,boost::recursive_variant< multi_array<int, 7> > ,boost::recursive_variant< multi_array<int, 8> > > multi_array_variant_t; multi_array_variant_t _multi_array_variant; /* snip stuff */ protected: // visitors class multi_array_variant_dotproduct_compare_equality : public boost::static_visitor<bool> { public: template <typename T, typename U> bool operator()( const T &, const U & ) const { return false; // different dimensions will always be unequal } template <typename T> bool operator()( const T & lhs, const T & rhs ) const { return lhs==rhs; } } /* snip more stuff */ public: // op tests bool operator==(multi_array_2to8 const &rhs) const { return boost::apply_visitor (multi_array_variant_dotproduct_compare_equality() ,_multi_array_variant ,rhs._multi_array_variant ); } /* snip the rest of the stuff */ } For initially filling a vector would probably require a visitor using fusion, or 7 visitors, fusion's invoke sounds a lot easier though, and could simplify a few other visitors too, and use using fusion (and maybe crtp, depending on how you build it) would reduce the necessary code a lot as well. If you are still interested in my other way, I can try to remember what I was thinking and finish the paragraph I was doing, but that above seems better overall... For now though, sleep, and since I am tired enough that the screen is blurry, I make no remarks on my above coding quality, hopefully it works as-is, but consider it pseudo-code otherwise.
On Thu, Aug 13, 2009 at 7:14 AM, OvermindDL1<overminddl1@gmail.com> wrote:
On Thu, Aug 13, 2009 at 6:36 AM, Alle Meije Wink<a.m.wink@gmail.com> wrote:
Dear OverMind
Hope I'm not making this thread too long, I've tried to search the archives but did not get further than browsing.
After looking at different options, your idea of wrapping the different dimensions/dimensionalities in a struct sounds like the best idea, and closest to the type of data and problems I'm dealing with.
Would you have a few more clues (maybe code, maybe URLs) as to how to shape this? I had a look at http://en.wikipedia.org/wiki/Curiously_recurring_template_pattern but that is a very broad description. As I understand it, the struct that encapsulates the possible dimensionalities and the CRTP class refer to the same solution? Or are they different things?
Thanks! Alle Meije
Date: Fri, 7 Aug 2009 18:09:11 -0600 From: OvermindDL1 <overminddl1@gmail.com> Subject: Re: [Boost-users] question about multi_array use
You could always just wrap up each of the possible dimensions in a struct that can handle the (de)serialization and everything too, perhaps use crtp to put identical functionality in a non-virtual base class.
On Fri, Aug 7, 2009 at 2:04 PM, Anton Daneyko<self@mezhaka.com> wrote: Hello,
I might be wrong, but it seems the answer is "no you can't do that", because the dimensionality is a template parameter multi_array<int, 2> and multi_array<int, 3> are different types. However I would like to hear some else' opinion as well.
Regards, Anton Daneyko.
On Fri, Aug 7, 2009 at 3:48 PM, Alle Meije Wink <a.m.wink@gmail.com> wrote:
Hello,
I would like to use the multi_array container to handle imaging data. The image data can be anything up to 8-dimensional (usually up to 4) and dimension information is found in records that accompany the binary data.
Would it be possible to create an image class that takes the dimensionality and extents list as parameters, so that images of all dimensionalities can be treated as one class?
For example, could the dimensionality be passed as a size_t argument?
What I understand from the multi_array paper is that the extents list would only be required whan the array is initialised.
Bit busy currently so no real-code to test with, but I might have pseudo-code.
Hmm, I just (saved out first and) cut a large paragraph I was writing because I just came up with another idea.
Why not use Boost.Variant stuffed with all possible types of matrices? A simple templated visitor could handle them all for any operations you want, and it optimizes out quite well. If you are worried about memory use you could wrap each matrix in the variant in a boost::recursive_variant so that it is dynamically allocated as necessary as well. Perhaps something like this:
typedef boost::variant <boost::recursive_variant< multi_array<int, 2> > ,boost::recursive_variant< multi_array<int, 3> > ,boost::recursive_variant< multi_array<int, 4> > ,boost::recursive_variant< multi_array<int, 5> > ,boost::recursive_variant< multi_array<int, 6> > ,boost::recursive_variant< multi_array<int, 7> > ,boost::recursive_variant< multi_array<int, 8> > > multi_array_variant_2_to_8_t; And of course you could template it so you could specify the int type in a template parameter as well.
Then make a variant for each of your operations, something along the lines of this for an equal test for example: class multi_array_variant_dotproduct_compare_equality : public boost::static_visitor<bool> { public: template <typename T, typename U> bool operator()( const T &, const U & ) const { return false; // different dimensions will always test unequal }
template <typename T> bool operator()( const T & lhs, const T & rhs ) const { return lhs==rhs; } }
And you could probably template it all (maybe using fusion) so most of of your tests could be generated for simple things like == and so forth.
Then probably wrap it all up in a pretty class for ease of use, like this for a beginning shell: class multi_array_2to8 { protected: typedef boost::variant <multi_array<int, 2> ,boost::recursive_variant< multi_array<int, 3> > ,boost::recursive_variant< multi_array<int, 4> > ,boost::recursive_variant< multi_array<int, 5> > ,boost::recursive_variant< multi_array<int, 6> > ,boost::recursive_variant< multi_array<int, 7> > ,boost::recursive_variant< multi_array<int, 8> > > multi_array_variant_t;
multi_array_variant_t _multi_array_variant;
/* snip stuff */
protected: // visitors class multi_array_variant_dotproduct_compare_equality : public boost::static_visitor<bool> { public: template <typename T, typename U> bool operator()( const T &, const U & ) const { return false; // different dimensions will always be unequal }
template <typename T> bool operator()( const T & lhs, const T & rhs ) const { return lhs==rhs; } }
/* snip more stuff */
public: // op tests bool operator==(multi_array_2to8 const &rhs) const { return boost::apply_visitor (multi_array_variant_dotproduct_compare_equality() ,_multi_array_variant ,rhs._multi_array_variant ); }
/* snip the rest of the stuff */ }
For initially filling a vector would probably require a visitor using fusion, or 7 visitors, fusion's invoke sounds a lot easier though, and could simplify a few other visitors too, and use using fusion (and maybe crtp, depending on how you build it) would reduce the necessary code a lot as well.
If you are still interested in my other way, I can try to remember what I was thinking and finish the paragraph I was doing, but that above seems better overall...
For now though, sleep, and since I am tired enough that the screen is blurry, I make no remarks on my above coding quality, hopefully it works as-is, but consider it pseudo-code otherwise.
Er, do not use the recursive_variant parts, I just looked at the multi-array library and it is fully dynamic already, so yea, take out the recursive_variant parts and use it without those: typedef boost::variant <multi_array<int, 2> ,multi_array<int, 3> ,multi_array<int, 4> ,multi_array<int, 5> ,multi_array<int, 6> ,multi_array<int, 7> ,multi_array<int, 8> > multi_array_variant_t;
participants (3)
-
Alle Meije Wink -
Jesse Perla -
OvermindDL1