Multi_array: A "proper" iterator over a N>1 Multi_array?
I'm using multi_array types as containers for some things I'm working on, and the number of dimensions can get quite large - often over 7, sometimes as high as 11. I'd like to do things such as initialize these multi_arrays, access all elements within them, and other such 'reasonable' operations. Thing is, the only iterator interface to multi_array returns a reference to the nested containers. Thus, to iterate over all sub- elements, I'm forced to have 7-11 nested for-loops to iterate over the entire thing. Is there an existent means of simply iterating over the entire set of data in the multi_array? Has anyone implemented such a thing, or is such a thing something to be desired within boost::multi_array? It seems like such a iterator would be especially useful if also implemented/supported within subgroups, allowing the user to define sub-ranges of the multi_array, then iterate over each element therein, without having to worry about dimensionality, et cetera. Any thoughts, ideas, or pointers to how to easily (without tons of code) iterate over an 11-dimensioned object is appreciated. In conclusion then, I've got two problems. One is "How can I simplify my each-element access to a high-dimensional multi_array", and the second is "Should there be a direct-to-element iterator in multi_array, and does the documentation adequately address the issue of whether or not an iterator points to elements directly or to a sub- graph?" Thanks for your time and troubles, - Greg Link
I guess what you want is origin(). Iterate from origin() to origin+num_elements Antonio On 4/25/06, Greg Link <link@cse.psu.edu> wrote:
I'm using multi_array types as containers for some things I'm working on, and the number of dimensions can get quite large - often over 7, sometimes as high as 11. I'd like to do things such as initialize these multi_arrays, access all elements within them, and other such 'reasonable' operations.
Thing is, the only iterator interface to multi_array returns a reference to the nested containers. Thus, to iterate over all sub- elements, I'm forced to have 7-11 nested for-loops to iterate over the entire thing.
Is there an existent means of simply iterating over the entire set of data in the multi_array? Has anyone implemented such a thing, or is such a thing something to be desired within boost::multi_array? It seems like such a iterator would be especially useful if also implemented/supported within subgroups, allowing the user to define sub-ranges of the multi_array, then iterate over each element therein, without having to worry about dimensionality, et cetera.
Any thoughts, ideas, or pointers to how to easily (without tons of code) iterate over an 11-dimensioned object is appreciated.
In conclusion then, I've got two problems. One is "How can I simplify my each-element access to a high-dimensional multi_array", and the second is "Should there be a direct-to-element iterator in multi_array, and does the documentation adequately address the issue of whether or not an iterator points to elements directly or to a sub- graph?"
Thanks for your time and troubles,
- Greg Link
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Tue, 25 Apr 2006 20:15:37 -0300, Antonio Piccolboni <piccolbo@gmail.com> wrote:
I guess what you want is origin(). Iterate from origin() to origin+num_elements
I think you meant data(). I also think that does not work if you have a multiarray that is a subset of another, because the elements are not contiguous. I also would like to iterate easily over all elements of a multiarray. Maybe a foreach-like function is easier to write than a general iterator. Bruno
I would agree with you there - a 'foreach' would do what I need as well, but as I'm not 100% familiar with the way libraries such as boost are implemented, I worry that making your own control structure is much more difficult than making an accessor/modifier pair. I can't even imagine the syntax needed to convert the following into a theoretical boost-defined 'foreach' double previous = 0; double accumulator = 0; for(iterator_t myIterator = m_array.data(); myIterator != m_array.end (); myIterator++) { accumulator += (*myIterator); (*myIterator) *= previous accumulator = (previous > 7) ? (do_function(accumulator)) : (0); } Sure, it's a contrived example, but I just don't know how a primarily header library such as boost (or any library for that matter) would be able to define a foreach(m_array) { } in a syntactically and compiler-friendly manner. That doesn't mean, however, that I'm still not interested in the possibility of using .data() and iterating, I'm just concerned that if the data in the container isn't sequential, I'm violating some law. I know (from testing) that making an iterator myIterator = m_array.begin(); and incrementing it only goes as high as the size of the highest dimension before it hits m_array.end(), so I'm concerned that doing myIterator = m_array.data(); will end at the same point (or just walk off into nowheresville). Don't have my test code in front of me until tomorrow, where I'll test for sure. - Greg On Apr 25, 2006, at 9:02 PM, Bruno Martínez wrote:
On Tue, 25 Apr 2006 20:15:37 -0300, Antonio Piccolboni <piccolbo@gmail.com> wrote:
I guess what you want is origin(). Iterate from origin() to origin+num_elements
I think you meant data(). I also think that does not work if you have a multiarray that is a subset of another, because the elements are not contiguous.
I also would like to iterate easily over all elements of a multiarray. Maybe a foreach-like function is easier to write than a general iterator.
Bruno
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Tue, 25 Apr 2006 23:11:26 -0300, Greg Link <link@cse.psu.edu> wrote:
I would agree with you there - a 'foreach' would do what I need as well, but as I'm not 100% familiar with the way libraries such as boost are implemented, I worry that making your own control structure is much more difficult than making an accessor/modifier pair. I can't even imagine the syntax needed to convert the following into a theoretical boost-defined 'foreach'
I didn't mean new sintax, but a std::for_each work-alike.
double previous = 0; double accumulator = 0; for(iterator_t myIterator = m_array.data(); myIterator != m_array.end (); myIterator++) { accumulator += (*myIterator); (*myIterator) *= previous accumulator = (previous > 7) ? (do_function(accumulator)) : (0); }
You can do that with Boost.Lambda: #include <boost/lambda/lambda.hpp> #include <boost/lambda/if.hpp> #include <boost/lambda/bind.hpp> #include <algorithm> int main() { double previous = 0; double accumulator = 0; double d[3] = {1, 2, 3}; using boost::lambda::_1; using boost::lambda::if_then_else_return; using boost::lambda::var; using boost::lambda::bind; std::for_each(&d[0], &d[3], ( accumulator += _1, _1 *= previous, var(accumulator) = if_then_else_return(var(previous) > 7, bind(&do_function, accumulator), 0))); } I spend some time today writing a std::for_each-like function for multiarrays. This works correctly for subviews, but isn't as convenient as an iterator. You can't iterate through two multiarrays at the same time, for example. I only tested with MSVC8, so there may be some typenames missing. Here it is: template<class MA, class F, int dim> struct for_each_impl { F f; for_each_impl(F f) : f(f) {} void operator()(MA& ma) { std::for_each(ma.begin(), ma.end(), for_each_impl<typename MA::reference, F, dim-1>(f)); } }; template<class Ref, class F> struct for_each_impl<Ref, F, 0> { F f; for_each_impl(F f) : f(f) {} void operator()(Ref r) { f(r); } }; template<class MA, class F> void for_each(MA& ma, F f) { for_each_impl<MA, F, MA::dimensionality> impl(f); impl(ma); } And a test: typedef boost::multi_array<double, 3> array_type; typedef array_type::index index; array_type myarray(boost::extents[3][4][2]); typedef array_type::index_range range; array_type::array_view<3>::type myview = myarray[ boost::indices[range(0,2)][range(1,3)][range(0,4,2)] ]; for_each(myview, _1 = 56); Hope that helps. Bruno
Upon reflection I realized that a) My code was broken b) origin() returns an element* and not any smarter iterator, so that layout is important c) I would propose as a solution a call that returns an iterator pointing to the first element of a multi array and when incremented goes through all the elements one by one all the way to the end something like for (iter=marray.element_begin(); iter !=marray.element_end(); iter ++) { *iter+=1; } would increment all the elements of the multi_array marray by 1. This should work for views and independent of number of dimensions and layout. I think it would be easily implementable as a skip iterator where the increment operator increments a *element by a number that's the product of the appropriate strides. Dereferencing semantics is the same as element*. Now everything rests of what "appropriate" means, but I guess whoever implemented how to increment any index in a view knows how to pull this off as well -- that is Ron, any comments? I think this would be a powerful extension. Once this is in place there would be no need for specialized algorithms -- for_each, transform and even binary_search would do exactly what people want them to do, like in for_each(marray.element_begin(); marray.element_end(), _1++) equivalent to the loop above. Another possibility that reminds me of the statistical language R is that we could have a call as_vector that flattens an multi array into a 1D view. Then begin_element above would be equivalent to begin() on as_vector(marray). Antonio On 4/25/06, Bruno Martínez <br1@internet.com.uy> wrote:
On Tue, 25 Apr 2006 23:11:26 -0300, Greg Link <link@cse.psu.edu> wrote:
I would agree with you there - a 'foreach' would do what I need as well, but as I'm not 100% familiar with the way libraries such as boost are implemented, I worry that making your own control structure is much more difficult than making an accessor/modifier pair. I can't even imagine the syntax needed to convert the following into a theoretical boost-defined 'foreach'
I didn't mean new sintax, but a std::for_each work-alike.
double previous = 0; double accumulator = 0; for(iterator_t myIterator = m_array.data(); myIterator != m_array.end (); myIterator++) { accumulator += (*myIterator); (*myIterator) *= previous accumulator = (previous > 7) ? (do_function(accumulator)) : (0); }
You can do that with Boost.Lambda:
#include <boost/lambda/lambda.hpp> #include <boost/lambda/if.hpp> #include <boost/lambda/bind.hpp> #include <algorithm>
int main() { double previous = 0; double accumulator = 0; double d[3] = {1, 2, 3};
using boost::lambda::_1; using boost::lambda::if_then_else_return; using boost::lambda::var; using boost::lambda::bind; std::for_each(&d[0], &d[3], ( accumulator += _1, _1 *= previous, var(accumulator) = if_then_else_return(var(previous) > 7, bind(&do_function, accumulator), 0))); }
I spend some time today writing a std::for_each-like function for multiarrays. This works correctly for subviews, but isn't as convenient as an iterator. You can't iterate through two multiarrays at the same time, for example. I only tested with MSVC8, so there may be some typenames missing. Here it is:
template<class MA, class F, int dim> struct for_each_impl { F f; for_each_impl(F f) : f(f) {}
void operator()(MA& ma) { std::for_each(ma.begin(), ma.end(), for_each_impl<typename MA::reference, F, dim-1>(f)); }
};
template<class Ref, class F> struct for_each_impl<Ref, F, 0> { F f; for_each_impl(F f) : f(f) {}
void operator()(Ref r) { f(r); } };
template<class MA, class F> void for_each(MA& ma, F f) { for_each_impl<MA, F, MA::dimensionality> impl(f); impl(ma); }
And a test:
typedef boost::multi_array<double, 3> array_type; typedef array_type::index index; array_type myarray(boost::extents[3][4][2]); typedef array_type::index_range range; array_type::array_view<3>::type myview = myarray[ boost::indices[range(0,2)][range(1,3)][range(0,4,2)] ]; for_each(myview, _1 = 56);
Hope that helps.
Bruno
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (3)
-
Antonio Piccolboni -
Bruno Martínez -
Greg Link