Re: [boost] Interest in a static-sized version of cvalarray

From: Michael Tegtmeyer (tegtmeye_at_[hidden])
Date: 2007-06-05 12:36:04

Stjepan,

I appreciate you taking the time to have a look at the code.

> I would really like to see this library in boost, would you like any
> help getting it ready for submission?  I have recently learned how to
> use bjam, do boost-style docs etc, and have time to lend a hand.  If
> we wanted to work together on preparing this, it would be good to put
> the code in the boost sandbox so we can both access it via subversion.

Thanks, I am interested in whatever help you could offer. I have little 
experience in bjam nor the boost document style.

> use the boost general license
Not a big deal, see COPYING file.

> code readability and maintainability - there is a lot of code
> repetition in the .h file, also it could use to be chunked up into
> smaller files (it is really hard to look at a ~5000 line header file).
> Did you use a code generation script to make that file?  It could
> benefit from using macros for all the similar functions, which can be
> readily made from the code generation script if you used one.

I didn't use a code generator but most is copy-paste, At one point I had 
macros to generate each individual case but debugging (use code, not 
library code) became a pain. ie it was easier to track down a compiler 
error when it said  "no operator in expression<lots and lots of stuff> on 
line 42 which said __positive::T operator[](std::size_t n) const {return 
+(value[n]);} rather than MAKE_EXPRESSION_TIMES(...) which was a top-level 
macro and the real problem was missing operator + nested 8 layers. (real 
problem we had). So since the library has been stable for a few years, we 
got rid of the macros to aid debugging it's use.

I think that I have a better understanding of what you are looking for but 
let me get the following out of the way:

I'm not sure that a general purpose container is the best use for this 
class. cvalarray is meant to mimic the semantics and purpose of 
std::valarray. Which is specifically designed for numeric values. I think 
that if the purpose is general, then so should the interface, ie not mimic 
std::valarray. One could claim that the incorrect use of boost::array and 
by implication a general-purpose cvalarray is not our problem but IMHO, 
libraries should be easy to use correctly and hard to use incorrectly. 
Arguably, concepts solves some of these problems but in this case, I 
believe the better behavior is limiting the scope to numeric operations as 
does the well documented std::valarray-especially since the interface is 
identical.

Additionally, the next step for cvalarray here is for it to automatically 
use SIMD instructions to further speed up the numeric operations. Again 
making its use general purpose will unnecessarily complicate things.

On a side and somewhat historical note, if cvalarray morphs into anything 
else, one of the issues here for the adoption is the adding dependancies. 
Currently,  the libraries is one file, self-contained, and does not rely 
on anything outside of the language. That is very important in many 
peoples view. As it is now, one can drop the file into their own project 
and it is available everywhere there is a standard compliant compiler. 
Boost, unfortunately is still not everywhere so unless adding dependancies 
adds significant functionality, I'd like to avoid them. I also thought 
about submitting cvalarray to the c++ standard at one time. For it to ever 
be adopted there, it cannot have dependancies outside of the language.

All of that said, I believe that your general-purpose need/desires are 
legitimate. Therefore, I think that it is worth working for a 
general-purpose solution. I have may ideas towards this end but one of the 
biggest ones is the use of expressions. Currently, both std::valarray and 
cvalarray use expressions solely as an implementation detail allowed to 
under the standard:

26.3.1 Header<valarray> synopsis:

3 Any function returning avalarray<T>is permitted to return an object of 
another type, provided all the
const member functions ofvalarray<T>are also applicable to this type. This 
return type shall not add
more than two levels of template nesting over the most deeply nested 
argument type.254)

4 Implementations introducing such replacement types shall provide 
additional functions and operators as
follows:

for every function taking aconst valarray<T>&, identical functions taking 
the replacement types  shall be added;

for every function taking twoconst valarray<T>&arguments, identical 
functions taking every combination of const valarray<T>& and replacement 
types shall be added.

5 In particular, an implementation shall allow avalarray<T>to be 
constructed from such replacement  types and shall allow assignments and 
computed assignments of such types to valarray<T>, slice_array<T>, 
gslice_array<T>, mask_array<T> and indirect_array<T> objects.

However this is considered an implementation detail and whatever internal 
type is not officially exposed. I'd like to work toward an exposed 
expression type for general purpose use in a new general purpose 
container: For example in the case of std::valarray

std::valarray<float> f1, f2, f3;	// assume filled with 3 
interesting numbers

f1 = f2 + f3;	// all ok f1[i] = f2[i] + f3[i] for all n

template<typename T>
std::valarray<T> cross_product(const std::valarray<T> &lhs, const 
std::valarray<T> &rhs)
{
         std::valarray<T> temp;
         // do cross product
         return temp;
}

// unnecessary temporary
f1 = cross_product(f2,f3);

Notice that the above creates an unnecessary temporary that could easily 
be solved with an expression type. But since the expression type is not 
formally exposed, there is no way to write portable code to take advantage 
of it. I think in a new, general-purpose container with a general-purpose 
interface, we could have an exposed expression type that uses a boost 
concept that ensures that as long as (for example) the contained type has 
a valid [] operator, any additional functionality can take advantage of 
it. ie

template<typename T>
class expression_container ... {
         public:
                 typedef ??? expression_type;
};

then

expression_container<std::string> foo("Hello1","Hello2","Hello3");
expression_container<std::string> foo("World1","World2","World3");

Then:

template<typename T>
expression_container<T> join(expression_container<T> &lhs, 
expression_container<T> &rhs)
{
         expression_container<T> temp(lhs);

         for(std::size_t i=0; i<lhs.size(); ++i)
                 temp[i] += rhs[i];

         return temp;
}

And

template<typename T>
expression_container<T>::expression_type join(const 
expression_container<T>::expression_type &lhs, 
expression_container<T>::expression_type &rhs)
{
         return expression_container<T>::expression_type(lhs,rhs,op_plus)
}

expression_container<std::string> f1, f2, f3;

template<typename T>
void foo(expression_container<T>  &val) {...}

f1 = join(f2,f3); // no temporary
foo(join(f2,f3)); //makes temporary because it must

Thoughts?
Mike