I looked at the code and it appears flawed. What you want to do is to create an allocator that allocates memory based on the opencv CvMat type.

Look at storage.hpp and look at the class unbounded_array. This is the class that you need to clone and implement. It makes some assumptions on the allocator operations so you just can't define a new allocator.

Then you can define your matrix something like:

  template <class T >
class  CWrapCvMat: public boost::numeric::ublas::matrix< T, row_major , cvmat_array<T>  >
{
 ...
}

cvmat_array is where all the work is at (which is the equivalent of the unbounded_array class using CvMat). In this you need to handle allocation/dellation, iteration and the operator[](int), and a few other misc stuff to be compatabile.

I haven't looked in detail in cvmat datatype, but I hope it doesn't work like the IplImage data type, where each row is aligned on a boundary. So for IplImages you can't have 3x3 image of uchar, since a row will be aligned on a 4 byte boundary. Then you will have more work, because the indexing and iteration over this will need to account for these holes that occur for row alignment.

I have a wrapper class that takes an IplImage and presents a ublas::matrix facade to it, since the opencv indexing operations are poor.  This however has the restriction that the IplImage doesn't have holes, i.e. width * pelSize == widthStep