|
|
Ublas : |
Subject: Re: [ublas] Help with raw pointers and matrix/vector interoperability with other libraries through casts
From: Jesse Perla (jesseperla_at_[hidden])
Date: 2008-09-15 15:24:12
Thanks for all your help Vardan,
[[You have certainly figured out by now that I am a humble library user and
not really capable of writing adaptors/etc. that require detailed knowledge
of this stuff or reading the source code. So I am mostly looking for
patterns, existing code (and hacks if absolutely necessary) to do basic
things.]]
On Bounded Array/TinyVector:
Thanks for the pointer on the bounded array. I was confused because in:
http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Frequently_Asked_Questions_Using_UBLAS
it mentions that people should consider tvmet. If this is no longer the
common wisdom and I can use ublas for everything, then so much the better.
As for interoperability:=================
On STL vector:
Yes, I was confused and must have been looking at another library at the
time. I think I see what you mean about creating an adapter, but this is
beyond my level of competence. Sounds like I am stuck with just copying
data between then for now? As for the docs on this, effective ublas said
"you could" always use iterators to copy the data for vectors. But I also
found various discusions about the adaptors which added to my confusion.
On C arrays:
Yes, I made a mistake with int** instead of int* and realized it after I hit
send. I was trying different permutations and was seeing if storage was
non-linear by chance. So I could (theoretically, for dense row major) just
pass in: &(A2.data()[0]) to any C-style function that accepts a matrix? I
know it is unsafe, but as a worst case this would work (especially for
functions that have a matrix as IN)?
Coming from the other direction, if a library sends back a raw array/matrix,
it sounds like I need to copy the data by using the ublas iterators? No
simple hack?
On Other Libraries and existing bindings:
Yes, I am using bindings for lapack for linear algebra. However, as soon as
you use ublas for small vectors/matrices you end up with a bunch of
interoperability issues with the chaos surrounding vector/matrices in C++.
May ublas triumph over all.
Also, for most other libraries I would be interested in using, you end up
with either a matrix/vector input or a matrix/vector output. Rarely INOUT.
So safety worries me, but a bunch of swaps between the class managing the
data isn't necessary.
Initialization ==============
Especially when using ublas for small arrays, matrices, this comes up all of
the time. I still haven't found a good sample of how to do this, but if
there isn't then I will begrudgingly write a function to do it from a
C-style matrix?
You are right that I am missing chunks of C++/STL/UBLAS. But I have indeed
tried to read the docs/effective ublas/etc.
I find these docs to be excellent and they cover most of the simple things a
library user needs. But if the intention is to make this library accessible
for non-experts as a general purpose matrix library, then a few things are
still missing. Primarily: Docs on how to interoperate between all different
vector/matrix implementations and how to initialize and use the library as a
small matrix library are a little sparse. You may think that most of my
questions are covered in the mailing lists and those other docs, but the
answers were unclear for my level and were often mixed in with details for
the developers of ublas. I would guess that many users coming from
matlab/fortran will be at about my level of incompetence. After I have gone
through the process of getting aquainted, I will try to add a bunch of usage
patterns for how matlab/fortran programmers so that others can make a smooth
transition.
As for docs, I have read the wiki, effective ublas,
http://www.guwi17.de/ublas/examples/ and
http://www.cs.colostate.edu/~nate/tutorial/tutorial.html Am I missing
anything else?
Thanks again,
Jesse
On Mon, Sep 15, 2008 at 1:32 PM, Vardan Akopian <vakopian_at_[hidden]> wrote:
> Jesse,
>
> I think there is a lot of confusion in your understand of ublas, but
> also STL and c++ in general. I'll try some short answers to your
> questions.
>
> On Sat, Sep 13, 2008 at 12:32 PM, Jesse Perla <jesseperla_at_[hidden]>
> wrote:
> > Hi,
> > As I am using a variety of libraries for my projects, and while I will
> > standardize on using ublas::matrix and ublas::vector, the other libraries
> > use a variety of C-style array/matrix, stl::vector, and homegrown
> > libraries. I am looking for a relatively straightforward way to pass in
> my
> > ublas vectors/matricies to these functions without copying all of the
> > values, etc. For now, I can focus on dense column:major only, with
> > assumption of no reallocation/resizing.
> >
> > I have seen some discussion about getting the raw pointer to the
> > matrix/vector libraries, but I am having trouble getting it to work.
> Also
> > have heard about adaptors, but don't quite understand what people are
> > talking about.
>
> Because of the needs you describe above, it would definitely be a good
> idea to get a better understanding of the adaptors. In brief, they
> provide the underlying storage and handle all the memory management
> details for all the ublas containers. More in the storage section of
> he documentation
> http://www.boost.org/doc/libs/1_36_0/libs/numeric/ublas/doc/index.htm
> .
>
> >
> > It key cases are:
> > 1) std::vector
> > The faq suggests using a copy iterator to go between them, but I don't
> want
> > to copy the data. std::vector does support initialization from a data
> > pointer...
>
> No, std::vector never shares the pointer. In fact it does not have a
> constructor that takes a pointer. It has one which takes 2 iterators
> though, and in which case it copies the data.
>
> > I would love something like:
> >
> > ublas::vector<double> v(4);
> > std::vector<double>& v_std = ublas_to_std_vector<double>(v);
> > //Or better yet, if I could define a cast for this?
> >
>
> Casting is definitely not possible (nor desirable) for this types of
> issues. Reusing an existing std::vector for a ublas vector is possible
> _in principle_, but one would have to write a ublas adaptor that is
> based on std::vector (which in itself is not that difficult). Then one
> could implement a swap() type of method to take over std::vector's
> storage and use it for the ublas vector adaptor. See below for a
> different, unsafe way of reusing the storage of std::vector.
>
> > 2) C-style array and matrix:
> > a) Want to be able to pass in the matrix or vector to C style functions:
> > //Couldn't get matrix type operators to work. I tried to various
> > permutations on:
> > ublas::matrix<int> A2(2,2);
> > int** Ap2;
> > Ap2 = &(A2.data()[0]);
> > Ap2[0][0] = 1;
> > //It appears that for matrices, the data() returns back an int*
> instead
> > of an int** which is what most functions require. Perhaps there is an
> > unsafe cast that would solve this problem? I can't figure out how to
> trick
> > C style int** matrices to think they are of size NxN from a raw pointer
> > without allocating data to determine pointer offsets.
>
> Again this is not possible, since the storage for dense matrices (and
> in fact for most of the sparse ones as well) is linear. However, this
> is the standard way of designing a dense matrix class, and code that
> expects a matrix as int ** should be reexamined and fixed. Note that
> if you plan interacting with 3rd party libraries (e.g. lapack), you
> don't need this types of casts, there are ublas bindings available for
> most of the popular libraries.
>
> >
> > //The following seemed to work for ublas::vector though.
> > ublas::vector<int> A(2);
> > int* Ap;
> > Ap = &(A.data()[0]);
> > Ap[0] = 1;
> >
> > b) If C style functions return data, would like to put the ublas wrapper
> > around them. I assume this is done with calling a constructor on
> > ublas::matrix or ublas::vector with the pointer.
> >
>
> The preferred and safe way of using ublas containers is to copy the
> data, and thus allow ublas to handle all the tricky memory management
> associated with copy, assignment, resize etc. However there has been
> lots of discussions on this list on this subject, and ublas already
> has ways of doing this. Search for BOOST_UBLAS_SHALLOW_ARRAY_ADAPTOR.
> I'd strongly discourrage the use of this technique however, until
> you've proven (through profiling and such), that copying the data is
> where your bottleneck is.
>
> > 3) Home Grown matrix libraries:
> > A million exist, and nearly all seem to have constructors for passing in
> a
> > pointer to data, so I think the problem
> > Maybe a cast/function could be written to do this generically since the
> > concept of a copy constructor with pointers may be all that is needed
> from
> > the library? But I don't know enough about generic programming/casts to
> > write this myself. Is the best way to deal with this stuff to define a
> > reinterpret_cast? If so, any good examples of how to do so and how to
> add
> > these to libraries without modifying the underlying boost::ublas, etc.?
> >
> > Any ideas or example code for this kind of stuff?
>
> If a matrix constructor takes a pointer, it either does a copy and
> manages its own memory, or shares the pointer and does not manage the
> memory. Both capabilities are there in ublas. Use std::copy for
> copying data (recommended), and shallow array adaptor for sharing the
> data (not recommended).
>
> -Vardan
> _______________________________________________
> ublas mailing list
> ublas_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/ublas
>