Re: [Boost-users] Extending boost::function for mathematics?

Subject: Re: [Boost-users] Extending boost::function for mathematics?
From: peter_foelsche_at_[hidden]
Date: 2009-01-28 16:26:47

> On automatic differentiation:
> Russel:  I am glad you brought this up.  I think that this is about
> the coolest application of generic/functional programming I have ever
> seen, and was a driver for what I have been talking about here with
> math_function.

I wrote an AD class using forward method in 2002.
Meanwhile it is using boost::mpl to deal with different template instantiations containing different partial derivatives.
Cannot contribute it here, because my boss says, that we are not in the business of helping our competitors.

BTW -- it is possible to calculate higher derivatives by applying such a class to itself.

> * If anyone reading the "[Proto] implementing an computer algebra
> system with proto" thread is reading this, I believe it is a very
> similar type of problem.

don't see what proto has to do with this.
Or I still don't understand what proto is intended for.

> * No more use of expensive and inaccurate finite-difference since you
> will have a single direct function.

right -- finally

> * Because we are usually dealing with multi-dimensional functions, you
> typically need to think through how all the product/chain rule
> interacts with an array as well.

I don't understand
Simply replace double with the type of your to be written AD class.

> * Many implementations of AD use a text pre-processor for Fortran,
> etc.

antique

> 
> So you can immediately guess that one way to approach this is with
> expression templates.  The paper you mentioned is a great reference.
> Another thing for those interested to look at is the Sacado subproject
> of Trilinos(at Sandia).  See 
> https://cfwebprod.sandia.gov/cfdocs/CCIM/docs/Sacado_06.ppt
> Another implementation I was looking at is: 
> http://www.fadbad.com/download/FlexibleAD-talk.pdf
> 

there are books out there describing the forward approach of AD and the reverse approach. The forward approach is usually faster, since it does not need to create a stack of operations to be interpreted later.

> A few notes on these implementations:
> * In order to use expression templates and overloading, these have
> ended up having to subvert and replace the type systems or replace
> standard functions with their own templates.  This may be necessary in
> some form, but right now you end up having to write all of your code
> with non-standard data structures, types, and functions.  And you
> often need to set up a complicated framework to get AD support.
> * You need expression templates for all of the functions.  But boost
> has a lot of functions would want to use in
> http://www.boost.org/doc/libs/1_35_0/libs/math/doc/html/index.html
> * But for many problems, one needs to be able to have their own
> functions AD aware as well.  For example, my research usually involves
> solving a functional equation (dynamic programming).  You pass in
> another function as part of its parameters (For an example, see see
> http://jacek.rothert.googlepages.com/vfi.pdf)

other functions will have to be written as templates so that they can either accept a double or your to be written AD-type.

I sincerely hope that such a class makes it into boost soon.

Peter