Re: [boost] Review of a safer memory management approach for C++?

Subject: Re: [boost] Review of a safer memory management approach for C++?
From: Bartlett, Roscoe A (rabartl_at_[hidden])
Date: 2010-06-06 21:49:22

Hello,

This is the last email that I will write and respond to on this tread of emails about runtime vs. compile-time polymorphism (we have beet this dead horse about enough I think).  I will summarize my arguments and the baulk responses from people.

My perspective comes from having developed and maintained a templated package in Trilinos called Thyra that uses only the most basic type of templating; templating runtime abstract interfaces on the single Scalar type (i.e. double, float, complex<float>, complex<double>, etc).  This is the most trivial and basic type of templating that you can possibly imagine (and was the type of templating that templates in C++ was supposed to support in the first place).  Templating all of this code was not my first choice but in retrospect it was the right thing to do since you would have to duplicate all of the code (source and binary) multiple times (once for each scalar type) in order to support this variety of scalar types in C++ in a single program.  However, this has not been easy by any stretch of the imagination.  Now Trilinos is redoing is core linear algebra and linear solvers with the advent of the more heavily templated Tpetra package and will have similar issues.

Up until recently, I was just having to compile and link Thyra and client code using the implicit template instantiation mechanism on the *wide* variety of compilers that we are contractually obligated to support including several different versions of GCC, Intel, Sun, PGI, IBM, Pathscale, and Visual C++ on a whole bunch of platforms (Linux, various flavors of Unix, Windows, specialty HPC systems, etc.).  In many ways, I think that I have been the unfortunate first adopter of heavily templated code on many of these platforms.  Eventually, I figured out how to portably and maintainability manage explicit template instantiation (see GCG 28 in http://www.cs.sandia.gov/~rabartl/ThyraCodingGuideLines.pdf) which has allowed me to port this templated code to several platforms where it would otherwise just segfault or crash the compilers (e.g. Sun and PGI).  Going to carefully orchestrated explicit template instantiation has allowed me to recover a lot of the positive properties of non-templated code (i.e. fast recompile times, being able to compile just a single source files to work out compile problems, etc.).  However, even with this most trivial type of templating (a single template argument), explicit instantiation is not automatic to support and significantly complicates the design and maintenance of the software.  For new more heavily templated packages like Tpetra and the next generation of template software based on Tpetra in Trilinos that is being written, it is even more difficult and more complex to support explicit instantiation.

If have fought the good fight for the use templated code in this non-ideal HPC environment for many years and I have earned to right to express my experiences and wisdom from the trenches on this topic.  In return, the majority of the comments have basically been:

a) Common response from list members:  If have had trouble with templated code, it is because I don't know how to design or write software (just read some of the comments below and in some of the other email responses).  My response:  You can't get more simple templated code than a single Scalar template argument that is used in Thyra.  I can't imagine a more basic use of templating and I have no clue how to design this "better".  The only way to make this portable is to use explicit template instantiation which is not completely trivial to support, even for the most simplistic type of templating (i.e. just one Scalar template argument).

b) Common response from list members:  I should not bother supporting troublesome compilers like Sun, IBM, and PGI.  My response: As I have explained many times, if our code can't support *all* of these platforms, then our customers can't use any of this code and we need to find another job.  I don't know how much more direct I can state this.  I have fought many battles against our customers who demand to use bad C++ compilers but we are going to have one or two bad compilers that we need to support for many years to come that throws a monkey wrench into the otherwise "perfect design" of the software we would like to write.  You should see the emails that I get from people about "being in an ivory tower" when I complain about having to support non-ANSI C++ compilers.

If some of my responses have been testy, it is because I am a little warn out with the above two general responses to problems that I have had with heavily templated code.  Please don't just dismiss my opinions based on hard fought experience by assuming that I have no idea what I am doing.

This entire thread of many emails and responses was sparked by my original request to review the approach described in:

    http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf

which really has nothing to do with runtime vs. compile-time polymorphism.

There has been a lot of responses but has anyone taken the time to even read at least the reduced table of contents described in the Preface?  I know this is a long document at over 150 pages in the main text but the reduced table of contents is only something like 45 pages or so.

A few selected responses to some other emails below ...

>    1. Re: Review of a safer memory management approach for C++?
>       (Mathias Gaunard)
>    3. Re: FW: Boost Digest, Vol 2933, Issue 1 (joel falcou)

> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Sat, 05 Jun 2010 17:59:24 +0100
> From: Mathias Gaunard <mathias.gaunard_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] Review of a safer memory management approach for
> 	C++?
> Message-ID: <hudvpc$76f$1_at_[hidden]>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> ...
>
> > 2) Current compilers may not even be able to compile large amounts of
> template code at all.  There are compilers that we have to support
> (e.g. Sun and IBM) that will segfault or return "internal compiler
> error" when you give them too much template code.  These same compilers
> have no problem with virtual functions with standard OO.  This is
> reality.
> 
> It's up to you whether you want to support those compilers or not.
> I myself see little compelling reason to support the Sun compiler,
> almost likewise for IBM, at least for new C++ projects.
> 
> Balance your support for legacy compilers with your willingness to have
> nice code.

[Bartlett, Roscoe A] 

As explained above, I fight the battle all the time to avoid having to support such bad non-ANSI compilers but we will have to live with these problems for many years to come.  If we can't support *all* of these platforms, then they can't use *any* of the code that does not port to all of these platforms, period.  This is not a joke.  I suspect that Trilinos and other HPC projects are not the only software projects that have to live this reality.

> > 3) Templated programs can result in massive object-code bloat over
> equivalent runtime OO programs.  If you are really sloppy with
> templates and use it for everything, you can easily double, triple, or
> more (10x) the size of the object code and executables (see Item 44 in
> "Effective C++, 3ed edition").
> 
> They can, but they don't have to.
> If your template function is not a good inlining candidate, is likely
> to
> get instantiated with many different types and and if making it
> runtime-generic has sufficiently low costs, then you should consider
> doing it.

[Bartlett, Roscoe A] 

99% of Thyra-based code should not ever be inlined (see description of Thyra above or just go look at the Thyra Website).

> Note, however, that the size costs are usually grossly exaggerated.
> I've
> seen people use advanced template C++ code on microcontrollers (the
> kind
> with a few kilobytes of memory) just fine.

[Bartlett, Roscoe A] 

If you templated the algorithms instead of using Thyra interfaces, you would have new and different object code for every concrete set of operator/vector subclasses.  Just in Trilinos this would result in at least a 3x increase in the size of the object code.  There is no way around this.

> > As long as people keep suggesting that we template all of our codes
> for every little reason
> 
> No one is suggesting this.
> Make your code only as generic as your algorithms intrinsically are, no
> more.
> 
> Algorithms are what lay at the center of generic programming, not
> objects. This is quite a different view from OOP.

[Bartlett, Roscoe A] 

Using Thyra as the example, if you throw away the abstract interfaces, then 99% of your algorithmic code would be templated in order to achieve (compile-time) polymorphic behavior.  This is a very simple issue to analyze.

> > Thinking that we are going to drop runtime polymorphism and use
> exclusively compile-time polymorphism for everything is just fantasy
> land.
> 
> Compile-time polymorphism is better, and you can build runtime
> polymorphism on top of it with type erasure.
> 
> So why avoid compile-time polymorphism when you can do it?

[Bartlett, Roscoe A] 

The only way to manage large amounts of templated code is to explicitly instantiate it which is not trivial to code and maintain (again, see above).

> ------------------------------
> 
> Message: 3
> Date: Sat, 05 Jun 2010 19:08:55 +0200
> From: joel falcou <joel.falcou_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] FW: Boost Digest, Vol 2933, Issue 1
> Message-ID: <4C0A84A7.7060002_at_[hidden]>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> ...
>
> And I can tell you that, as a consultant and now as an assistant
> professor involved in variosu HPC project, the quality of code I saw in
> laboratory is usually sub-par.

[Bartlett, Roscoe A] 

What is this supposed to mean?

> > The size of object code does have some impact, especially some HPC
> machines.  Again, please teach us all how to do this because we
> apparently don't know what we are doing.
> >
> You can also learn to reply without actually barking at people. It
> helps
> having a sane conversation.

[Bartlett, Roscoe A] 

As described at the top, when the common response to problems is that "you don't know how to design software" one has almost no choice but to respond in such a way.  Again, the templating in Thyra could not be more simple.

> > The compilers segfault or return internal compiler errors or take
> hours to compile if they do finish at all in some cases.  See above.
> Reality!!!!!!!
> >
> Reality of sucky compilers from 2 century ago. It's lkike blaming
> people
> using new cars with biofuel whiel comapring them to Ford T.

[Bartlett, Roscoe A] 

As described at the very top, I fight the battle against bad C++ compilers constantly but we are going to have to live with some bad compilers for many years to come.  What do you expect us to do?  Should I just quit my job as a protest against bad C++ compilers?

> > My argument is that current accepted ways of writing OO code in C++
> make it too hard to write correct well designed programs and an
> affordable cost.  My argument is that the Teuchos MM approach described
> in:
> >
> >     http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf
> >
> > together with static analysis tools to help enforce the idioms (and
> keep raw pointers encapsulated) can largely eliminate undefined
> behavior in C++ programs due to the usage of memory and also yield
> programs that are more self-documenting at the same time.
> >
> >
> Well, you better be writing JAVA or OCaml ...

[Bartlett, Roscoe A] 

Not true.  As described in Section 6.1 in:

   http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf

the idioms using the Teuchos MM classes can yield better self documenting code than you can get with languages like Java and Python.  Did you bother to read the document?  Yes I know it is very long but I think there is some worthwhile material contained in it that should be given a far shake and not just dismissed by uninformed email responses.  (By uninformed, I mean you likely did not ready the document, not that you are fundamentally uninformed).

- Ross