Re: [boost] Review of a safer memory management approach for C++?

Subject: Re: [boost] Review of a safer memory management approach for C++?
From: Bartlett, Roscoe A (rabartl_at_[hidden])
Date: 2010-06-04 12:40:04

David,

Much more discussion of this topic over email is likely not very useful at this point but I will make some very brief responses below.

>   14. Re: FW: Boost Digest, Vol 2929, Issue 4 (David Abrahams)

> ------------------------------
> 
> Message: 14
> Date: Fri, 04 Jun 2010 09:42:36 -0400
> From: David Abrahams <dave_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] FW: Boost Digest, Vol 2929, Issue 4
> Message-ID: <m2hbliewxf.wl%dave_at_[hidden]>
> Content-Type: text/plain; charset=UTF-8
> 
> At Thu, 3 Jun 2010 17:29:15 -0600,
> Bartlett, Roscoe A wrote:
> >
> > David,
> >
> > I have some responses below ...
> >
> > > Message: 13
> > > Date: Tue, 01 Jun 2010 16:16:25 -0400
> > > From: David Abrahams <dave_at_[hidden]>
> > >
> > > I'm all for making essential complexity obvious and eliminating
> > > accidental complexity, but I'm not sure I see a compelling example
> > > that Teuchos does that, at least not when compared with what I
> > > consider normal state-of-the-art C++ programming, which is mostly
> > > value-based, dynamic allocations are rare, and those that occur are
> > > immediately managed, e.g. by appropriate smart pointers.
> >
> > [Bartlett, Roscoe A]
> >
> > I don't want to get into an argument about the "proper" way to use
> > C++ here (but I will a little below).  In Item 1 of "Effective C++,
> > 3rd Edition", Scott Meyers identifies four different major
> > programming paradigms that are supposed to be supported by C++:
> >
> > 1) C: raw pointers, low-level built-in datatypes (int, double,
> > etc.), statements, blocks, functions etc.
> >
> > 2) Object-oriented C++: Inheritance, virtual functions, dynamic
> > allocation, RTTI, etc. (i.e. runtime polymorphism)
> >
> > 3) Template C++: Typical generic programming, template
> > metaprogramming, etc. (i.e. compile-time polymorphism)
> >
> > 4) STL: Containers, iterators, algorithms, etc. and code like this
> > (not just using STL) (closely related to #3 Template C++).
> 
> Yes, I'm aware of that way of dividing up the picture.
> 
> > Of course any well written large C++ program is a mixture of all
> > four of the above (and more).
> 
> I don't take it for granted that any well-written large C++ program
> will use any given paradigm.
> 
> > What you seem to be arguing above is that most good C++ software
> > will use templates in #3 and #4 but almost none of #2
> > object-oriented C++.
> 
> Did I say anything about templates?  No.
> 
> I _will_ argue that OOP has costs to software complexity that are
> often under-appreciated.

 [Bartlett, Roscoe A] 

Any approach can be misused, including anything you might be trying to advocate (but I am not 100% sure what that is exactly).

> > Note that C++ was first designed in the early
> > 80s primarily to support #2 object-oriented C++.  I can see the
> > reason for this bias for #3 and #4 given the nature of most of the
> > Boost libraries and you can have that opinion if you would like.
> 
> I don't, so please read what I wrote and try to address that rather
> than to your preconceived notions of my ?bias.?

[Bartlett, Roscoe A] 

I am not trying to put words in your mouth.  I am just trying to pin down your argument/position.  You can say that I have a "bias" for good software if you would like :-)  Seriously, I am sorry if I misinterpreted you position with the position of others on the boost list who were arguing for a more template-heavy approach.  I guess that you are arguing for more of a stack-based approach (see below).

> > I would argue that the current accepted approaches to writing OO
> > programs in C++ (i.e. #2 OO C++ features) make it too easy to create
> > programs with undefined behavior (i.e. segfaults or worse) and as a
> > result, the paranoia of undefined behavior in C++, memory leaks
> > etc. lead to lots of bad programming practices and designs (see
> > Section 1 in
> > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf).
> 
> Seriously?  Are you saying that a disciplined application of
> shared_ptr, weak_ptr, new_shared, et. al when dynamic allocations are
> called for is insufficient for most OO programs?

[Bartlett, Roscoe A] 

As argued in the Teuchos MM document

    http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf

these boost classes are a good start but they are *not* sufficient to build a safe and effective memory management system that encapsulated *all* raw pointers.  What is missing for single objects, as described in Section 5.14 and other sections, is the Teuchos::Ptr class.  Without integrating a class like Ptr in with the reference-counted pointer classes, you can't build a safe and effective memory management system in C++ that comes closer to eliminating undefined behavior.  Also, as described in Section 5.9.2, separating weak_ptr from shared_ptr is not ideal from a design perspective since it reduces the generality and reusability of software (see the arguments for this in the document).

Did you read at least the sections called out in the reduced table of contents given in the Preface?  I think these sections should address all of these questions (and more).

As for arrays, the current software in C++0x and boost is less sufficient (again, see Section 5.14 and the sections it refers to).  What is in boost and C++0x is a good start but it has not gone far enough (i.e. the encapsulation of *all* raw pointers in *all* code, except where non-compliant code must be called of course).

> > > > Trying to over design a program to avoid all shared ownership is
> > > > what make C++ programming so unproductive and has all the
> negative
> > > > consequences described in Section 1 in:
> > > >
> > > >
> http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf
> > > >
> > > > Designs with object sharing can be much less complex overall than
> > > > designs without sharing.  You just need decent tools to detect
> circular
> > > > reference problems (and that is what the Teuchos::RCP class has).
> > >
> > > Well, I fundamentally disagree with all of the above.  Overuse of
> > > runtime polymorphism, thus dynamic allocation, and thus shared
> > > ownership (it's almost an inevitable progression) is one of the
> things
> > > that has made C++ programming unproductive, and those people I know
> > > who most zealously avoid it tend to be more productive than
> everyone
> > > else.  IMO.
> >
> >  [Bartlett, Roscoe A]
> >
> > When runtime performance or other issues related to dynamic
> > allocations and classic OO are not a problem, classic OO C++
> > programs using runtime polymorphism are typically superior to highly
> > templated C++ programs using static polymorphism
> 
> Straw man; I'm not arguing for ?templated C++ programs using static
> polymorphism.?

[Bartlett, Roscoe A] 

Okay, the runtime vs. compile-time polymorphism debate is another issue (but a very important one).  Are you instead arguing for a static and stack-based approach to programming?  I haven see people try this and it leads to very inflexible software that is hard to maintain, reuse, and achieve optimal performance.  One particularly good (or bad) example of this approach that I have seen is given in:

    http://www.trip.caam.rice.edu/software/rvldoc/rvl.html

If you look deep in the RVL code, you will see a few Gotchas that destroy the true safely that is supposed to be afforded by the approach.  Actually, ironically, they use stack-based object to try to avoid memory management problems (but not sufficiently) but yet at the same time, they rely on other types of mutable shared objects with implicit updates that make my very unconformable.

> > > In my experience, designs with object sharing tend to increase
> > > complexity in all kinds of ways beyond the memory management issues
> > > that you describe.  Perhaps the most important way is, they tend to
> > > obscure the overall computation being performed by hiding it in
> > > networks of interacting objects.
> >
> > [Bartlett, Roscoe A]
> >
> > This is not going to magically change by templating everything.
> 
> Straw man; I'm not arguing for ?templating everything.?
> 
> > If a lot of objects collaborate together to perform computations, it
> > does not matter if you use static or runtime polymorphism; the code
> > will be hard to figure out and you will need print statements and/or
> > run it in a debugger to see what is really happening.
> 
> Yes, and an ?OO mindset? has tended to lead to partitioning systems so
> that ?a lot of objects collaborate together to perform computations,?
> which tends to obscure the computation performed.

[Bartlett, Roscoe A] 

Yes but good OO design tries to minimize the web of object dependences.  There are lots of good references on the need and methods for reducing the endless web of object dependencies (see http://www.cs.sandia.gov/~rabartl/readingList.html).

> > > I would like to see an example of a design with shared object
> > > ownership that is ?much less complex? than all other functionally
> > > equivalent designs that eschew shared ownership.  I'm not saying
> such
> > > applications don't exist, but IME they are rare, and a concrete
> > > example would help to bring this whole discussion back to earth.
> >
> > [Bartlett, Roscoe A]
> >
> > One design where shared mutable objects likely makes things overall
> > simpler and more natural is the PDE expression mechanism in the
> > Sundance code (http://www.math.ttu.edu/~klong/Sundance/html/).  If
> > the clients all expect remote updates of shared objects, then things
> > work great.
> 
> I'm sorry, your reference isn't specific enough for me to see the
> design piece you're referring to.

[Bartlett, Roscoe A] 

It would take a significant effort to pin-point exactly the approach being used in the Sundance expression mechanism and then to describe an alternative design that does not rely on the kind of object sharing.  I think that this example in Sundance is the exception and not the rule.  In general, I don't like lots for mutable shared objects for the same reasons that you state.  However, there are cases where using shared mutable objects is the best design to solve a given problem.

As for dynamic vs. stack-based programming, in summary, if you like the approach used in:

    http://www.trip.caam.rice.edu/software/rvldoc/rvl.html

which tries to to make *every* object a stack-based object, then you and I can safely agree to disagree about classic OO vs. purely stack-based programming.

Sorry I misinterpreted your position (or projected other peoples positions on you) about the runtime vs. compile-time polymorphism debate.

Cheers,

- Ross