- Boost - lists.boost.org

Re: [boost] [bjam][msm] help building tests
by Christophe Henry 07 Jun '10

07 Jun '10

>> I am almost ready for release, except that I run into a (admittedly >> expected) little snag. I have in the trunk a libs/msm/test. I also >> have a jamfile, which even compiles. But doesn't link. >> The reason being that bjam does not find the Boost.Test libs. Ok I >> understand I should be telling bjam to do it, but looking at other >> libraries' jamfiles, I don't find it either, so I'm wondering how this >> is supposed to work. Am I missing something? > > You can change libs/msm/test/Jamfile.v2 like this > > project msm > : > requirements > <include>. > <library>/boost/test//boost_unit_test_framework/<link>static > ; > > to link against Boost.Test libraries. I'm not sure but I think that's > all you need. > >> On a related note, I'm also wondering how to get the tests to show up >> on the trunk regression page. > > Look in status/Jamfile.v2 and follow the pattern (add line for msm/ > test in the appropriate place) to get your product's tests running. > > Hope that helps. > > -- Noel Hi Steven, Noel, Jürgen, Vicente, This is what I call getting good help fast! :) Thanks to all, I just did the changes and committed. Tests ran successfully (well, one 1 compiler...) and status/Jamfile.v2 now contains msm, so I eagerly wait for the results ;-) So I now go further through the check-list. From what I saw, the inspection report now considers msm as one of the worst offenders. Can't have that. ^^ Big big thanks to all! Christophe

1 0

[bjam][msm] help building tests
by Christophe Henry 07 Jun '10

07 Jun '10

Hi, I am almost ready for release, except that I run into a (admittedly expected) little snag. I have in the trunk a libs/msm/test. I also have a jamfile, which even compiles. But doesn't link. The reason being that bjam does not find the Boost.Test libs. Ok I understand I should be telling bjam to do it, but looking at other libraries' jamfiles, I don't find it either, so I'm wondering how this is supposed to work. Am I missing something? On a related note, I'm also wondering how to get the tests to show up on the trunk regression page. Thanks, Christophe

5 4

Re: [boost] [regex, xpressive] interesting(?) perf benchmark
by Christophe Henry 07 Jun '10

07 Jun '10

>> How about a warning "you may not be using the optimal tool for the job" >> until then? > > I'm having a hard time getting excited about that. :) >> Are the finite state machine libraries in Boost usable for this at all? > > I don't know. At BoostCon, Christophe presented some interesting results > pitting MSM against xpressive for some simple regexing. An interesting > research project would be to investigate whether xpressive's hybrid > static/dynamic NFA design can be applied to DFAs as well, with similar > perf wins. This would be a non-trivial undertaking, like completely > reimplementing xpressive. I think MSM could be a possibility, and it seems to be quite fast for some cases. I was actually thinking of handling static Xpressive as a MSM-compatible front-end, so that the user could switch at will between Xpressive and MSM, which happen to be complementary because most performant on different types of regexes. But there is a long way (and a fair number of technical issues) between a toy example and a real library so it could take a while before I get to it (besides, I am not a domain expert). Note: there is an example provided with the BoostCon slides. I have more and I can pack them with MSM's examples. Christophe

1 0

identifying file system type
by Roger Martin 07 Jun '10

07 Jun '10

Hi, I'm looking for a way code can automatically decide whether it is running on a parallel or non-parallel file system. Need to retrieve the information about the file system type. Get values like "ext3", "nfs" "gpfs", "pvfs", "Lustre" etc. Is there any file system info obtainable in code?

1 0

[trac] "Building Boost" vs. "build"
by Vladimir Prus 07 Jun '10

07 Jun '10

It looks like somebody went via Trac and moved some issues from the "Building Boost" component to "build" component. See, for example: https://svn.boost.org/trac/boost/ticket/4147 I certainly appreciate help with triaging issues, but in general, such change seems wrong -- "Building Boost" is for issues affecting exactly that -- building boost, while the 'build' component contain issues that are not necessary affecting Boost C++ Libraries at all. Unfortunately, the changes are all anonymous, so it would be great if whoever made them speak up, and discuss the rationale. Thanks, -- Vladimir Prus CodeSourcery vladimir(a)codesourcery.com (650) 331-3385 x722

4 5

Active Tickets report and accepted info
by vicente.botet 07 Jun '10

07 Jun '10

Hi, in https://svn.boost.org/trac/boost/report/1 I see the following: "If a ticket has been accepted, a '*' is appended after the owner's name" Is there an error on the report or the maintainers do not use to accept the tickets? I find this information very useful, if it states that the maintainer accept to take care of this ticket. Best, _____________________ Vicente Juan Botet Escribá http://viboes.blogspot.com/

5 7

Re: [boost] Review of a safer memory management approach for C++?
by Bartlett, Roscoe A 07 Jun '10

07 Jun '10

David, Much more discussion of this topic over email is likely not very useful at this point but I will make some very brief responses below. > 14. Re: FW: Boost Digest, Vol 2929, Issue 4 (David Abrahams) > ------------------------------ > > Message: 14 > Date: Fri, 04 Jun 2010 09:42:36 -0400 > From: David Abrahams <dave(a)boostpro.com> > To: boost(a)lists.boost.org > Subject: Re: [boost] FW: Boost Digest, Vol 2929, Issue 4 > Message-ID: <m2hbliewxf.wl%dave(a)boostpro.com> > Content-Type: text/plain; charset=UTF-8 > > At Thu, 3 Jun 2010 17:29:15 -0600, > Bartlett, Roscoe A wrote: > > > > David, > > > > I have some responses below ... > > > > > Message: 13 > > > Date: Tue, 01 Jun 2010 16:16:25 -0400 > > > From: David Abrahams <dave(a)boostpro.com> > > > > > > I'm all for making essential complexity obvious and eliminating > > > accidental complexity, but I'm not sure I see a compelling example > > > that Teuchos does that, at least not when compared with what I > > > consider normal state-of-the-art C++ programming, which is mostly > > > value-based, dynamic allocations are rare, and those that occur are > > > immediately managed, e.g. by appropriate smart pointers. > > > > [Bartlett, Roscoe A] > > > > I don't want to get into an argument about the "proper" way to use > > C++ here (but I will a little below). In Item 1 of "Effective C++, > > 3rd Edition", Scott Meyers identifies four different major > > programming paradigms that are supposed to be supported by C++: > > > > 1) C: raw pointers, low-level built-in datatypes (int, double, > > etc.), statements, blocks, functions etc. > > > > 2) Object-oriented C++: Inheritance, virtual functions, dynamic > > allocation, RTTI, etc. (i.e. runtime polymorphism) > > > > 3) Template C++: Typical generic programming, template > > metaprogramming, etc. (i.e. compile-time polymorphism) > > > > 4) STL: Containers, iterators, algorithms, etc. and code like this > > (not just using STL) (closely related to #3 Template C++). > > Yes, I'm aware of that way of dividing up the picture. > > > Of course any well written large C++ program is a mixture of all > > four of the above (and more). > > I don't take it for granted that any well-written large C++ program > will use any given paradigm. > > > What you seem to be arguing above is that most good C++ software > > will use templates in #3 and #4 but almost none of #2 > > object-oriented C++. > > Did I say anything about templates? No. > > I _will_ argue that OOP has costs to software complexity that are > often under-appreciated. [Bartlett, Roscoe A] Any approach can be misused, including anything you might be trying to advocate (but I am not 100% sure what that is exactly). > > Note that C++ was first designed in the early > > 80s primarily to support #2 object-oriented C++. I can see the > > reason for this bias for #3 and #4 given the nature of most of the > > Boost libraries and you can have that opinion if you would like. > > I don't, so please read what I wrote and try to address that rather > than to your preconceived notions of my ?bias.? [Bartlett, Roscoe A] I am not trying to put words in your mouth. I am just trying to pin down your argument/position. You can say that I have a "bias" for good software if you would like :-) Seriously, I am sorry if I misinterpreted you position with the position of others on the boost list who were arguing for a more template-heavy approach. I guess that you are arguing for more of a stack-based approach (see below). > > I would argue that the current accepted approaches to writing OO > > programs in C++ (i.e. #2 OO C++ features) make it too easy to create > > programs with undefined behavior (i.e. segfaults or worse) and as a > > result, the paranoia of undefined behavior in C++, memory leaks > > etc. lead to lots of bad programming practices and designs (see > > Section 1 in > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf) > > Seriously? Are you saying that a disciplined application of > shared_ptr, weak_ptr, new_shared, et. al when dynamic allocations are > called for is insufficient for most OO programs? [Bartlett, Roscoe A] As argued in the Teuchos MM document http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf these boost classes are a good start but they are *not* sufficient to build a safe and effective memory management system that encapsulated *all* raw pointers. What is missing for single objects, as described in Section 5.14 and other sections, is the Teuchos::Ptr class. Without integrating a class like Ptr in with the reference-counted pointer classes, you can't build a safe and effective memory management system in C++ that comes closer to eliminating undefined behavior. Also, as described in Section 5.9.2, separating weak_ptr from shared_ptr is not ideal from a design perspective since it reduces the generality and reusability of software (see the arguments for this in the document). Did you read at least the sections called out in the reduced table of contents given in the Preface? I think these sections should address all of these questions (and more). As for arrays, the current software in C++0x and boost is less sufficient (again, see Section 5.14 and the sections it refers to). What is in boost and C++0x is a good start but it has not gone far enough (i.e. the encapsulation of *all* raw pointers in *all* code, except where non-compliant code must be called of course). > > > > Trying to over design a program to avoid all shared ownership is > > > > what make C++ programming so unproductive and has all the > negative > > > > consequences described in Section 1 in: > > > > > > > > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf > > > > > > > > Designs with object sharing can be much less complex overall than > > > > designs without sharing. You just need decent tools to detect > circular > > > > reference problems (and that is what the Teuchos::RCP class has). > > > > > > Well, I fundamentally disagree with all of the above. Overuse of > > > runtime polymorphism, thus dynamic allocation, and thus shared > > > ownership (it's almost an inevitable progression) is one of the > things > > > that has made C++ programming unproductive, and those people I know > > > who most zealously avoid it tend to be more productive than > everyone > > > else. IMO. > > > > [Bartlett, Roscoe A] > > > > When runtime performance or other issues related to dynamic > > allocations and classic OO are not a problem, classic OO C++ > > programs using runtime polymorphism are typically superior to highly > > templated C++ programs using static polymorphism > > Straw man; I'm not arguing for ?templated C++ programs using static > polymorphism.? [Bartlett, Roscoe A] Okay, the runtime vs. compile-time polymorphism debate is another issue (but a very important one). Are you instead arguing for a static and stack-based approach to programming? I haven see people try this and it leads to very inflexible software that is hard to maintain, reuse, and achieve optimal performance. One particularly good (or bad) example of this approach that I have seen is given in: http://www.trip.caam.rice.edu/software/rvldoc/rvl.html If you look deep in the RVL code, you will see a few Gotchas that destroy the true safely that is supposed to be afforded by the approach. Actually, ironically, they use stack-based object to try to avoid memory management problems (but not sufficiently) but yet at the same time, they rely on other types of mutable shared objects with implicit updates that make my very unconformable. > > > In my experience, designs with object sharing tend to increase > > > complexity in all kinds of ways beyond the memory management issues > > > that you describe. Perhaps the most important way is, they tend to > > > obscure the overall computation being performed by hiding it in > > > networks of interacting objects. > > > > [Bartlett, Roscoe A] > > > > This is not going to magically change by templating everything. > > Straw man; I'm not arguing for ?templating everything.? > > > If a lot of objects collaborate together to perform computations, it > > does not matter if you use static or runtime polymorphism; the code > > will be hard to figure out and you will need print statements and/or > > run it in a debugger to see what is really happening. > > Yes, and an ?OO mindset? has tended to lead to partitioning systems so > that ?a lot of objects collaborate together to perform computations,? > which tends to obscure the computation performed. [Bartlett, Roscoe A] Yes but good OO design tries to minimize the web of object dependences. There are lots of good references on the need and methods for reducing the endless web of object dependencies (see http://www.cs.sandia.gov/~rabartl/readingList.html) > > > I would like to see an example of a design with shared object > > > ownership that is ?much less complex? than all other functionally > > > equivalent designs that eschew shared ownership. I'm not saying > such > > > applications don't exist, but IME they are rare, and a concrete > > > example would help to bring this whole discussion back to earth. > > > > [Bartlett, Roscoe A] > > > > One design where shared mutable objects likely makes things overall > > simpler and more natural is the PDE expression mechanism in the > > Sundance code (http://www.math.ttu.edu/~klong/Sundance/html/) If > > the clients all expect remote updates of shared objects, then things > > work great. > > I'm sorry, your reference isn't specific enough for me to see the > design piece you're referring to. [Bartlett, Roscoe A] It would take a significant effort to pin-point exactly the approach being used in the Sundance expression mechanism and then to describe an alternative design that does not rely on the kind of object sharing. I think that this example in Sundance is the exception and not the rule. In general, I don't like lots for mutable shared objects for the same reasons that you state. However, there are cases where using shared mutable objects is the best design to solve a given problem. As for dynamic vs. stack-based programming, in summary, if you like the approach used in: http://www.trip.caam.rice.edu/software/rvldoc/rvl.html which tries to to make *every* object a stack-based object, then you and I can safely agree to disagree about classic OO vs. purely stack-based programming. Sorry I misinterpreted your position (or projected other peoples positions on you) about the runtime vs. compile-time polymorphism debate. Cheers, - Ross

4 7

FW: Boost Digest, Vol 2929, Issue 4
by Bartlett, Roscoe A 07 Jun '10

07 Jun '10

David, I have some responses below ... > ------------------------------ > > Message: 13 > Date: Tue, 01 Jun 2010 16:16:25 -0400 > From: David Abrahams <dave(a)boostpro.com> > To: boost(a)lists.boost.org > Cc: "mathias.gaunard(a)ens-lyon.org" <mathias.gaunard(a)ens-lyon.org> > Subject: Re: [boost] Review of a safer memory management approach for > C++? > Message-ID: <m2hblmcxuu.wl%dave(a)boostpro.com> > Content-Type: text/plain; charset=UTF-8 > > ... > > > > At Thu, 27 May 2010 20:02:45 +0200, > > > Ingo Loehken wrote: > > > > > > > > if I understand "hard to reason about" in the right why : like > there > > > > is no need for shared ownership at all, this also means that > there > > > > is no use for COM Programming - and of course there is. > > > > > > I think you don't understand it the right way. Shared ownership > (at > > > least in the presence of mutation) is hard to reason about because > > > seemingly-local modifications can have non-local effects. > > > > [Bartlett, Roscoe A] > > > > Yes, but let's focus on the "essential complexity" of object sharing > > Okay. > > > and not get bogged down in the "accidental complexity" of memory > > management. The approaches described in > > > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf > > > > make explicit the "essential complexity" of object sharing while > > trying to eliminate the "accidental complexity" of memory management > > (see Section 6.1 in > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf) > > I'm all for making essential complexity obvious and eliminating > accidental complexity, but I'm not sure I see a compelling example > that Teuchos does that, at least not when compared with what I > consider normal state-of-the-art C++ programming, which is mostly > value-based, dynamic allocations are rare, and those that occur are > immediately managed, e.g. by appropriate smart pointers. [Bartlett, Roscoe A] I don't want to get into an argument about the "proper" way to use C++ here (but I will a little below). In Item 1 of "Effective C++, 3rd Edition", Scott Meyers identifies four different major programming paradigms that are supposed to be supported by C++: 1) C: raw pointers, low-level built-in datatypes (int, double, etc.), statements, blocks, functions etc. 2) Object-oriented C++: Inheritance, virtual functions, dynamic allocation, RTTI, etc. (i.e. runtime polymorphism) 3) Template C++: Typical generic programming, template metaprogramming, etc. (i.e. compile-time polymorphism) 4) STL: Containers, iterators, algorithms, etc. and code like this (not just using STL) (closely related to #3 Template C++). Of course any well written large C++ program is a mixture of all four of the above (and more). What you seem to be arguing above is that most good C++ software will use templates in #3 and #4 but almost none of #2 object-oriented C++. Note that C++ was first designed in the early 80s primarily to support #2 object-oriented C++. I can see the reason for this bias for #3 and #4 given the nature of most of the Boost libraries and you can have that opinion if you would like. However, there are clearly types of programming problems that call for more of #2 and less of #3 and #4 in the overall design. Languages like Java and Python would be typically used to write safe object-oriented programs in a productive environment (free of complications of dynamic memory allocation and most undefined behavior that is so easy to expose with current C++ approaches). I will argue below that more C++ programs are better served by using more of #2 object-oriented C++ and to a lesser extent template-based methods in #3 and #4 (except where it is called for in lower-level code). Instead of arguing if we should be using #2 object-oriented C++ (runtime polymorphism) or instead prefer #3 template programming (static polymorphism), let's assume that we want C++ to do a good job of supporting #2 object-oriented C++ if we decide to use it. Therefore, the real question is if C++ is indeed supposed to support #2 object-oriented programming, does C++ it support it well enough so that I would choose to use C++ instead of Java or Python (assuming my programming team knows those languages)? By all reasonable measures, writing object-oriented programs in Java or Python is more productive because of garbage collection and the elimination of undefined behavior that is so common in C++ programs (I can find lots of references for these opinions and even data from studies to support this claim). However, to achieve performance in CSE software, I have to write some code in C/C++ or some other appropriate language. No-one has been able to demonstrate sufficient performance for CSE codes written entirely in Java or Python. Also, many of the "safer" and "more productive" OO languages (e.g. Java and Python) are not even available on some high-end HPC platforms. Even if they were, mixed language programming has lots of problems that discourages wide-spread use. That leaves C++ as the only fully portable viable language for high-performance CSE (sorry Fortran programmers). I would argue that the current accepted approaches to writing OO programs in C++ (i.e. #2 OO C++ features) make it too easy to create programs with undefined behavior (i.e. segfaults or worse) and as a result, the paranoia of undefined behavior in C++, memory leaks etc. lead to lots of bad programming practices and designs (see Section 1 in http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf) What the approach described in the Teuchos MM report tries to do is to create a set of classes and idioms that can allow C++ to be used to safely and productively develop #2 OO C++ software closer to the safety and productively that you can get with languages like Java and Python. However, there is still too much "accidental complexity" in C++ to have it ever fully compete with productivity of writing in Java and Python for basic OO features (IMO). In any case, C++ offers other advantages over any other language that would have you choose it over these other languages. > > Trying to over design a program to avoid all shared ownership is > > what make C++ programming so unproductive and has all the negative > > consequences described in Section 1 in: > > > > http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf > > > > Designs with object sharing can be much less complex overall than > > designs without sharing. You just need decent tools to detect circular > > reference problems (and that is what the Teuchos::RCP class has). > > Well, I fundamentally disagree with all of the above. Overuse of > runtime polymorphism, thus dynamic allocation, and thus shared > ownership (it's almost an inevitable progression) is one of the things > that has made C++ programming unproductive, and those people I know > who most zealously avoid it tend to be more productive than everyone > else. IMO. [Bartlett, Roscoe A] When runtime performance or other issues related to dynamic allocations and classic OO are not a problem, classic OO C++ programs using runtime polymorphism are typically superior to highly templated C++ programs using static polymorphism for the following (indisputable) reasons: 1) Well written OO C++ software builds and (more importantly) rebuilds *much* faster then heavily template code. This massively speeds up the development cycle. The difference can be orders of magnitude of time savings in recompile/relink times. I have seen cases where carefully using runtime OO can result in recompile/relink cycles take less than 20 seconds where using all template code would take 20 minutes. I am *not* exaggerating here at all. 2) Templated programs can result in massive object-code bloat over equivalent runtime OO programs. If you are really sloppy with templates and use it for everything, you can easily double, triple, or more the size of the object code and executables (see Item 44 in "Effective C++, 3ed edition"). 3) The error messages you get when mistakes are made when using interfaces and virtual functions are far superior to the mess that most compliers put out when you make a mistake with implicit compile-time template interfaces. This is a huge issue. Runtime errors, as opposed to cryptic compile-time errors, can be handled by throwing exceptions with very good error messages that programmers can 100% control the quality of. Yes, of course you prefer compile-time errors to runtime errors all things being equal, and am not arguing with that. This is generally not true of template programming (but there are some tricks you can use to improve compile-time error messages, but not perfectly). For example, if you make a mistake with STL iterators/algorithms you can get very cryptic error messages that dumfound most (non-expert and expert) C++ programmers. 4) Well written OO C++ software allows for significant runtime flexibility. If you use dynamic libraries, for instance, you can add new implementations of objects without even having to relink customer executables. Also, you can fix many types of bugs in updated shared libraries without having to recompile/relink customer executables. In a large-scale software setting, this can be very helpful. Believe me, I see the use of template C++ code with lots of static typing and lots of local logic and control but it is not the end all and will never replace classic OO C++ for most high-level designs and software architecture. I challenge anyone to refute this. C++ compilers will need to become many orders of magnitude faster that what they are today for this fact to change (but then the software will just grow that much larger and erase the compiler speed improvements). > In my experience, designs with object sharing tend to increase > complexity in all kinds of ways beyond the memory management issues > that you describe. Perhaps the most important way is, they tend to > obscure the overall computation being performed by hiding it in > networks of interacting objects. [Bartlett, Roscoe A] This is not going to magically change by templating everything. If a lot of objects collaborate together to perform computations, it does not matter if you use static or runtime polymorphism; the code will be hard to figure out and you will need print statements and/or run it in a debugger to see what is really happening. > I would like to see an example of a design with shared object > ownership that is ?much less complex? than all other functionally > equivalent designs that eschew shared ownership. I'm not saying such > applications don't exist, but IME they are rare, and a concrete > example would help to bring this whole discussion back to earth. [Bartlett, Roscoe A] One design where shared mutable objects likely makes things overall simpler and more natural is the PDE expression mechanism in the Sundance code (http://www.math.ttu.edu/~klong/Sundance/html/) If the clients all expect remote updates of shared objects, then things work great. The problem is when an object is changed by another client and you don't expect it to change (i.e. the classic problem of change propagation). As for sharing of objects, it is critical for many types of software in the Computational Science and Engineering (CSE) domain. You have to share large objects in order to cut down on runtime and storage overhead. As one example, consider the Petra object model with the Trilinos packages Epetra and Tpetra (http://trilinos.sandia.gov/) In the Petra object model, you have objects with significant storage overhead like Maps and Graphs that are shared between many different objects like Vectors, MultiVectors, and Matrices. Yes, you could create deep copies of Maps and Graphs but that would waste a large amount of storage. The bottom line is that by sharing large objects like Maps and Graphs instead of making large numbers of copies, we can fit and solve much larger simulation problems on a computer than we could otherwise. You could template and statically allocate everything and you will never get around the fundamental need to share the data contained in large objects in these types of applications. Note, however, that Maps in Epetra/Tpetra are shared as const immutable objects. That means that no-one can change a Map after it is first created. Therefore, all of the problems with unexpected updates goes away (as you mention above). However, the shared Maps must go away when they are not needed anymore and that is what the Teuchos::RCP class makes easy and robust. Under the covers, Tpetra uses Tpetra::ArrayRCP for similar purposes and much more. The situation with Graphs is different and issues of change propagation of shared objects are still present in some cases. Come to think of it, most of the object sharing that used in the CSE software that I use and write mostly just share const immutable objects so problems of change propagation mostly goes away. However, there are some important use cases where all of the clients are not holding RCPs to const objects and the problem of change propagation remains. Again, to save in significant runtime and storage overhead, we can't just create deep copies of all of these objects. Templating and purely stack-based programs are not going to solve that problem. The Teuchos MM approach, I believe, largely solves the memory management problems with dynamic memory allocation and object sharing while still yielding very high performance and mostly eliminating undefined behavior associated with the incorrect usage of memory. The most significant contribution of the Teuchos MM approach is the presents of Teuchos::Ptr and Teuchos::ArrayView and how they interact with the other classes. Cheers, - Ross

6 9

How to retrieve original string back from hashed string using string_hash()?
by Tiwari, Sumit [Tech] 07 Jun '10

07 Jun '10

Hi, I can hash a string using string_hash(), is there any function available to get back the string value back by supplying hash value? Thanks, Sumit

2 1

Re: [boost] Review of a safer memory management approach for
by Bartlett, Roscoe A 07 Jun '10

07 Jun '10

Fernando and Jeffrey, A few responses ... > ------------------------------ > > Message: 6 > Date: Fri, 04 Jun 2010 00:14:25 -0300 > From: Fernando Cacciola <fernando.cacciola(a)gmail.com> > To: boost(a)lists.boost.org > Subject: Re: [boost] Review of a safer memory management approach for > C++? > Message-ID: <hu9r1a$57q$1(a)dough.gmane.org> > Content-Type: text/plain; charset=UTF-8; format=flowed > > ... > > For instance, it's been so long since I last wrote a virtual function > that > recently, I made the silly mistake of forgetting that, in a derived > class, an > overrided function doesn't have to specify the virtual keyword ;) [Bartlett, Roscoe A] High warning levels (i.e. with GCC) catch most such mistakes that the language proper allows to go through. BTW, are you gravitating to stack-based programming because of the memory management problems or because of problems of change propagation of shared mutable objects? These are almost 100% unrelated issues in my opinion. Is the C++ boost community really ready to throw away OO programming? That just seems crazy to me. > ------------------------------ > > Message: 9 > Date: Thu, 03 Jun 2010 21:12:52 -0700 > From: "Jeffrey Lee Hellrung, Jr." <jhellrung(a)ucla.edu> > To: boost(a)lists.boost.org > Subject: Re: [boost] FW: Boost Digest, Vol 2929, Issue 4 > Message-ID: <4C087D44.1080909(a)ucla.edu> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > ... > > > As for sharing of objects, it is critical for many types of software > in the Computational Science and Engineering (CSE) domain. You have to > share large objects in order to cut down on runtime and storage > overhead. As one example, consider the Petra object model with the > Trilinos packages Epetra and Tpetra (http://trilinos.sandia.gov/) In > the Petra object model, you have objects with significant storage > overhead like Maps and Graphs that are shared between many different > objects like Vectors, MultiVectors, and Matrices. Yes, you could > create deep copies of Maps and Graphs but that would waste a large > amount of storage. The bottom line is that by sharing large objects > like Maps and Graphs instead of making large numbers of copies, we can > fit and solve much larger simulation problems on a computer than we > could otherwise. You could template and statically allocate everything > and you will never get around the fundamental need to share the data > contained in large objects > in > > these types of applications. > > > > Note, however, that Maps in Epetra/Tpetra are shared as const > immutable objects. That means that no-one can change a Map after it is > first created. Therefore, all of the problems with unexpected updates > goes away (as you mention above). However, the shared Maps must go > away when they are not needed anymore and that is what the Teuchos::RCP > class makes easy and robust. Under the covers, Tpetra uses > Tpetra::ArrayRCP for similar purposes and much more. The situation > with Graphs is different and issues of change propagation of shared > objects are still present in some cases. > > > > Come to think of it, most of the object sharing that used in the CSE > software that I use and write mostly just share const immutable objects > so problems of change propagation mostly goes away. However, there are > some important use cases where all of the clients are not holding RCPs > to const objects and the problem of change propagation remains. Again, > to save in significant runtime and storage overhead, we can't just > create deep copies of all of these objects. Templating and purely > stack-based programs are not going to solve that problem. > > > > The Teuchos MM approach, I believe, largely solves the memory > management problems with dynamic memory allocation and object sharing > while still yielding very high performance and mostly eliminating > undefined behavior associated with the incorrect usage of memory. The > most significant contribution of the Teuchos MM approach is the > presents of Teuchos::Ptr and Teuchos::ArrayView and how they interact > with the other classes. > > I don't see how shared object *ownership* is necessary in this > example... :/ I don't think Dave was arguing that holding large > objects > by reference, or even dynamically allocating objects (large or not), is > necessarily complex or "bad"... > > Or did I miss something? [Bartlett, Roscoe A] Have you read the key sections in the Teuchos MM approach in http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf or are you just responding to the list emails? You should *never* hold an object by reference as a class data-member (except perhaps in very well encapsulated very low-level code). That is what leads to undefined behavior with dangling references. Early versions of Epetra did just that and it blew up in people's faces. Consider writing a factory method such as: Epetra_CrsMatrix* createMyMatrix(....) { Epetra_Map rowMap(...); Epetra_Map colMap(...); ... Epetra_CrsGraph graph(rowMap, colMap); ... Epetra_CrsMatrix *matrix = new Epetra_CrsMatrix(graph); ... return matrix; } In early Versions of Epetra, maps and graphs were held and references and the above factory function returned an Epetra_CrsMatrix object with now deleted map and graph objects. Later, Epetra was refactored to make Epetra_Map and Epetra_Graph handle classes with internal reference-counted handle/body implementation objects. This fixed the problem but it create lots of new problems including that now copying graphs silently went from deep copy to shallow copy (which broke some code in some rare cases). The new Tpetra package has gone the down the road of making shared ownership explicit by exposing RCP-wrapped objects making factory functions similar to the above safe and obvious in behavior. It is working very well. Cheers, - Ross

8 8