This may have been discussed earlier and been proved impractical or impossible. I'm a big fan of the boost library; high-quality; great utility; great range. What's not to like? Unfortunately while I've been trying to introduce boost at the places I've been working on I find that the sheer scope of the boost library makes it discouraging. Often it's desirable to just demand a certain compiler and build enviroment (gcc or Visual Studio) then the developers run check out from subversion (for instance) and run the build script and they are on the way. boost isn't as easy as it could be IMO (due to its size and scope) to include into an existing project with many developers (many who might be unwilling to receive something that complicate the build enviroment). So let's take the project I'm working on now as an example. What I would like to do is just to download and use the smart pointers in boost to introduce the other developers to the possibility of reusing high-quality smart pointers instead of writing your own or worse not use smart pointers at all. In my world I would then like to download the smart pointer boost package (with dependencies?) which should be a fairly small download. Due to a more manageable size I would then foresee that it would easier for me to get it into the source control system and into the build environment. Also, it would be easier for me to convince the systems architect that boost is worth more than the hassle it introduces (right now I can't). Later when people see the benefit of boost and we want to do some text parsing I'd like to download the boost.spirit package and set it up. And so on. I would also think that packaging boost in different packages (a small base package and then perhaps each individual library as a package) would give benefits to the development of the new releases. Right now I have the feeling that getting a new release is rather daunting task. Which I guess comes from boost being a monolithic, large library that supports many different compilers and platforms. By packaging boost into several different packages one could perhaps have different release schedules for different packages? Different supported compilers on different packages ( ie VC6 I guess is a pain to get working with mpl but shared_ptr<> should work rather straightforward with that aging but still widely used compiler)? What do you think? I realize that this would be a rather daunting task in itself (impossible?) since I guess that the dependencies between the different libraries in boost might not be 100% documented and well understood. I do however think that there would be benefits in terms of spreading the popularity of boost and managing boost releases if it could be done. Regards, Mårten Rånge
I have never used it, but I found a tool which could help you http://www.boost.org/tools/bcp/bcp.html PS: Since I use a precompiled package of Boost, I know nothing about Boost build system and Jam files. Best Regards, Manuel Fiorelli www.fioreltech.net
on Sun Apr 01 2007, "Mårten Rånge" <marten.range-AT-gmail.com> wrote:
In my world I would then like to download the smart pointer boost package (with dependencies?) which should be a fairly small download. Due to a more manageable size I would then foresee that it would easier for me to get it into the source control system and into the build environment. Also, it would be easier for me to convince the systems architect that boost is worth more than the hassle it introduces (right now I can't).
As mentioned elsewhere by Manuel Fiorelli, http://www.boost.org/tools/bcp/bcp.html
Later when people see the benefit of boost and we want to do some text parsing I'd like to download the boost.spirit package and set it up.
Likewise.
And so on.
I would also think that packaging boost in different packages (a small base package and then perhaps each individual library as a package) would give benefits to the development of the new releases. Right now I have the feeling that getting a new release is rather daunting task. Which I guess comes from boost being a monolithic, large library that supports many different compilers and platforms.
No, boost is not even one library much less a monolithic one. Boost is a library collection. I've seen the dependencies among the Boost libraries graphed, and unlike some other development frameworks, which just show up as a big blob, Boost is neatly partitioned and the dependencies tend to be a unidirectional DAG.
By packaging boost into several different packages one could perhaps have different release schedules for different packages?
We've been over that question many times, and many of us disagree. Though Boost is not monolithic, there are enough dependencies among libraries that it's important they can all be released together in a state where they interoperate correctly. We do have a plan to speed up releases. Please see this thread: http://archives.free.net.ph/thread/20060514.212258.a8a17ce6.en.html Unfortunately the link to beman's original proposal is dead :(. Beman, do you think you could make that available again? Maybe just post the text to this list? As 1.34 approaches it's probably a good idea to get that back into the collective consciousness. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
David Abrahams wrote:
on Sun Apr 01 2007, "Mårten Rånge" <marten.range-AT-gmail.com> wrote:
I would also think that packaging boost in different packages (a small base package and then perhaps each individual library as a package) would give benefits to the development of the new releases. Right now I have the feeling that getting a new release is rather daunting task. Which I guess comes from boost being a monolithic, large library that supports many different compilers and platforms.
No, boost is not even one library much less a monolithic one. Boost is a library collection. I've seen the dependencies among the Boost libraries graphed, and unlike some other development frameworks, which just show up as a big blob, Boost is neatly partitioned and the dependencies tend to be a unidirectional DAG.
Whether boost is 'monolithic' or a 'library collection' seems to me a rather academic question, at least as far as boost users are concerned, as they can't download and install individual libraries. As far as the release process is concerned, it would certainly help if the regression testing would be at least somewhat modular. Having test runs involve not only all libraries, but also all build variants and all (available) toolchains is a very significant slowdown. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
on Sun Apr 01 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
David Abrahams wrote:
on Sun Apr 01 2007, "Mårten Rånge" <marten.range-AT-gmail.com> wrote:
Whether boost is 'monolithic' or a 'library collection' seems to me a rather academic question, at least as far as boost users are concerned, as they can't download and install individual libraries.
They can only download the whole thing at once, but as for installation, if they want a subset of headers, what's wrong with bcp? If they want a subset of binaries, what's wrong with our bjam/configure options?
As far as the release process is concerned, it would certainly help if the regression testing would be at least somewhat modular. Having test runs involve not only all libraries, but also all build variants and all (available) toolchains is a very significant slowdown.
Unfortunately, I don't believe all build variants _are_ tested. But anyway, I don't understand what you mean about modularity and slowdowns. The tests are distributed across many machines. If you dedicate a single testing machine to one toolchain and build variant, you can't do this much faster. Many testers do incremental testing, so only the changed stuff gets rebuilt. Anyway, I hope after 1.34 ships we can get to work on a system for testing much more effectively. The current one is definitely showing its age. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
David Abrahams wrote:
on Sun Apr 01 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
David Abrahams wrote:
on Sun Apr 01 2007, "Mårten Rånge" <marten.range-AT-gmail.com> wrote: Whether boost is 'monolithic' or a 'library collection' seems to me a rather academic question, at least as far as boost users are concerned, as they can't download and install individual libraries.
They can only download the whole thing at once, but as for installation, if they want a subset of headers, what's wrong with bcp? If they want a subset of binaries, what's wrong with our bjam/configure options?
How many boost users are actually building boost themselves, compared to those who use prepackaged 'system' libraries ? (And from those who do build themselves, how many are sufficiently familiar with the infrastructure to know how to use these fine tools well (compared to those who are very happy to just be able to get the whole thing up and running so they don't need to touch it ever again) ?
As far as the release process is concerned, it would certainly help if the regression testing would be at least somewhat modular. Having test runs involve not only all libraries, but also all build variants and all (available) toolchains is a very significant slowdown.
Unfortunately, I don't believe all build variants _are_ tested. But anyway, I don't understand what you mean about modularity and slowdowns. The tests are distributed across many machines. If you dedicate a single testing machine to one toolchain and build variant, you can't do this much faster. Many testers do incremental testing, so only the changed stuff gets rebuilt.
You are right, the total execution time is still the same. However, a more modular system is more easily parallelizable. More testers could contribute cycles as the resource requirements wouldn't be quite as high. This, then, makes the cycles from checkin to report containing associated test results smaller, helping to get fixes in quicker. Etc. etc.
Anyway, I hope after 1.34 ships we can get to work on a system for testing much more effectively. The current one is definitely showing its age.
That would be good indeed. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
on Sun Apr 01 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
Unfortunately, I don't believe all build variants _are_ tested. But anyway, I don't understand what you mean about modularity and slowdowns. The tests are distributed across many machines. If you dedicate a single testing machine to one toolchain and build variant, you can't do this much faster. Many testers do incremental testing, so only the changed stuff gets rebuilt.
You are right, the total execution time is still the same. However, a more modular system is more easily parallelizable.
I still don't know what you mean by "more modular"
More testers could contribute cycles as the resource requirements wouldn't be quite as high. This, then, makes the cycles from checkin to report containing associated test results smaller, helping to get fixes in quicker. Etc. etc.
Maybe you're suggesting that tests of the whole suite on a single compiler and platform could be distributed across many machines? That could be vulnerable to small platform differences, but maybe there's a way around that. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
David Abrahams wrote:
on Sun Apr 01 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
Unfortunately, I don't believe all build variants _are_ tested. But anyway, I don't understand what you mean about modularity and slowdowns. The tests are distributed across many machines. If you dedicate a single testing machine to one toolchain and build variant, you can't do this much faster. Many testers do incremental testing, so only the changed stuff gets rebuilt. You are right, the total execution time is still the same. However, a more modular system is more easily parallelizable.
I still don't know what you mean by "more modular"
I mean a system that provides many smaller test suites, so users can offer to run them individually. (That would give the additional benefit of test-suite specific parametrization. For example, boost.python may be tested against different python versions, something that doesn't make any sense for any other part of boost.)
More testers could contribute cycles as the resource requirements wouldn't be quite as high. This, then, makes the cycles from checkin to report containing associated test results smaller, helping to get fixes in quicker. Etc. etc.
Maybe you're suggesting that tests of the whole suite on a single compiler and platform could be distributed across many machines? That could be vulnerable to small platform differences, but maybe there's a way around that.
If there are potentially 'small platform differences' they need to be captured by the testing harness anyway. Right now we get potentially multiple test runs with the same label (i.e. same toolchain / platform), with potentially differing results. One way to enhance the testing harness is to more strictly control the environment test suites are executed in. buildbot (http://buildbot.sourceforge.net/) would provide excellent tools to move into that direction. (Rene has been suggesting to do that for a long time, for example here: http://article.gmane.org/gmane.comp.lib.boost.devel/119457) Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin...
on Sun Apr 01 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
David Abrahams wrote:
on Sun Apr 01 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
Unfortunately, I don't believe all build variants _are_ tested. But anyway, I don't understand what you mean about modularity and slowdowns. The tests are distributed across many machines. If you dedicate a single testing machine to one toolchain and build variant, you can't do this much faster. Many testers do incremental testing, so only the changed stuff gets rebuilt. You are right, the total execution time is still the same. However, a more modular system is more easily parallelizable.
I still don't know what you mean by "more modular"
I mean a system that provides many smaller test suites,
We already have that. Each library has its own test suite. What we don't have is any way to partition Boost among different testers.
so users can offer to run them individually. (That would give the additional benefit of test-suite specific parametrization. For example, boost.python may be tested against different python versions, something that doesn't make any sense for any other part of boost.)
Agreed.
More testers could contribute cycles as the resource requirements wouldn't be quite as high. This, then, makes the cycles from checkin to report containing associated test results smaller, helping to get fixes in quicker. Etc. etc.
Maybe you're suggesting that tests of the whole suite on a single compiler and platform could be distributed across many machines? That could be vulnerable to small platform differences, but maybe there's a way around that.
If there are potentially 'small platform differences' they need to be captured by the testing harness anyway. Right now we get potentially multiple test runs with the same label (i.e. same toolchain / platform), with potentially differing results. One way to enhance the testing harness is to more strictly control the environment test suites are executed in. buildbot (http://buildbot.sourceforge.net/) would provide excellent tools to move into that direction. (Rene has been suggesting to do that for a long time, for example here: http://article.gmane.org/gmane.comp.lib.boost.devel/119457)
I hope you'll be at BoostCon; I'd like to get your ideas in the mix for a discussion of (and maybe a sprint on) the testing architecture. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
On Sun, 01 Apr 2007 21:18:59 -0400 Stefan Seefeld <seefeld@sympatico.ca> wrote:
How many boost users are actually building boost themselves, compared to those who use prepackaged 'system' libraries ? (And from those who do build themselves, how many are sufficiently familiar with the infrastructure to know how to use these fine tools well (compared to those who are very happy to just be able to get the whole thing up and running so they don't need to touch it ever again) ?
We build it ourselves. Well, I am actually the one who got it all working for our environment, and am supposed to make sure it stays that way. 1.34 scares me to death. I had barely come to terms with the highly unusual build environment of bjam. The vast amount of effort and changes necessary to support the new build environment has me extremely nervous. I imagine the only way we will move to 1.34 is when we MUST have something in there... or unless someone can convince me that my fears are nothing more than those children have of the boogeyman.
If you really want a poll, I do build boost myself. I can honestly say that I am not familiar enough with bjam even after a really long time tinkering with it (for what thats worth). Currently I build boost as part of the build on one project and use pre-built on another. I'm finding building it myself (using my own build system) is a lot better as I can apply stuff like profiling flags or changing compilers on a whim. To build boost I did need to make sure my build system could handle the same sort of things that bjam does (which it did). Personally, I don't care if boost takes 2 years to release, so long as the release is solid and after being on this list, I don't think there is any fear of that. PS: Congratulations on 1 remaining test failure! -----Original Message----- From: boost-bounces@lists.boost.org on behalf of Stefan Seefeld Sent: Sun 4/1/2007 6:18 PM To: boost@lists.boost.org Subject: Re: [boost] Boost packages? How many boost users are actually building boost themselves, compared to those who use prepackaged 'system' libraries ? (And from those who do build themselves, how many are sufficiently familiar with the infrastructure to know how to use these fine tools well (compared to those who are very happy to just be able to get the whole thing up and running so they don't need to touch it ever again) ?
Sohail Somani wrote:
If you really want a poll, I do build boost myself. I can honestly say that I am not familiar enough with bjam even after a really long time tinkering with it (for what thats worth). Currently I build boost as part of the build on one project and use pre-built on another. I'm finding building it myself (using my own build system) is a lot better as I can apply stuff like profiling flags or changing compilers on a whim. To build boost I did need to make sure my build system could handle the same sort of things that bjam does (which it did).
Personally, I don't care if boost takes 2 years to release, so long as the release is solid and after being on this list, I don't think there is any fear of that.
It is a problem when Boost takes a long time between releases, such as the current 15 months so far since 1.33.1. Of course it is great that the releases are solid, but the time between major releases is also an issue. Most organizations will not deploy Boost from source code changes, currently on CVS. Furthermore organizations may want to use new libraries/features of Boost but, following the previous proviso, can not get them until a new release with them is ready. Other implementors of 3rd party libraries, which use Boost libraries under the cover, may be very reticent to make changes to their 3rd party library while fixes between releases are being applied to Boost as a whole, while testing is still ongoing, so they too will wait for the next release. Following Beman Dawes suggestion I think it would be advantageous to consider officially releasing individual Boost libraries and/or dependent groups of libraries before the wholesale release of all of Boost, although I understand the confusion this might cause with incompatible sets of libraries on an end users machine and the reticence Boost might have in doing so. But I think this problem could be largely overcome by some form of version tagging of each library, purely for the end users benefit, and by insisting that the release of any individual library, between full releases, always include the release of all dependent libraries which have changed since the last full release at the same time. Obviously this could not be done very easily for some libraries which depend on many other libraries, but for some of the libraries which are highly independent of others I think this would work well, and provide the ability for end users to pick up useful changes in a particular library before the next full release. I would say that new libraries added to Boost between full releases could also be distributed in this way so that end users would not have to wait for the next full release to use the new library. This would of course mean that someone might have version 1.34, let's say, of the full Boost on their machine but have version 1.34.1 of library X and version 1.34.2 of library Y etc. But if 1.34.1 of library X were guaranteed fo work with 1.34,2 of library Y and 1.34 of everything else, I see nothing wrong with this schema.
On Sun, 01 Apr 2007 20:13:47 -0400 David Abrahams <dave@boost-consulting.com> wrote:
We do have a plan to speed up releases. Please see this thread: http://archives.free.net.ph/thread/20060514.212258.a8a17ce6.en.html Unfortunately the link to beman's original proposal is dead :(. Beman, do you think you could make that available again? Maybe just post the text to this list? As 1.34 approaches it's probably a good idea to get that back into the collective consciousness.
Dave, Is it a good idea to list the problems with the 1.34 release? I'm not trying to throw stones at all, and I'm not looking for names. I'm more interested in the issues so they can be addressed in some way. I just think it may help with future efforts, and may also help others understand what went on. I've learned a whole lot more about release management from the things that went wrong (or at least appeared to do so) than anything that happened to go right.
on Sun Apr 01 2007, Jody Hagins <jody-boost-011304-AT-atdesk.com> wrote:
On Sun, 01 Apr 2007 20:13:47 -0400 David Abrahams <dave@boost-consulting.com> wrote:
We do have a plan to speed up releases. Please see this thread: http://archives.free.net.ph/thread/20060514.212258.a8a17ce6.en.html Unfortunately the link to beman's original proposal is dead :(. Beman, do you think you could make that available again? Maybe just post the text to this list? As 1.34 approaches it's probably a good idea to get that back into the collective consciousness.
Dave,
Is it a good idea to list the problems with the 1.34 release? I'm not trying to throw stones at all, and I'm not looking for names. I'm more interested in the issues so they can be addressed in some way. I just think it may help with future efforts, and may also help others understand what went on.
It's a free country :) Feel free to start any thread you want. It might be better to wait until after 1.34 for a postmortem, but that's just an idea. If you want to discuss it now, be my guest.
I've learned a whole lot more about release management from the things that went wrong (or at least appeared to do so) than anything that happened to go right.
I'm sure a discussion of release management by someone who has learned a few things would be useful. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
David Abrahams wrote:
No, boost is not even one library much less a monolithic one. Boost is a library collection. I've seen the dependencies among the Boost libraries graphed, and unlike some other development frameworks, which just show up as a big blob, Boost is neatly partitioned and the dependencies tend to be a unidirectional DAG.
Really? I very much would like to see this graph. If this is available somewhere I think it would be nice to show it somewhere on the web! Besides being a nice eye-candy it would support those in need for these kind of arguments. I would have volunteered in providing this graph, but unfortunately I must have been doing something completely wrong when I was trying to generate such a graph on my own. It reminded me very much of the wool ball my cat likes to play with than a well structured graph. :-( Roland
on Mon Apr 02 2007, Roland Schwarz <roland.schwarz-AT-chello.at> wrote:
David Abrahams wrote:
No, boost is not even one library much less a monolithic one. Boost is a library collection. I've seen the dependencies among the Boost libraries graphed, and unlike some other development frameworks, which just show up as a big blob, Boost is neatly partitioned and the dependencies tend to be a unidirectional DAG.
Really? I very much would like to see this graph.
It was in a preview of code structure analysis software from this company: http://headwaysoft.com/index.php. They were interested in making an analysis of Boost publicly (I think) available online. I've Bcc'd my friend there; if he wants to pipe up, he will (Paul, you have to register if you want to post on this mailing list).
If this is available somewhere I think it would be nice to show it somewhere on the web!
Agreed, for real.
Besides being a nice eye-candy it would support those in need for these kind of arguments.
Ditto.
I would have volunteered in providing this graph, but unfortunately I must have been doing something completely wrong when I was trying to generate such a graph on my own. It reminded me very much of the wool ball my cat likes to play with than a well structured graph. :-(
Well, at the time, Headway's software really did show a remarkable difference between Boost and ACE in that respect. -- Dave Abrahams Boost Consulting www.boost-consulting.com Don't Miss BoostCon 2007! ==> http://www.boostcon.com
participants (8)
-
David Abrahams -
Edward Diener -
Jody Hagins -
Manuel Fiorelli -
Mårten Rånge -
Roland Schwarz -
Sohail Somani -
Stefan Seefeld