Re: [boost] CMake - one more time

Subject: Re: [boost] CMake - one more time
From: Paul Fultz II (pfultz2_at_[hidden])
Date: 2016-04-22 13:42:37

> On Apr 20, 2016, at 4:39 PM, Raffi Enficiaud <raffi.enficiaud_at_[hidden]> wrote:
> 
> Le 20/04/16 17:49, Robert Ramey a écrit :
>> Inspired by the recent discussion regarding CMake, I've spend some more
>> time looking at this.  As a result, I've done the following.
> 
> Hi,
> Thanks for summarizing this very very (very) long thread.
> 
>> 
>> a) I experimented with using CMake in the more customary manner in my
>> canonical project Safe Numerics - (see Boost Library Incubator). The
>> current version does distribute CMakeLists.txt around the subdirectories
>> of the library root.  My original objections to doing this were assuaged
>> when I realized that boost build does the same thing.  That is I
>> typically have Jamfiles in side of test, example and sometimes other
>> directories.
>> 
>> b) One of the features that I much like about Boost Build is the
>> possibility of arbitrary nesting of Jamfiles.  This permits one to
>> build/test a portion of one's library (e.g. test, example) as well as
>> the whole library.  For a large library this is indispensable. In spite
>> of what many people think, this is not easily supported with CMake.
>> However, I have found that with some extra effort, it is possible to
>> support this to the extent we need it.  So in this project ONE CAN BUILD
>> (actually create an IDE project or makefile that builds) ANY OF THE
>> "SUBPROJECTS" (TEST, EXAMPLE) AS WELL AS THE LIBRARY "SUPERPROJECT"
> 
> If you think about nesting as being able to run something like "make doc" from any library, then yes, cmake is definitely lacking that. OTOH, I have the feeling that:
> 1- everything in the current boost is coupled to the build system. I read ppl wanting to be modular, but it is given the fact that there is an adequate build system
> 
> 2- b2 is a bit cheating: it knows the full "namespace" (it flattens it) and can then eg. sort out the targets dependencies and build order. So even if you think that by typing "make doc" in your library, you are hitting only the build system (boostbook/doxygen toolchain, etc) plus your library, this is wrong: every target in b2 has the knowledge of the full DAG of dependencies, which makes it highly coupled to any other library, at least to the superproject.
> In CMake this nesting of CMakeLists is more compartmented: one CMakeList is supposed to be more or less independent from its siblings (but not from the parent). This is also what would make the transition of the super project to CMake very difficult: for instance dependencies have to be explicitly stated in a main orchestrating CMakeLists.txt (in b2 this is I believe implicitly done, certainly is several parsing passes).
> 
> 3- b2 imposes a structure of directories as well: for instance, if I do
> """
> using quickbook ;
> using doxygen ;
> using boostbook ;
> """
> those features should be in files relative to some paths of the b2 location wrt. the superproject (please correct me if I am wrong). Also, when I "make" a library, it goes magically to the bin.v2 folder of the root of the superproject. I have the feeling that some behaviour of b2 in terms of relative paths are hard-coded somewhere.
> That is to say that: "cd doc; b2" is not at all independent from the superproject folder structure, and the apparent modularity is not a real one. But again, I agree that is just works for the purpose of boost, it is just tightly coupled to hidden things.
> 
> 4- It is in fact - I believe - possible to do a "cd doc; make" with CMake, but not from the source tree, but from a subfolder of the build tree. You have to generate the superproject (or part of it) first though. OTOH, this first generation of the superproject is done intrinsically by b2 anyway: b2 does it in memory right before executing your command (look at the time it takes before starting processing the "cd doc; b2"). So if we think about nesting: CMake is better in the sense that it has a hierarchy of project, so the dependencies imposed to be a tree, while b2 fakes (to my opinion) the nesting (apparent hierarchy) and flattens everything to extract a DAG.
> 
> 
>> c) CMake is indispensable to me:
>> 
>> * it creates IDE projects for "any?" platform.  I use IDE alot.
>> * everyone else uses it - so it makes it easier to promote my work to
>> others.
>> * it's easier to make work - still a pain though
>> * There is lots of information, around the net about how to use CMake,
>> how easy it is etc.  Although they help when you're looking for an
>> answer (which is all the time) - they really betray how complex,
>> arbitrary and complex the system is.
>> * It has an almost of idiot proof GUI version which I use a lot.  I
>> really like this.
>> * CMake is well maintained and supported by it's promoters.
> 
> So what I do now is also that: I am maintaining my CMakeLists.txt, for the purpose of having a proper development environment, but it has no other purpose at all. Also I have to say that it does not work well with this horrible "b2 header": in my cmake, I am hitting the headers of the library (in libX/include), and not of the main superproject. My IDE shows me 2 different files because of that. But apart from that, it is good enough (and yes, I have one CMakeLists.txt for the build+doc+test, I do not see any good reason for splitting that).
> 
>> 
>> d) Boost Build
>> * either just works (great!) or doesn't.  If it doesn't it's almost
>> impossible fix without getting help
>> * I've never run across anyone outside of boost who uses it.  It makes
>> it harder to promote my work.
>> * It's natural to compose projects into "super projects"
>> * it's almost impossible to integrate with and IDE.  At one time, I had
>> things setup so I could debug executables created with boost build with
>> the Visual Studio IDE and debugger.  But it was just too fragile and
>> time consuming to keep everything in sync.
>> * it has a lot of "automatic" behavior which can be really, really
>> confusion. A small example: you've got multiple compilers on your
>> system. When it discovers this, it just picks the "best" one and you
>> don't know which one you got until the project builds (or not).  I'm
>> sure this was implemented this way to make usage of boost build "simple"
>> but it has the opposite effect.  Much better would be fail immediately
>> with a message "multiple compilers found:... use toolset=<compiler name>
>> to select desired one."
> 
> I do not like the current state of b2 for many reasons (even though I think it could be really a good build system), but CMake is not covering many features that are currently required by the boost superproject. Until the point where we can consistently build the lib (including the possibly many flavor of the same library - STATIC/SHARED at the same time, etc), run the tests, and generate the documentation (including the possibility to have the boostdoc/quickbook/doxygen toolchain), I do not see any *good* reason to move to cmake.

Cmake can do all that you listed there. In addition, I found its easy to generate targets at configure time. So the Fit library actually adds additional tests to test examples and header files. Plus, I can create configuration headers based on whether a something compiles or runs. I believe BB can do the same using virtual targets or something, but I haven’t clearly figured it out.

Finally, for boost, it could provide some high-level cmake functions so all of these things can happen consistently across libraries.

> 
>> 
>> Some Conclusions - I'm trying to not make this a rant
>> 
>> a) The ideal, platform independent build system does not yet exist.  I
>> guessing it never will.  I'm sure it won't happen in my life time - but
>> then I'm 68 - maybe you'll get lucky.
>> 
>> b) Both systems are much more fragile, complicated and opaque than their
>> promoters try to make you believe.  It's not that they're lying, they
>> truely believe in their own stuff.  There is much re-invention of the
>> wheel - The each created their own (half-assed) little language for
>> goodness sake!!!
>> 
>> c) Neither has really resolved the issue of nested projects in a clear
>> way.  Boost Build probably does or can do this.  CMake has a system of
>> "packages" and a whole 'nuther layer about "finding" them.  Assuming it
>> can be made to work - given the amount of time I've invested in CMake, I
>> should know how to do this by now.
> 
> At least, one is more or less working right now, and the other has not proven to be working in all variety of cases.
> 
>> 
>> d) I think it's time for Boost to be a little more open about
>> tolerating/encouraging alternative build systems.  I think our
>> documentation approach is a model.  Yeah it's a hodgepodge.  But the
>> various ways of doing pretty much work and generally don't stop working
>> and we don't have to constantly spend effort bringing things up to the
>> latest greatest system (which we couldn't agree upon anyway).  We have
>> libraries which have been going strong 15 years - and people can still
>> read the docs.
> 
> I have to say this does not scale at all, especially wrt. the resources (your point e/ below). I do not know if the documentation is a good example also: we have many different systems, I like quickbook, I believe it is properly serving the purpose of a "documentation", where Doxygen fails.
> What is the cost of maintaining many tools compatibility? I would promote the opposite in fact: some methods proved to be good for boost, if someone wants to integrate with a currently unsupported tool, it should not so much impact the ppl that are eg. maintaining the travis.yml (or they should do the work). Also the tools are at the core of boost, we should not neglect them and rather promote them as we do for boost libraries, and we should not integrate technologies that are weak, "just because" it makes boost more appealing for a few developers.
> 
>> 
>> e) We should find some way to recognize those who have made the system
>> work as well it has.  Doug Gregor (boost book), Eric Niebler, Joel
>> Guzman (quickbook). Vladimir Prus, Rene Rivera, Steve Watanabe.  I know
>> there others but these come to mind immediately.
>> 
>> Note that I have only addressed the issue of library development which
>> is my main interest.  I'm really not seeing this issues related to users
>> of libraries.  In particular, CMake has the whole "find" thing which I'm
>> still not even seeing the need for.  If I want to use a library, I can
>> build the library and move it to a common place with the headers,
>> specify the include directory and I'm on my way.  I'm sure someone will
>> step up to enlighten me on this.
>> 
>> Robert Ramey
>> 
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Unsubscribe & other changes:
>> http://lists.boost.org/mailman/listinfo.cgi/boost
>> 
> 
> 
> 
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost