Re: [boost] Boost CMake support - Request for Comment

Subject: Re: [boost] Boost CMake support - Request for Comment
From: Edward Diener (eldiener_at_[hidden])
Date: 2018-10-08 20:21:45

On 10/8/2018 6:15 AM, Mike Dev via Boost wrote:
>> -----Original Message-----
>> From: Boost <boost-bounces_at_[hidden]> On Behalf Of Edward Diener via Boost
>> Sent: Monday, October 1, 2018 8:00 PM
>>
>> On 10/1/2018 1:07 PM, Robert Ramey via Boost wrote:
>>> Background
>>> ==========
>>>
>>> My personal knowledge of build systems is somewhat limited.
>>>
>>> a) I don't really understand CMake all that well.  I have made
>>> CMakeLists.txt files for both the serialization library and the safe
>>> numerics libraries as an aid to build and maintain IDE and make files
>>> used in running tests and posting results to CDash.
>>>
>>> b) I don't really understand our current system, Boost Build, very well
>>> either.  I've made jam files for building and running tests for the
>>> serialization and safe numerics libraries.
>>>
>>> c) I recently reviewed C++Now 2017: Daniel Pfeifer's excellent video
>>> from C++Now 2017  “Effective CMake" -
>>> https://www.youtube.com/watch?v=bsXLMQ6WgIk . I recommend it highly.
>>>
>>> Despite my lack of knowledge in this area, or perhaps because of it,
>>> I've volunteered to direct the effort to evolve Boost so that users of
>>> Boost can avail themselves of the benefits of CMake.  This effort will
>>> be organized as follows:
>>>
>>> a) A discussion on the boost developers mailing list with the goal of
>>> reaching a consensus as to what benefits CMake can and should provide to
>>> users and developers of Boost.  This effort will commence with the
>>> posting of this notice.  I hope that this discussion can be more or less
>>> resolved within 30 days.
>>>
>>> b) I will then synthesize from the above discussion a call for proposals
>>> which lists the requirements and requirements.
>>>
>>> c) Those proposing CMake for boost will have approximately 90 days to
>>> prepare their proposals.  Their proposal are expected to look similar to
>>> a boost library proposal.  That is they are expected to have some
>>> (CMake) code, along with tests, documentation including tutorials and
>>> reference.  This is only an expectation.  It's conceivable that such a
>>> proposal might only contain conventions and and samples of what boost
>>> library should contain to fulfill the requirements. As with boost
>>> libraries, submissions are not required to totally complete.  But they
>>> have to be sufficiently complete to convince reviewers the the
>>> submission fulfills the stated requirements and can be finished in good
>>> time.
>>>
>>> d) After the 90 day period, submissions will be reviewed simultaneously.
>>>    Note that this is at a variance from normal boost procedure of
>>> reviewing submissions one at a time.  Since we expect to receive a
>>> number of submissions and only one can be accepted, the normal boost
>>> protocol can't really function.  Target date for this review is 1
>>> February 2019.
>>>
>>> e) After the review period the review manager will prepare a report
>>> which includes the decision of which submission will be accepted into
>>> the official boost distribution.  Currently, I, Robert Ramey, expect to
>>> be review manager.  However, it's possible that this by the time the
>>> review is undertaken, this coveted position could be assigned to some
>>> more worthy candidate.
>>>
>>> Scope, Requirements and Features
>>> ================================
>>>
>>> We presume that submissions will fulfill the applicable requirements of
>>> any boost submission.  In addition we would like to see the following
>>> addressed.
>>>
>>> Library Build
>>> -------------
>>>         “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk
>>> Recommends that all libraries - including header only libraries have a
>>> CMakeList.txt file.  What should this include for Boost Libraries?
>>>
>>>
>>> Library Test
>>> ------------
>>>       Should facilities for "testing" be only done by developers? or
>>> should users who acquire library packages also be able to test libraries.
>>>       Should CMake testing results posting be used - CDASH?
>>>       Should we just skip the issue of Library Testing and continue to
>>> depend on Boost Build.
>>>
>>> Library Packaging
>>> -----------------
>>>       Is the library packaging facility provide by CMake - CPACK - useful
>>> to boost.  Should boost libraries be updated to support it?
>>>
>>> Dependencies
>>> ------------
>>>       If the above is implemented, can we depend upon CMake to handle
>>> library dependencies?  Or does some special functionality
>>>       Do circular dependencies constitute a problem?
>>>
>>> Modularity
>>> ----------
>>>       Currently boost is organized as a tightly integrated group of
>>> libraries.  This organization manifests itself in a number of ways.
>>>           We have a Boost "super project" in github.  Boost libraries are
>>> "sub projects" of this super project.  This known structure is exploited
>>> by and depended upon by boost tools.  Should CMake support continue this
>>> policy/design.
>>>           We distribute boost as "super project".  Should we continue to
>>> do this or distribute libraries on a library by library basis.  Does
>>> CMake/CPACK etc. make this "simple"?
>>>
>>> Documentation/testing
>>> ---------------------
>>>       What support should be require for uses of Boost CMake
>>> implementation. Documentation, examples, templates for helping library
>>> developers.  How should the CMake design/implementation be tested?
>>>
>>> Other considerations
>>> --------------------
>>> add your requirement/features here.
>>>
>>>
>>> Useful Resources
>>> ===============
>>>       “Effective CMake" - https://www.youtube.com/watch?v=bsXLMQ6WgIk
>>>       https://gist.github.com/mbinna/c61dbb39bca0e4fb7d1f73b0d66a4fd1
>>>       https://github.com/purpleKarrot/Boost.CMake
>>>       https://www.youtube.com/watch?v=eC9-iRN2b04
>>>
>>> Please try to keep this discussion on this list.
>>>
>>> I know that there has been a lot going on on slack. But having it in two
>>> places makes it harder for me to review and summarize.  Also slack
>>> doesn't conveniently maintain comment hierarchy.  So please try to keep
>>> this discussion on this list.
>>
>> I strongly suggest that the goal of using CMake with Boost begin with
>> making Boost libraries, whether header-only or built, available to CMake
>> projects. This also means that we have a way of testing whether or not
>> such a solution actually works for any given Boost library. My own
>> preference for testing whether or not such a solution would work or not
>> would be to use our current b2/Boost Build test jamfiles, but if this is
>> too difficult to do we still need some way to test the solution for each
>> library so each library maintainer can see whether or not the solution
>> we choose works for his library.
>>
>> This should be our first goal and all other goals should be tabled until
>> and when we can have a solution where this works flawlessly and there is
>> a general agreement that whatever proposal is chosen to implement this
>> is best.
>>
>> Only when the above is fully accomplished should Boost then look into
>> the goal of using CMake internally, perhaps for test, docs, building
>> libraries, or what-not. We need to work this way because every solution
>> is bound to end with comments that it does X but does not do Y or it
>> does X and Y but does not do Z, and we all know where these endless
>> arguments end: that we end up doing nothing because we start by trying
>> to do too much and we can never reach agreement.
> 
> I agree with you in so far, that the focus should be on the interface
> (how to consume boost libraries) and less on the implementation details,
> but I see two practical reasons, why it is beneficial to immediately
> use cmake (at least) for building too:
> 
> 1) Creating a CMake target for a separately compiled binary that can be
>     consumed by the user is about the same effort as using cmake to build
>     the library (if necessary) in which case you get the interface
>     information for free (switching to cmake for unit-tests and docs is a
>     different topic).
>   
> 2) "making Boost libraries [...] available to CMake projects" for me also
>     means I have an easy way to compile those libraries with specific
>     compiler flags that match my project settings - otherwise, what is the
>     advantage compared to the current mechanism that is already build into
>     cmake?
>     Ensuring consistent build flags is relatively easy if all libraries use
>     the same build system (in this case cmake) but can become quite complex
>     if you want to implement an automatic translation from cmake to b2 or
>     vice versa.
> 
> So yes, while a library (build-) interface and consummation should be the
> focus of the review, I'd expect that at least building the libraries with
> cmake will be the simplest way to implement that interface.
> 
> Just to be completely clear: Just because the "reference implementation"
> of a cmake interface is probably a cmake-based build, that doesn't mean
> everyone has to actually adopt that reference implementation as long as
> the interface is consistent.

Boost Build has a feature which allows the programmer to test at build 
time what C++ features are supported for a particular compiler 
implementation and compile parameters, and change the build internally 
on-the-fly accordingly. I do not know if any Boost non-header only 
library uses this feature but if one does it will be difficult 
translating this to CMake terms, if such a similar feature exists in CMake.

It is for such a reason that as soon as you start talking about using 
CMake for anything else but identifying Boost libraries for the purposes 
of end-user use of Boost libraries in their own CMake scripts, that you 
may run into much more work than you can imagine. You just can't tell 
individual library developers to translate all their own use of Boost 
Build into CMake, as if this will be simple for them to do, and they 
will just do it with little time spent and as if it will be no work for 
them. Boost Build, for better or worse, has some extensive features 
which have no easy correspondence to CMake. Unless you or someone else 
is willing to develop a bridge between Boost Build and CMake for 
features of Boost Build which libraries rely on, then individual Boost 
libraries which use such Boost Build features are going to find it very 
difficult to transition to CMake, possibly even for something as 
seemingly innocuous as building the library itself using a CMake script.

This is why I have urged the move to CMake by Boost to proceed slowly by 
baby steps, rather than to try doing things that may take a huge initial 
effort to get done, and therefore may end up never accomplishing anything.

> 
> Mike