Re: [boost] Draft copy - Call for Submissions - CMake for Boost

Subject: Re: [boost] Draft copy - Call for Submissions - CMake for Boost
From: Robert Ramey (ramey_at_[hidden])
Date: 2018-10-24 03:46:48

On 10/23/18 2:35 PM, Raffi Enficiaud via Boost wrote:
> On 18.10.18 22:43, Robert Ramey via Boost wrote:
>> The following is a draft of the document of I have planned to post on 
>> behalf of Boost on or around 1 November 2018 as the first step in our 
>> next attempt to accommodate CMake in Boost.
>>
>> Robert Ramey
> 
> I have a couple of questions and remarks, inline.
> Maybe there is a newer revision?

Not yet.  Most of the comments so far have been about wording and not 
enough to generate a new version.
> 
>>
>>
>> Call for Submissions - CMake for Boost
>> ======================================
>>
>> For some time, there has been interest on many users and developers of 
>> Boost Libraries to make Boost easier to use with CMake. Discussions in 
>> the past have not resulted in an implementation which has been widely 
>> accepted.  Never the less, hope springs eternal, and we intend to 
>> attempt this once again.  It seems that often there are difficulties 
>> before such projects even start in that there are wide discrepancies 
>> as to what should be the goals and expectations of Boost support of 
>> CMake. The aim of this document is to try to build a consensus on this 
>> question before encouraging CMake developers and experts to submit 
>> proposals for Boost support of CMake.
>>
>> Requirements and Scope
>> ======================
> 
> Where is the scope of the submission explained?

I titled it "Facilities and features" below. I see now that this was 
confusing.  Sorry about that.
> 
>>
>> Requirements
>> ------------
>>
>> Submissions will fulfill the applicable requirements of any boost 
>> submission.  This will include:
>>
>> Â Â  a) any necessary code.
>>
>> Â Â  b) specifications/requirements for things like directory structure 
>> and contents.
>>
>> Â Â  b) A useful document. Document should provide information, examples 
>> and templates so that a typical C++ and CMake user should be able to 
>> do any of the following in a short time - say 10 minutes.  Document at 
>> a minimum should be viewable as html and/or PDF file.
> 
> Can this be a few wiki pages on Github as well? I started looking at the 
> way CMake builds its own documentation, and I believe this is overkill.

I think it's going to be significantly more than that. CMake 
documentation is not a great model.  I see the new book:

Professional CMake: A Practical Guide
Version: 1.0.3
© 2018 by Craig Scott

as a model.  It's 420 pages long. I would like to see manual in this 
style for Boost CMake.  Of course it would be much, much smaller.  It 
would be more "cookbook" style.  Of course I believe in testing.  And 
such a manual is easy to test.  At the next CPPCon round up some 
volunteers and challange them to implement CMake for a given library. 
It should take under an hour. If it doesn't, the test failed.

>> Â Â Â Â  1) A library user should be able to incorporate a Boost Library 
>> into the CMake script.
> 
> Does b2 do that? I personally always used b2 for installing boost within 
> a prefix and then using it from there.

Right.  Once you've automagically built the libraries using b2, then 
normally one just uses his build system to incorporate the prebuilt 
libraries into his app.  If b2 is being used to build the app, then the 
incorporation of a library into one's app can be very smooth.  But b2 
isn't widely used outside of the building and testing of boost libraries 
- as far as I know.

> CMake contains already the FindBoost module:

I've tried this but it didn't go smoothly for me.  I forget why.

> - Is the intent here to be able to replace CMake's FindBoost it with our 
> own?

I believe that CMake for boost would create FindPackage for each boost 
library.

> - Should that be on a freshly cloned repository without prior build step?

I've left the requirements pretty open.  In the minimum, it would just 
rely on libraries built by b2.  In the max, it might build the boost 
libraries to be used.  Note that in some cases, users incorporate the 
source of the libraries into their app, so presumably that might be 
supported as well.  I've specified "scope" in such a way that a 
submitter can make his submission cover those things that CMake can 
really help. I don't want someone to create a version of b2 made by 
CMake.  I'm suggesting that CMake support those things that CMake is 
really built or and leave the things it can't to some other tool.  I 
seek improvement of our current system - I'm not demanding perfection or 
realization of some build paradise.

> I am not found of this idea. I would rather focus on improving FindBoost 
> in CMake for library users and stick to the current workflow: 
> build/install boost, then source boost.

A submitter might propose that.  By my perusal of CMake books and 
documentation leads me to be surprised were FindBoost would be suggest.

Of course when submissions are opened for review, you'll have plenty of 
opportunity to spitball.

> Also, there are different ways of incorporating. I am pretty sure that 
> there are very conflicting opinions on this.

Right - that's what the review is for.
> 
>  >Â Â Â Â Â  2) A library user should be able to determine which additional
>  > Boost Libraries he needs.
> 
> Does he? If I need boost.test, as a library user I do not care about the 
> dependencies of boost.test
it will have to be determined somehow.  How exactly this is to be done 
or not done would be part of the submission.  It's my intent not to 
overspecify this.

>> Facilities and features
Scope?
>> -----------------------

>> Â Â *a) incorporating selected Boost Libraries into a user's application
> 
> There are some libraries in Boost for which having a CMake/b2 one-to-one 
> feature mapping is close to impossible with CMake (without hacking CMake 
> at least).

> So what do you mean by "incorporating"? 

I'm thinking something like FindBoost, FindPackage, or ... ?

> Should that cover all the 
> possible use cases that currently b2 offers?

Not a requirement.  It doesn't have to replace b2.  It just has to be 
something that lots of users would find helpful.

>> Â Â Â  *3) link only tests - must pass/ must fail
> 
> Do we have link only tests in b2? Can somebody point me to an example?

I believe we do but I'd have to search for an example which I'm 
disinclined to so right now.
> 
>> Â Â Â Â  4) should be available to both users and developers
>> Â Â Â Â  5) should not depend upon tools other than C++.
> 
> Building documentation for instance requires other tools.

Hmmm - not necessarily.  But yeah.  Boost does depend on other tools 
like xslt.  But I'm thinking requiring python would be too much.  Of 
course the extra burden of other tools would be something to be 
evaluated during the review of the submissions.

>> Â Â Â Â  6) test should have the option of posting the results to some 
>> public "dashboard"
>> Â Â Â Â  7) option for recursive testing and/or building of libraries 
>> might be nice.  That is if library A uses library B and I test library 
>> A, this would automatically test library B beforehand.
>>
>> Â Â *f) In CMake, dependent libraries are specified as part of the 
>> library CMakeLists.txt files.  There several ways to do this and the 
>> subject is pretty confusing to get right.  Any submission 
>> documentation has to explain to library developers how do this.
> 
> I would like to remove "CMakeLists.txt" here, as this is part of the 
> implementation.

Hmmm - I had just assumed that CMake with boost would require 
CMakeLists.txt and that this would include dependencies.  But maybe not. 
I would like to leave the option of manual specification of dependencies 
to the library author.  That is my intent here. If this is a rash 
assumption I suppose I could remove the explicit mention of 
CMakeLists.txt here without changing the intent.

> Also, few libraries are needing external libraries for compilation. I 
> have in mind GIL that can work with JPEG/TIFF/PNG external libraries.
> If CMake offers facilities for sourcing the external dependencies, those 
> should be favored (for instance find_package(JPEG)).

Note that being able to use CMake to build boost libraries is not an 
explicit requirement.  It would be nice, but the most important goal is 
to support users who CMake to build their own app to incorporate boost 
into their projects.  So.... I'm reluctant to specify requirements which 
apply to building boost itself.  If someone's submission includes the 
facility to build (or recurrsively build) boost libraries and it really 
works well, that would be great though.  Also I'd be happy to consider a 
submission which fulfills the minimum requirements but leaves the door 
open to added functionality in the future.

>> Â Â *j) support of library modularity. This would mean:
>> Â Â Â  *1) independence of things outside the library directory 
>> structure. To wit - no presumption about any "super project".

> I do not understand here. I believe this deserves an example.
> (of course, I understand what a superproject is, but as soon as there is 
> a dependency within libraries, then a library should somehow know about 
> its dependencies, which means that a higher level is orchestrating.)

Currently using b2, the build system discerns much of the information 
needs by looking in the direction structure outside the the library 
itself.  This means if you move just the library somewhere else, the 
library/tests can't be built.  This feature may make sense for b2 - but 
I see it a very bad idea in this context.  using, building, testing, 
etc. of a library should not depend on things outside the library. 
(other than tools of course).

>> Â Â Â  *2) building out of tree
> Is this relevant?

To me this is a very important requirement.  Currently with b2, running 
tests, building or whatever changes the directory structure.  This 
creates a whole lot of problems.  Not the least of which if you run the 
same command twice, you might not get the same results.  This is 
probably not a real issue as CMake supports and promotes out of tree 
builds as a fundamental feature.

> 
>> Â Â Â  *3) no need for installing libraries not used by the build target
> 
> Do you mean "... build targets and any of its dependencies"?

LOL - I wanted to avoid the word dependencies so I phrased it this way.

> 
>>
>> Â Â k) ...
>>
>> Submission Rules and Procedures
>> -------------------------------
>>
>> a) This document constitutes a "Request for Submissions".  It will be 
>> announced on the boost announcements list, boost developer's list, 
>> slack/boost, isocpp website and twitter account, reddit at r/cpp. 
>> Submissions will be open to anyone.
>>
>> b) Submissions will be accepted at an email address to be specified. 
>> Received submissions will receive a "prescreening" to determine that 
>> they meet the requirements specified for such submissions at the top 
>> of this document.  Those that don't will be returned to the author.
> 

> There are so many ways to misinterpret things in the requirements, would 
> the prescreening be open some time before the submission deadline?

Right.  These requirements ARE loose. That is, some things are really 
required, somethings are "nice to have" etc.  This gives submitters lots 
of leeway to include "cool features" which are easy to include and to 
exclude "nasty implementations" which are not strictly required.

I have no idea how many submissions I might receive - honestly NO idea. 
It could well be zero - not a surprise and probably a relief. It could 
be 20.  I've planned on to a pre-review to verify that submissions 
actually fulfill the requirements listed here.  For example, if someone 
submits a proposal without documentation, I can easily just reject it 
for not meeting the rules.  Same if it fails to meet necessary 
requirements - those marked with a *, they can be rejected immediatly as 
well.  If it's very clearly not a serious submission, I would feel 
justified in rejecting that as well.  Review is serious work, I'm not go 
to ask people waste their time.  Of course some people will be upset 
about getting summarily rejected.  But it's unavoidable.

Just for fun here is my guess:

10 submissions received
5  rejected for have no or almost no documentation
2  rejected for just being ridiculous
2  seriously reviewed.

> 
>>
>> c) The name of the author of a submission will not be included in the 
...
>>
> But we hope that it will serve as 
>> a tangible token of our gratitude for solving one of Boosts most 
>> pressing and difficult problem
> What you are suggesting is good to some extent, but it surprises me that 
> no body mentioned that: it is a highly competitive submission scheme, at 
> the limit of toxic for boost.

The anonymity has been question.  It's pretty novel.  I really don't 
know how it would turn out.  But remember that this whole idea is novel 
for boost.  It's sort of like the XPrize for C++.  Note  anonymity is 
not without precedent.  Submissions to scientific journals are reviewed 
by reviewers who don't know who the submitters are.  So the idea is not 
totally off the wall.  And I sincerely worry that the boost review 
process can be so toxic that it my dissuade potential submitters.  This 
goes double given that we're hosting a "contest". Suppose we were 
hosting a "context" for a new library X.  You start working on your 
submission. Just as your about to submit it, it's announced that a 
submission from Howard Hinnant has just been received.  Hmmm how would 
you feel about submitting now?

Another thing is that boost will soon be criticized (if it hasn't 
already) for being an exclusionary, elitist, faux meritocracy. I'm 
thinking of stepping up and confronting this head-on by reinforcing the 
fact that boost IS a meritocracy, proud of it, and willing to prove it. 
I concede I'm sort of enamored with the idea.  But I will that's it 
pretty radical for a 20 year old organization mostly run by really old 
people.

> You pointed yourself that this is new for boost that we will have 
> several submissions for one specific tool. Yet, there are other ways of 
> achieving this, don't you think?

So far this is the only complete proposal.  We've been asking for this 
for 10 years.  This is the third attempt.  If you've got a better idea, 
now is the time to propose it.

> I believe that nobody will get it right the first time, and this should 
> be a common effort to get there. What you are suggesting is a one-shot 
> commitment.

LOL - no I'm not.  This is a sufficiently novel idea that the odds of 
success can't be rated at more than 50%.  If it fails, that'll be the 
third time.  There will be an opportunity for a fourth attempt.

If no acceptable submissions are received or the review process deems 
none acceptable, the we're back to where we started - no harm no foul.

> So nothing for collaboration, and nothing for incremental support of cmake.

I'm not sure I get this.  Boost is all about collaboration.  I'm trying 
to apply what we've learned about promoting the development of quality 
libraries to the development of quality tools.

> How can we fix that? I would love to collaborate openly on this.

a) craft you're own submission.
b) participate in the review process
c) add your own PR to the github for the tool
d) offer friendly constructive advice to the author/maintainer of the 
Boost CMake facility.

Robert Ramey

> 
> Raffi
> 
> 
> _______________________________________________
> Unsubscribe & other changes: 
> http://lists.boost.org/mailman/listinfo.cgi/boost