Re: [boost] CMake Announcement from Boost Steering Committee

Subject: Re: [boost] CMake Announcement from Boost Steering Committee
From: rleigh_at_[hidden]
Date: 2017-07-21 14:25:18

On 2017-07-21 09:30, Paul A. Bristow via Boost wrote:
> The mailing list has recently been focussing on just that - the *how*.
> 
> The *how* is looking feasible, but the *who* remains entirely unclear.
> 
> So the Steering isn't going to happen without an engine of some expert
> volunteers with enthusiasm and time.
> 
> I ask again ... who?

If you would like volunteers to do this, can I sign up to assist?  I've 
been keen on being able to build Boost with CMake for years, but didn't 
get involved due to not being a core Boost person, just an end user, and 
also because it looked like there were also a bunch of keen people 
already working on it.  But if you want additional people on board, I'd 
be happy to contribute in whatever way I can.

As a bit of background, I'm a C++ programmer using Boost daily, but I 
also maintain CMake support stuff for my day job, and occasionally this 
involves helping upstream projects with their build systems as well.  
For example, my latest CMake work was adding CMake support to the 
venerable Apache Xerces-C++ library, and I've previously contributed 
CMake support to libtiff, bzip2 and others.  I'm also one of the people 
who maintains FindBoost.cmake in the upstream CMake git repository, as 
well as other CMake features.

Also, to expand upon one of Niall Douglas' points in his earlier email, 
regarding library developers not having the same worldview as the rest.  
Could I point to this trac ticket: 
https://svn.boost.org/trac10/ticket/1094 (this is about pkg-config 
support for Boost).  This is one of the key deficiencies in Boost, as a 
project, in being difficult to use by downstream consumers, but it's 
been largely ignored and treated as unimportant.  Ask yourself this 
question: I need to use Boost library XX.  What are the transitive 
dependencies I need to link against in order to successfully build my 
stuff with XX?  You have to find out, then hard-code it.  Example: 
filesystem headers use system headers, and I need to know to link 
against system *and* filesystem if I use filesystem.  And if these 
dependencies change in a new Boost release, your build breaks.  I was 
already having problems with this a decade back when I filed the ticket, 
and it's only gotten worse.  The reason I'm bringing this up here is 
that this ticket never got resolved because it required some knowledge 
of Boost.Build to complete.  I didn't have that expertise, and 
apparently no one else did either.  Even after asking several times on 
IRC, the lists etc., no one ever offered to help with this, and so it 
got stalled.  Having a custom and baroque build system prevents people 
like me, who are not Boost.Build experts, from contributing to the 
project.  I still have a PR on Boost.Units stalled; it's a one line 
change, but it requires some knowledge of Boost.Build to integrate a 
testcase; again it's hindered a trivial but useful contribution to the 
project.  I just want to point out that no matter how wonderful 
Boost.Build might be in theory, in practice being different from the 
rest of the word places a practical barrier upon usage of and 
participation in the development of Boost.  And that barrier is a huge 
one; most of us don't have the time to dedicate to learn yet another 
build system just for a single project, *but* learning a common tool 
like CMake which is used in thousands of projects was worth the cost--I 
get to reuse and extend that knowledge daily using and contributing to 
dozens of projects.  Please don't discount this in your discussions.

Could I also point to this block in 
https://gitlab.kitware.com/cmake/cmake/blob/master/Modules/FindBoost.cmake#L554 
.  This is what we do in the absence of proper dependency information 
being provided by Boost, i.e. no pkg-config, cmake config or similar.  
We hard-code *every* *single* *dependency* from 1.33 up to 1.64 at the 
time of writing.  I wrote a custom parser to extract this metadata from 
the autolink information in the headers, similarly to the above ticket.  
*We* pay the cost of this significant defect in the Boost project in the 
form of a high maintenance burden and breakage with every new Boost 
release.  Most projects hardcode the information, but it's fragile and 
should be unnecessary, and so we took on that burden so that CMake users 
could use Boost without worrying about the problems.  That's why this 
file is such a huge and over-complicated nightmare.  As and when Boost 
provides proper configuration information, which I would hope the CMake 
conversion work would provide, we can throw this entire monster in the 
bin (most CMake Find modules are a couple dozen lines at most; this one 
is just shy of 2000).

Could I also point to the lengths I have to go to to build Boost as part 
of a larger project:

https://github.com/ome/ome-cmake-superbuild/blob/249f06aa633a0feb2bb99c13fc6b6c8b49812267/packages/boost/superbuild.cmake#L6
- this is a fragile mapping from compiler to the boost-specific toolset 
name (why does Boost even care?  just to put it in the library name, a 
feature which I always disable because it's largely pointless?)
https://github.com/ome/ome-cmake-superbuild/blob/249f06aa633a0feb2bb99c13fc6b6c8b49812267/packages/boost/patch.cmake
- this is a manual patching of the Jamfiles to build with ICU on 
Windows; the hardcoded naming assumptions are broken.  With CMake, 
FindICU.cmake (which I wrote), would pick up the correct path 
automatically and avoid the hardcoding
https://github.com/ome/ome-cmake-superbuild/blob/249f06aa633a0feb2bb99c13fc6b6c8b49812267/packages/boost/configure.cmake
- we have to configure twice for Unix and Windows
- we have to hand-patch the Unix project-config.jam to use the correct 
toolset
https://github.com/ome/ome-cmake-superbuild/blob/249f06aa633a0feb2bb99c13fc6b6c8b49812267/packages/boost/build.cmake
- building is ridiculously complicated; look at all that stuff being 
manually set
- again duplicated logic for Unix and Windows
- the hardcoded assumptions about the names of the zlib and bzip2 
libraries don't always hold on Windows (incidentally, the autolink 
information is also broken--it hardcodes the names and assumes they are 
built as part of the Boost build, which isn't always true, particularly 
in larger projects such as this where multiple components depend upon 
zlib and bzip2 independently of Boost).
- the zlib and bzip2 include paths must be the same; it breaks if they 
differ (a bug in the Jamfile?  workaround is to always have them in the 
same location)
- the install paths for the Windows DLLs needs fixing up; they go into 
the libdir while every other project puts them into the bindir (because 
they go on the PATH)

I hope the above isn't too upsetting or controversial.  I wanted to 
provide some perspective from the point of view of an outsider and end 
user, and potential contributor, about what things are like from the 
other side.  Boost is a wonderful collection of libraries, but there are 
some key things which could be improved, and Boost.Build/b2 has I am 
sorry to say been one of the worst aspects of Boost.  End users need to 
go through some pain to work around defects in Boost.Build, as well as 
Boost.Build doing things differently to most other build systems.  Some 
might be due to my Boost.Build inexperience, but either way this is what 
non-experts experience.  It makes using Boost unnecessarily painful and 
difficult.

Kind regards,
Roger Leigh