Re: [boost] CMake and Boost Build tests

Subject: Re: [boost] CMake and Boost Build tests
From: Robert Ramey (ramey_at_[hidden])
Date: 2017-07-26 21:03:55

On 7/26/17 1:28 PM, Edward Diener via Boost wrote:
> On 7/26/2017 3:33 PM, Robert Ramey via Boost wrote:
>> On 7/26/17 11:49 AM, Edward Diener via Boost wrote:
>>> Following John Maddock's appeal for practical solutions related to 
>>> the move to CMake, I would like to know what the CMake equivalent is 
>>> to the Boost Build unit test functionality.
>>>
>> I can tell you what I did for the serialization library.  This 
>> predates the current discussions so will likely be considered out of 
>> date by the current contributors to the discussion.  On the other 
>> hand, it does actually work and you can test it yourself.
> 
> Robert I appreciate your answer. Is this the CMakeLists.txt in the CMake 
> subdirectory of the serialization library 

right

? Remember I am talking about
> header-only libraries where nothing is "built" and I am only trying to 
> run the unit tests in the library's 'test' subdirectory using CMake 
> rather than Boost Build using CMake's equivalent of a script file rather 
> than a Boost Build jamfile.

right that's what it does.  It builds the library and executes the tests

>>> In other words what do I write for CMake in order to do a Boost Build 
>>> compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
>>
>> One adds targets (which for me is just the name of the 
>> source/excutable without the cpp, exe, etc. to a list to get compile 
>> and run functionality.  Its pretty simple.
> 
> I want to see actual cmake code. Saying "Its pretty simple" only means 
> it is pretty simple for those who already know CMake well.

aaa - the CMakeLists.txt IS the "cmake code".  You invoke CMake - I use 
the GUI version and it produces and makefile, or an IDE file (I generate 
the one for XCode).  They you build the library (if it has one) and run 
the tests from the project it created for you.  It's a two step process.

> 
> <RANT>
> 
> I find the CMake documentation at https://cmake.org/cmake/help/v3.9/ 
> nearly impossible to understand due to the fact that there is no 
> overriding explanation of how CMake is supposed to work as practical 
> examples of setting up CMake scripts. While I realize that all the 
> information is supposedly there I have just about no idea how it all 
> comes together. I am guessing that the only way I can really learn CMake 
> is buying the "Mastering CMake" book, which I don't mind doing so if I 
> have to do it. If I am wrong and someone would like to point me to an 
> actual overview which explains CMake in a regular and meaningful way, 
> rather than a bunch of loosely related topics which is the actual CMake 
> documentation in the above URL, it would be appreciated.

This is a serious short coming of CMake.  Given this it's a mystery that 
CMake has been successful.  But then then look at all the competition 
it's ever worse!!!  To addressed this, I wrote a walk through in the 
boost library inclubator to help library developers get started with 
CMake.  It uses on the safe numerics library CMake as an example.  This 
example/narrative addresses only header-only files.
> 
> </RANT>
> 
>>
>> I couldn't make the compile-fail and run-fail targets to work so I 
>> just removed them from the cmake test suite.  So if your thinking that 
>> CMake has a ways to go to be able to offer what we expect, you're right.
> 
> So you are saying that you cannot run a unit test with CMake that is 
> meant to show a failure. Wow, what a brilliant system !

Right - I presume that there's a way around it if one has nothing else 
to do.  I didn't find a simple one. This fact, combined with the fact 
that the system is well maintained, continually enhanced, and responsive 
to it's users for over 10 years says a lot the demands and state of 
library development.  It seems almost no one ever asked kitware for this.

>>> Does CMake have an equivalent to the Boost Build alias rule ? or the 
>>> Boost Build project rule ?
>>
>> LOL - I honestly don't remember what these rules do in boost build.  
>> But then I've only been using boost build for 15 years.
> 
> The project rule for unit tests is normally used to set usage 
> requirements for all tests being run. The alias rule for unit tests is 
> normally used as a means of grouping some unit tests together so that 
> the end-user of the tests can test individual groups rather than all the 
> tests.

My real point is that even one with a lot of experience with boost build 
(I would guess that the serialization library carpet bombing test suite 
is the most complex one in boost) still can't sit down and use it 
without spending huge amounts of time re-figuring it out.  This is a 
serious indictment of boost build which has now brought us to this 
unhappy juncture.

>>> The great majority of the Boost libraries are header-only libraries 
>>> where Boost Build is used to run the unit tests and, 
>>
>> true and this case is pretty simple in both cmake and boost build.
> 
> Once again saying it is pretty simple in cmake isn't showing me anything.

I've provide two working examples for two very different libraries and a 
narrative with the information I gleaned from making them.  I would hope 
they would be helpful. It's the best anyone can/has done.

>>> optionally, Boost Build is used to build the documentation using a 
>>> quickbook, boostbook, doxygen toolset to generate html and pdf files. 
>>
>> Personally, I could never make this tool chain work despite a large 
>> investment of effort.  I gave up on doxygen and quickbook after 
>> concluding them to be essentially non-functional.  I did manage to use 
>> the xml tools loaded with boost tools along with some downloaded FOP, 
>> a version of XMLMind to create toolchain which is about 100 times 
>> better and 1/100 as complex as the boost one for creating documents.  
>> So you're life will be better when you're forced to leave these behind.
> 
> I heartily disagree. Using quickbook and doxygen is a no-brainer for me, 
> but there is no point of debating this. Are both perfect ? No. But using 
> them makes creating documentation supremely easy.

Right - we just have to disagree.

> I did say that there is no Boost requirement for creating documentation 
> using Boost Build. Some documentation is straight html. But for those 
> libraries that do go the quickbook and doxygen way using Boost Build we 
> need a CMake way of doing this if that is what the Boost Steering 
> Committee is mandating.

Actually we don't.  We can just use a small shell script or batch file. 
It's only about 5 lines long and it's about 20 times simpler than going 
into boost build trying to train an ant to train a flee.  This is 
another think we'll just have to disagree on.
> 
>>
snip

>> But you might want to experiment a little with the following useful 
>> orthogonal ideas.
>>
>> a) investigate how CMake Find<your library> should work and consider 
>> adding the files necessary to support it to your library.  This 
>> wouldn't mess up any of your other work.
>>
>> b) Consider making a simple CMake project to build library and run 
>> tests such as I did for the serialization library.  CMake gurus it 
>> seems dismiss this effort.  But it has worked for me.  The main 
>> benefit for me is that permits to to build a working IDE which I can 
>> used to build and debug.  For me, this has turned out to be a great 
>> time saver.
> 
> I do not know how you learned enough of CMake to do what you want but, 
> apropos my RANT, I am finding my ability to do this severely hampered by 
> the piecemeal documentation which CMake deems its own docs.

Ahh docs ...  This is not a CMake issue. It's a software development 
issue.  Documentation for software libraries and tools is generally 
atrocious.  The thing that bugs me the most though is that developers 
don't agree with this statement.  they think their stuff is pretty good 
an that the users just need to be smarter.  They're dead wrong. They 
design crappy stuff then package then (under duress) write some docs 
which only serve to illustrate how lacking in coherence their original 
designs are in the first place.  But then it's too late and everyone 
else is left with the job of shoveling a path through it.

I've got a lot to say about this and I will be giving a presentation on 
the subject at CPPCon 2017 for anyone who might be interested.  That is 
the both of you.

Robert Ramey