Re: [Boost-cmake] Analysis of the current CMake system

Subject: Re: [Boost-cmake] Analysis of the current CMake system
From: Brad King (brad.king_at_[hidden])
Date: 2009-01-15 15:48:10

troy d. straszheim wrote:
> I don't quite get "That doesn't mean we can't test some tools without
> log-scraping".
> 
> I see two different cases here.  There's the developer working under
> visual studio or emacs who wants to run some tests.  This guy knows (or
> should know) how to find what compile/link flags were used, chase down
> warnings and whatnot.  In this case the message "test X failed, go look
> at the logs" is fine.  Let the user use his tool.

How do we know that a VS IDE build works unless we test it?  Testing it
requires log scraping unless we write some kind of plugin, which may not
be possible for all tools.  Even if we do log scraping for VS IDE
builds, we can still use "CWrap" for generators where it can be implemented.

(We can use the term "CWrap" for this discussion and come up with a
better name later.)

> One problem with the current implementation is that every test is a
> toplevel target;  this is easy to fix, if you just write the tests to
> flatfiles in the build hierarchy (like ctest does it) and use a driver
> script to find the files and run the tests (that is nice and clean for
> tests that involve just running compiled binaries, I haven't thought
> about how it would work for compile and compile-fail tests).

As I said in my original post, compile and compile-fail tests will work
fine with --build-and-test.  You shouldn't need to create your own
python testing script.  We can add features to CTest as needed.

> If I remember correctly the 'log scraping' was more about ctest having
> to scrape
> the build log to find compile errors.   Is this no longer necessary?

CTest has several parts.  One is to monitor builds, another is to run
the tests.  Currently it uses log scraping to monitor builds since the
native tools don't provide per-rule reports and we haven't created
"CWrap" to work around the limitation.  For running the tests it has
always separated output on a per-test basis.

>> It runs all the tests through python command wrappers to capture
>> individual output, and therefore has to generate its own compiler
>> command line invocations instead of using CMake's knowledge of the
>> native tools.  Currently -c and -o options are hard-coded AFAICS.
> 
> Right... do you see a different way to get this done?

Use --build-and-test and with "CWrap" enabled (or currently the python
approximation of it).

>> We could make this a CMake feature by teaching the generators to wrap
>> the compiler up with a tool we distribute with CMake.  Then you won't
>> have to hack the compilation rule variables for Boost or depend on
>> python.
> 
> Presumably the name of this tool (let's call it CWrap?) would have some
> interface that you could implement however you like... and if that is
> implementable in python, you've given your users lots of ways to tweak
> their build system.   We're OK with being dependent on python.

Whatever interface we create to tell the generators to do this can also
specify the wrapper (hook) command.  We can provide a simple tool to be
used by default and just specify a command-line interface for a custom tool.

>> Testing with ctest's --build-and-test feature.  The entire build and
>> execution of every test would be captured independently of other tests.
> 
> Or a python script that does what ctest's --build-and-test does...
> IMV a lot more flexibility, a lot less code.

How is duplicating functionality less code?  If --build-and-test is
missing something, we can extend it.

>> The code in question tells CMake to generate a python script that looks
>> like this (on Windows):
>>
>>   sys.path.append("c:\path\with\backslashes\to\some\file.txt")
>>   #                                        ^^ escape sequence?
>>
> 
> I'd have to look back.  This stuff was indeed working on windows;
> anyhow, looks like we can detangle a lot of this with some of your help
> making tweaks to cmake itself.

Python seems to leave the backslashes if the sequence doesn't happen to
be a supported escape.  You may just be getting lucky.

-Brad