Re: [Boost-build] [run] need help with b2

Subject: Re: [Boost-build] [run] need help with b2
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2016-10-25 00:57:23

On Mon, Oct 24, 2016 at 2:40 PM, Raffi Enficiaud <
raffi.enficiaud_at_[hidden]> wrote:

> Dear all,
>
> There are several threads in this ML that I raised and are left open
> without any positive outcome on my side.
>

Sorry about that.. I don't always have time to devote to responding as much
as I would like (last week was one of those times).

I have some fixes in boost.test and I would like to avoid any regression of
> the kind I had for 1.62 in the future. Those fixes are ready to merge, but
> blocked by the b2 issues.
>
> To avoid regressions, I need several improvements of the "run" commands.
> Those would be to my opinion a good thing for b2:
>
> - being able to [run binary] without being forced to rebuild the binary,
> on all platforms (see my post "[run] error on Win32: trying to create a
> binary out of target name", 19/10/16). Basically a "[run ]" should be a
> command that is not so tightly coupled with a build (but would trigger a
> build if needed).
>

Since this is something that works on some platform and not on Windows..
I'd consider it a bug. It will take some time to figure out a fix though as
I don't usually test on Windows. But perhaps you can help me by seeing if
we can come up with a minimal repro project. I checked into BB an example
that has the above "multiple tests with one binary". Can you go to
boost-root/tools/build/example/complex-testing and do a "b2 -a" on Windows
(or other platforms) to see if that works or not? [1]

- being able to [run binary] several times with several command line
> options, without any aliasing/troubleshooting among those runs
>

This already works assuming you build the exe once, as in the example I
reference above, and you specify different target names for each set of CLI
options combinations. Is there something else you are asking for?

- being able to discover the output folder of a test (see post "[run]
> parameter containing the build directory"), in order to assert the
> postconditions of a run when some system behaviour is expected
>

The key problem with getting that directly is that of course it's not know
until late in the process of generating the targets. As you know the
location is determined when the final target is created. Anyway.. One
option for this is to run the tool that checks the post conditions as a
separate step. And giving that post check tool as an input file argument
the test it needs to check. For example:

==
run success.cpp : arg1 arg2 : : : success-a ;
run post.cpp : : success-a : : post-a ;
==

When post-a is run it will have a single argument with the full relative
path to the "success-a.test" file. In your post check program you would
take the directory of that path to get the location of the side-band
generated data you are after. In that same complex-testing example above
there's a working example of this also. [1]

> - being able to propagate easily the requirements, in case the tests
> should not run on specific platforms (inside the thread "[run] parameter
> containing the build directory")
>

I'm not understanding what you are asking for that doesn't already exist.
Can you give me a concrete, hopefully small, example of the problem?

- being able to declare environment variables for [run ...] commands
> (thread "[run] specifying environment variables for running tests")
>

Yeah, that's currently not possible. As.. (a) the current test actions
don't support setting env vars. (b) It would be rather messy to account for
the various ways to set env vars on the various OS shells. (c) The "b2"
core engine doesn't have support in it for setting env vars when executing
actions. If we wanted to support this (c) would likely be the easiest to
implement [2]. The usual solution would be to change your test programs to
accept arguments to alter the behavior instead of using env vars.

Those are the basics I use with cmake. I would really appreciate to have
> those features in b2 directly and avoid a quick and dirty fix with cmake on
> my private build agents.
>
> I do not know how I may work this out with a b2 specialist, but I have to
> say I am a bit blocked and I am starting feeling a bit frustrated by the
> time it takes for me to get something done with b2.
>

I should say that the current testing.jam implementation is horrible. Note,
not blaming anyone though. It was an expedient port of the b1 functionality
that was enough to get the existing use cases working. And as such it
operates in a rather nasty kludgy way. And this is certainly something
worth redoing. If not now, certainly in a near future [3].

[1] <
https://github.com/boostorg/build/blob/develop/example/complex-testing/jamroot.jam
>
[2] <https://trello.com/c/HIfba2d9>
[3] <https://trello.com/c/s7ceRe1H>

-- 
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail