Re: [BBv2] fix for response file regeneration

From: Reece Dunn (msclrhd_at_[hidden])
Date: 2005-10-21 03:03:39

David Abrahams wrote:
> Reece Dunn <msclrhd_at_[hidden]> writes:
> 
>>I have the original Jam and Matt's Jam sources from Perforce
>>(public.perforce.com Perforce connection) and am going to start
>>looking at merging the changes into BJam.
>>
>>>I recall we had a long discussion about relative merits of those approaches, 
>>>and did not decide anything. There are some usability questions, like 
>>>removing or not removing files when build succeeds, or when it fails.
>>
>>We can get an implementation up and running, then modify it to best suit 
>>our needs. Having a "native" solution would be a lot better than the 
>>BBv2 implementation we currently have.
> 
> That just leads me to ask what's missing from the bjam core that we
> can't implement the same behavior without resorting to extending bjam
> specifically for response files?

1. The @(...) syntax is missing :). This would need to set up a special 
"phantom" target, so the BBv2 logic can then generate the response file.

2. The main problem with the current response file code is that RSP 
files are being treated as a target, so if you specify to always 
generate the RSP file, it will recompile and relink the project that 
uses them. Thus, we need some sort of "PHANTOM" or "NONTARGET" type that 
will mean that we can have:
NONTARGET $(rsp) ;
ALWAYS $(rsp) ;

where NONTARGET will not cause the specified target to be used in 
determining whether other targets should be rebuilt. Thus:

EXE --> CPP --> RSP
\--> RSP

will be seen as:

EXE --> CPP

(I like the name PHANTOM or PHANTOM_TARGET :))

>>Q: What will happen when we port over to BBpy?
> 
> In the near term, we'll keep the same build engine, so nothing much
> will happen.

Ok.

> In the long term, we might use Scons' build engine, but
> that's just Python. We can extend it (if necessary) to do anything we
> like.

What impact would moving over to SCons have on people who are familiar 
with the BBv2 way of doing things? What about the attempt to port BBv2 
to Python? Would it be better to go down that route and have BBv2py?

> However, I'd like to answer the question I posed above before
> thinking of making that extension. Response files don't seem like
> such a special case that they should require their own build engine
> features.

Ok. I have made an initial port of the code from Matt's response files, 
but haven't had chance to test it beyond noting that it builds and 
doesn't break existing BJam projects. I have noted that:

1. The BJam and Perforce Jam sources have now diverged to a point where 
it is not trivial to integrate changes to BJam from PJam.

2. Matt's Jam branch (MJam) has builtin support for the @(...) syntax.

3. MJam response files are created as temporary files so it won't be 
easy to move them into bin/gcc/release style directories.

4. MJam uses tmpunix.c for creating NT/Unix temporary files: this would 
require porting to the other platforms (e.g. MacOS).

5. The code will require reformatting as the two sources use different 
formatting conventions.

So, as a result, I now agree with you: by adding phantom target support 
(and optionally the @(...) syntax) we can fix the problem, but I have no 
idea how to implement such functionality :(.

Doing this will mean that - with the existing BBv2 response file support 
- response files are deleted if the target succeeds and are retained if 
it fails. When rebuilding a failed target, the response file will be 
regenerated. Thus this fixes the initial response file problem.

NOTE: On the changing properties - if we keep the response file, we can 
use that as our basis for comparison as this stores includes and 
defines. (I am not sure about support for changing other flags/settings).

- Reece