Re: [BBv2] fix for response file regeneration

From: David Abrahams (dave_at_[hidden])
Date: 2005-10-21 10:10:33

Reece Dunn <msclrhd_at_[hidden]> writes:

> David Abrahams <dave <at> boost-consulting.com> writes:
>> Reece Dunn <msclrhd <at> hotmail.com> writes:
>> > David Abrahams wrote:
>> >> Reece Dunn <msclrhd <at> hotmail.com> writes:
>> >> That just leads me to ask what's missing from the bjam core that we
>> >> can't implement the same behavior without resorting to extending bjam
>> >> specifically for response files?
>> >
>> > 1. The <at> (...) syntax is missing :). This would need to set up a
> special 
>> > "phantom" target, so the BBv2 logic can then generate the response file.
>> 
>> Yes, of course. The syntax counts, but that's now what I was
>> concerned about. If we move to a Scons engine we can process syntax
>> however we like but we'll want to create something like real targets
>> under the covers.
>
> Sure.
>
>> > 2. The main problem with the current response file code is that RSP 
>> > files are being treated as a target, so if you specify to always 
>> > generate the RSP file, it will recompile and relink the project that 
>> > uses them. Thus, we need some sort of "PHANTOM" or "NONTARGET" type that 
>> > will mean that we can have:
>> > NONTARGET $(rsp) ;
>> > ALWAYS $(rsp) ;
>> >
>> > where NONTARGET will not cause the specified target to be used in 
>> > determining whether other targets should be rebuilt. 
>> 
>> Isn't that what TEMPORARY is supposed to do? The idea is that you can
>> delete TEMPORARY object files after creating a library without causing
>> the library to be rebuilt.
>
> The problem is that if the target fails, the response file is kept around *but*
> the action to regenerate the response file isn't called, so the old response
> file is used. We could fix TEMPORARY so that the action will be called if the
> target exists but in such a way that will prevent this from causing a rebuild of
> the dependant target unless that target needs updating. I would be happy with
> this solution.

Have you looked at the RMOLD rule?

>> > > In the long term, we might use Scons' build engine, but
>> >> that's just Python. We can extend it (if necessary) to do anything we
>> >> like.
>> >
>> > What impact would moving over to SCons have on people who are
>> > familiar with the BBv2 way of doing things?
>> 
>> It depends how much work we're willing to do to preserve the
>> "rule+actions" interface.
>
> I am not too bothered about the rule+action interface, more the things that BBv2
> adds like the features, toolsets, etc.

That stuff *is* BB, so any version on a new foundation would preserve
it.

>> > What about the attempt to port BBv2 to Python? Would it be better to
>> > go down that route and have BBv2py?
>> 
>> No, the low-level Jam build engine is inscrutable at best. I don't
>> want to be saddled with maintaining it forever, even after translating
>> it to Python. Best to let the Scons team maintain the low-level build
>> engine.
>
> So if I understand it:
> * BJam will be BJam+Python allowing you to run Python code from
> Jamfile/Sconsfile build scripts
> * Scons will be responsible for the rule+action+target logic that Jam was
> previously responsible for
> * BBpy will add features, toolsets and all the fancy stuff I expect from BB :)

If we get to the point of replacing the build engine with Scons, yes.

>> > 2. Matt's Jam branch (MJam) has builtin support for the <at> (...) syntax.
>> >
>> > 3. MJam response files are created as temporary files so it won't be 
>> > easy to move them into bin/gcc/release style directories.
>> 
>> Why would we want to move them?
>
> Response files are created in bin/gcc/release/main.rsp with BBv2 whereas with
> Matt's code (as far as I understand it) they will be in /tmp/file1214.rsp which
> is not where BBv2 wants them (especially when we want to keep the response files
> for failed targets).

Why would we want to keep the response files for failed targets? Oh,
right, it's that whole mess with wanting to recreate the command
manually :(

>> > Doing this will mean that - with the existing BBv2 response file support 
>> > - response files are deleted if the target succeeds and are retained if 
>> > it fails. When rebuilding a failed target, the response file will be 
>> > regenerated. Thus this fixes the initial response file problem.
>> 
>> That sounds right. I think that's also what BBv1 does :)
>
> If BBv1 has a fix for this situation (failed compile, change defines/includes,
> RSP file not regenerated), we may need to look at porting the fix over to BBv2
> but that depends on how different v1 and v2 are.

Oh, that one's easy: BBv1 doesn't attempt to put anything other than
source files in RSPs. :)

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com