Boost logo

Boost-Build :

From: Reece Dunn (msclrhd_at_[hidden])
Date: 2005-10-31 03:20:47


Rene Rivera wrote:
> Reece Dunn wrote:
>
>>The results of the discussion on the "[BBv2] fix for response file
>>regeneration" thread was to have builtin support for response files
>>using the @(...) syntax based on Matt Armstrong's perforce Jam branch.
>>
>>The bjam-response.diff patch adds @(...) syntax support for NT builds of
>>BJam. This is based on Matt's code but was written from scratch and is
>>improved from my initial submission. When you use @(...) the code (in
>>variable.c) expands the variables in the '...' to a temporary file and
>>then outputs the location of that file.
>
> How does Matt's version of the syntax @() work? I'm asking because I was
> going to apply, with some mods, your patch but I decided not to because
> I have some concerns. Instead I just refactored the tmpdir function in
> preparation, with my own simpler implementation (both Windows and Unix).

Matt's code created a cmd_string method that was a duplicate of
var_string, but with a check for @(). It reads the string between the
parentheses, expands that to a temporary buffer using var_string and
then writes that to a temporary file. The temporary files are (in Matt's
code) kept in a linked list and managed by a tmpunix.c file.

I simplified Matt's code somewhat, moving the response file logic into
var_string. The var_string_file function was added to avoid writing the
strings to a buffer first.

> My basic concern is that as you implemented @() is the limited response
> file specific functionality. I really loath adding generic looking
> syntax that isn't generic. The limitation is that there is no control
> over what the name of the temporary files are. In particular they are
> fixed to have an ".RSP" extension.

This was based on what Matt's code did, but having the ability to name
the file is a good idea.

> Since I'm not one to usually be
> critical without having some suggestions one how to fix things here's
> what I'm thinking...
>
> The "@()" is ultimately a "here document" functionality. With that in
> mind I'd rather have it be as flexible as possible. My suggestion is to
> use the standard variable expansion syntax to manage control of where
> and how the document is put together. Like so:
>
> 1) For a temporary file:
>
> @($(TMPFILE):S=.txt:E="This is some content.")
>
> And like your changes the @() would evaluate to the name of the
> temporary file.

I suggest that we also have the ability to control the lifetime of the
file. Note that your syntax will complicate parsing and processing:

@($(<[1]:W):F=.rsp:E=""$(>)" "-I$(INCLUDES)" -D$(DEFINES)")

vs.

@("$(>)" "-I$(INCLUDES)" -D$(DEFINES))

How about:

@("$(>)" "-I$(INCLUDES)" -D$(DEFINES))="$(<[1]:W).rsp"

> 2) For doing std output:
>
> @($(STDOUT):E="Some console output")
>
> This would allow writing actions that take standard input without having
> to resort to echo. For example:
>
> actions xyz
> {
> @($(STDOUT):E="$(>)") | sort > $(<)
> }
>
> 3) Similarly for STDERR.

How about:

@("$(>)" "-I$(INCLUDES)" -D$(DEFINES))=$(STDOUT)
@("$(>)" "-I$(INCLUDES)" -D$(DEFINES))=$(STDERR)

> Note: the $(TMPFILE) and $(STDOUT) could just as well be TMPFILE and
> STDOUT (without the $()). But the $() seems more fitting to me.

Sure. I like the $() syntax as well.

> The above has enough flexibility to be useful in many context. In
> particular, in addition to the response file use, it would allow for a
> less kludgy implementation of the "print" module. It would no longer
> need to do the funky string escape manipulations to get over the shell
> echo command use. And because of that reduce the memory consumption
> (print.text is one of the top 3 memory consuming rules in BBv2). And
> apply that to any other place where BBv2 needs to escape strings so they
> can be handed to the shell echo or some other command that can handle
> std input.

That would be really neat. Note that in my code (with var_string_file),
each string is separated by a newline character, not a space. Otherwise,
you get a long like that is not too readable in the response file. If
this is an issue, we sould pass the separator in the @(...) syntax, e.g.:

@("$(>)" "-I$(INCLUDES)" -D$(DEFINES)|$(nl))="$(<[1]:W).rsp"
^^^^^^

- Reece

 


Boost-Build list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk