[boost] Re: Preliminary submission: command line & config file library

From: Vladimir Prus (ghost_at_[hidden])
Date: 2003-02-04 03:49:08

[bringing to list again]

Hi David,

David A. Greene wrote:
> Vladimir Prus wrote:
> 
>>> -fcopyPropagation={--maxTransforms=32 --keepStats}
> 
> 
>> Hmm... that's pretty interesting example. While I'm not sure how 
>> Gennadiy's
>> code can handle such things, I have some ideas w.r.t my library.
> 
> 
> Good!  Glad I could inspire some thought.  I went over this a couple of
> years ago on a long Boost CLA thread and got absolutely nowhere.  It
> seems like many people are opposed to such nesting.  I've found it to
> be extremely useful.

You might find it interesting that the first version of my library was
created for use in research assembler optimizer, which had several modules
with private parameters. So I understand you desire. I've used nested
options names, line "copyPropagation.maxTransforms" (and I guess I suggested
that), but your approach is much less verbose.

>> 1. You'd need some way to extract those composite options from the 
>> command line. I assume that matching braces suffice. Then, you'd write 
>> an additional
>> parser (just a function), which can extract "copyPropagation" and
>> "{--maxTransforms=32 --keepStats}" from the above option. That 
>> function will
>> be passed to the command line parser.
> 
> 
> Ok, I'm not completely clear about what you mean here.  I was thinking
> about doing things in a Spirit-like fashion with nested grammars.  The
> nested options in fact look exactly like "top level" options.  They
> just get parsed/processed internally by the copy propagation filters.

Yea, the copy propagation module provides a function to parse its options,
which function may well do a second pass of parsing.

> Someone had the idea of building a CLA option tree.  I think that's not
> a bad idea because validation may become more powerful.  For example,
> one could check that conflicting options are not specified.

I am not sure that is "option tree" -- don't remember ever hearding this idea.
Is it meant that after parsing you have a tree of option values: modules at
the top level, module options at the second, and so on?

>> 2. Since you say that only CopyPropagation module should care about 
>> content of
>>    braces, I think that module can simply register option 
>> "copyPropagation"
>>    and a custom function ("validator"), which will be responsible for
>>    interpreting the content of value. For example, it can split the 
>> content
>>    on whitespaces and parse it like a command line of it's own.
> 
> 
> That's essentially what our software does now.  It's just that our
> parser is an ugly piece of hand-written code and I want to clean it
> up.  :)

Okay, then it looks like perfectly possible!

>> The only problem with this scheme is that now additional parser can 
>> operate on
>> one token only. However, this restriction is not fundamental and can 
>> be lifted easily.
> 
> 
> In our system we get one token with a big string value but we just
> recursively invoke the CLA parser to break up the string into
> multiple tokens.

My command line parser operates on argv elements without concatenating them
(for example, to allow option values/arguments with spaces in them). So
currently there's no way to extract the string between "{", "}". But as
I said, this is easily fixable. Then, the string between braces can be
tokenized and parser, just as you say.

>> How does the above suggestion look?
> 
> 
> I think it's pretty close to what our system has now, only better
> formalized.  I'm kind of intrigued by the idea of a two-pass system
> where the parser simply builds an option tree and a validator
> checks it and/or invokes actions based on the values.  These
> validators could be nested such that the top-level validator
> invokes sub-validators (for copy propagation in this example).

I think we might try to do some reality check. If you send me
description of options for a couple of modules, I might try to
come with a working solution. (Of course, you command line
parsing code would be interesting to look at, if possible).

> This assumes that all nested options have a common format which
> may not be desirable.  Of course we could also use the Spirit
> idea of nested grammars to overcome this difficulty.

There's no assumption of common format, in fact. If you have command line

   -fcopyPropagation={ ..... }

then it's required to clearly delimit the option's value. In this case, the
braces are just fine. After that, the validator deals with concent of braces.
It can use the same parser recursively, or it can construct a Spirit parser 
and use it, or just split the content on commas.

- Volodya