Re: [boost] Re: Preliminary submission: command line & config file library

From: Vladimir Prus (ghost_at_[hidden])
Date: 2003-01-15 08:03:54

Gennadiy,

> About month ago, while working on Boost.Test issues I was faced with the
> need for the more or less full featured command line argument parser. I
> recall that you were working on one and took a look on some of your
> preliminary code in vault area. 

What code, specifically? The only think I remember placed there is
a low-level command line parser, which was never meant to be the
single solution. In fact, it is now meant only for

1. "I don't want to depend on Boost" use cases
2. Possibly, for adding some very custom hooks.

> And ... was really disappointed. I do not
> believe that any "solid" design could fit for variety of different
> needs/expectations that exist. One big class with many many options/styles
> does not look like good design decision. I was not able to look into deep
> details with your current submission, but from what I view even if it better
> it still has similar design. While we are at it I could make 2 remarks about
> your submission: a) None of 5 compiler configurations installed on my XP
> could not compile it. 

:-(
I've made some corrections recently. Did anything improve?

> b) see separate post regarding you config file design
> with is also unsatisfactory IMO.

I have not seen any separate post. Are you sure you've sent it?

>    So I made my move and implemented my own solution, that I will be call:
> CLA framework. It's still missing some bits and pieces, particularly docs is
> not started yet. But I will present it as an alternative to your submission.
> (Though I may not be able to work on this within following month due to
> extreme load with other activities). Here is short description of my design.

Is it possible to take a look at the code, too?

> Rationale.
>   While working on this design I was trying to concentrate on:
>      1. Usability
>      2. Flexibility
>      3. Usability
>      4. Flexibility
>      5. Minimal dependencies (specifically on other boost libs)
>      6. Clarity/simplicity

That's quite general goals, and I had them too.

> cla::parser
>    Class cla::parser is core class in a framework. It's definition looks
> like this:
> template<typename LookupPolicy>
> class parser : public LookupPolicy {
>     typedef typename LookupPolicy::policy_base_parameter
> policy_base_parameter;
> public:
>     parser& register_parameter( policy_base_parameter const& parameter );
>     template<typename Modifier>
>     parser& accept_modifier( Modifier const& m )
> 
>     iterator first_argument() const;
>     iterator last_argument() const;
> 
>     std::string const& error_msg() const;
> 
>     bool is_valid() const;
> 
>     bool parse( int& argc, char** argv );
> };

OK, disagreement begins. I thought that someone could suggest using policies.
But considering it, I came to conlustion that they don't give anything. 
Program options are just (name, value) pairs. If you add a mechanism to
store them into program variables, you have everything.

Further, assume this situation. My program would like to handle options that 
can be specified at
1. Command line
2. Config file (ini)
3. An XML config file
4. Windows registry

Things like validation, and default values really apply to all of them.
I'd like to declare all options once, and then get them from those sources.
In your design, I'd have to declare 4 different parsers, feed them all with
options, and the parse all the sources, right?

[...]
> Lookup policy
>    Lookup policy is in a sense an algorithm for parsing the arguments from
> the input provided. 

If it's the parsing algorithm, then why it's called "lookup policy"?

> Each specific lookup policy could incur what type of the
> parameters/modifiers it's supports. Currently framework itself present
> following lookup policies;
> 
> key_lookup_policy          - each parameter is identified by unique key
> positional_lookup policy  - each parameter is identified by position in
> input
> dual_lookup_policy         - each parameter is identified by dual (most
> probably long/short) key
> getopt_lookup_policy      - getopt style option parsing (with sticky
> options)

 From your description it looks like some of policies deal with parsing
and some with parameter naming. Or are they responsible for both?

> simple_xml_lookup_policy - as a prove of concept lookup policy that allows
> to parse parameters passed in XML format

I don't understand this one. Can you explain?

>   So, how the programmer will define parameters. Here an example:
> cla::parser<cla::key_lookup_policy<> > cla_parser;
> cla_parser << cla::named_parameter<>( "arg1" ) << cla::optional
> 
> << cla::handler<std::string>( &arg1_handler )
>                  << cla::named_parameter<std::list<int> >( "arg2" ) <<
> cla::required
>                  << cla::named_option( "debug" ) << cla::required <<
> cla::default_value( false ) << cla::place_to( is_debug );
> 
> Parameter definition format is defined by cla::parser class and always looks
> like this:
> <parser> [<< lookup policy modifiers]
>                [<< parameter [<< parameter modifiers]]*;

There are only syntactic differences with

     http://zigzag.cs.msu.su:7813/program_options/html/variables_map.html

However, some of them are important for me. I'd like to have as little
typing as reasonable, so
1. "cla::named_option" is extra typing for a very common case
2. "cla::required" is extra typing for a very common case.
I've tried to make such common case as simple as possible:

    options_description desc;
    desc.add_options()
    ("foo", "arg", "interesting option")
    ("bar", "arg?", "another one")
    ;

I really think that the "arg?" notation is superiour to any use
of enums or manipulators. See the first section of

http://zigzag.cs.msu.su:7813/program_options/html/rationale.html

> You could define as many parameters as you want. Allowed types of the
> parameters are defined by the lookup policy (you will need to see the code
> to see how it's working). Any policy may define and support any modifiers
> (modifier is similar to iostream manipulator concept). For example
> key_lookup_policy supports ignore_mismatch modifier, in which case lookup
> algorithm will ignore the errors in input and skip them.
> Any parameter type could also define modifiers. Some of the modifiers are
> supported by all parameters. For example:
> optional,
> requires,
> description( string )
>   Note again that any custom policy could define it's own parameters and
> modifiers

Say, I'm writing a generally-usefull module. Unfortunately, it has about 40
options, which should be customizable by users. With your design the module
would add those options to the global parser. How to use the module inside
other application? The parser has different lookup policy, and the code which
adds options no longer works.

I'm not sure policies are good here.

> Argument access.
> 
> The framework provide several ways to access the parsed argument values
> 1. Through first_argument/last_argument interface. It will work if parameter
> will create typed formal argument (inherited from typed_argument<T> )
>    It will look like this:
>    cla::parser<...>::iterator it = parser.first_argument();
>    cla::argument* arg = *it
>    long value = cla::get_value<long>( arg  );

What happens if no type is declared for this value? How get_value works?

> 2. Most parameter types supports cla::place_to modifier that allows to set
> the argument value directly to the variable supplied.

OK. (My library supports this too)

> 3. Most parameter types supports cla::handler modifier that allows to set
> specific handler (any handler with predefined interface) for any parameter
> parsing event. That allows to implement event driven parameters parsing.

Would be interesting to see it's interface.

> 4. Since parser is publicly inherited from LookupPolicy it could always
> extend it's interface. Particularly key_lookup_policy add operator[] access
> with key argument. You will need to use cla::get_value then to get the
> value.

I'd like to backtrack again to command line + Registy use case.
How can it be supported? Do I have to copy all arguments from command line
to my own location and do the same with registry? Although my library
does not handle registry now, adding the support is trivial. Usage would be

   variables_map vm;
   // init vm from command line
   registry_variables_map r_vm(...);
   vm.next(&r_vm);

   vm["server"]; // Look up the value in vm, and then in r_vm,
                 // if not found in vm.

> Public interfaces
>    Even though I was trying to minimize the dependencies. class framework
> may still be a burden to include in many compilation units. In case if one
> wants to access cla::parser in different (from where it was created/parsed)
> file, lookup policies supports public virtual interface that defined in one
> header with no dependencies. So including namely this one header is enough
> in most case to get an access to the argument values.
> 
> Key lookup policy design
> 
> class template key_lookup_policy defined like this:
> 
> template<typename KeyPolicy = default_key_policy<> >
> class key_lookup_policy : public KeyPolicy, public key_based_interface {
> public:
[...]
> };
> 
> This class is parameterized with Key policy that encapsulate logic with Key
> identification, key prefix and key/value separator. This policy allows to
> set arbitrary string/char as allowed prefix/separator (or even several of
> them), may successfully skip mismatched tokens and more.

What I don't like about it is that you seem to mix syntactic and semantic 
things. The "Key_policy" parameter as you describe it just returns the
set of option,value pairs. The key_lookup_policy seems to control how
options are accessed. Why don't separate this two aspects? I mean
making KeyPolicy a parameter of parser, not of key_lookup_policy

> Hope this overview gave you some idea how the framework will work.

I'm still very much interested in seeing the code. If you plan to
sumbit your library as an alternative, it's quite important that
we two don't say "other's library is bad", but rather summarize all
the differences, in a form that revievers can understand.

Personally, I hope that you'll at least agree with validity of
some concerns I state above.

- Volodya