[boost] [tokenizer] Some remarks

Subject: [boost] [tokenizer] Some remarks
From: Robert Kawulak (robert.kawulak_at_[hidden])
Date: 2013-10-09 16:32:10

Hi,

I have a few proposals/questions related to boost::tokenizer:
- the first template parameter of char_separator and other tokenizer functions
could be defaulted to char to save typing and make it consistent with the
defaults of tokenizer;
- making the constructor of char_separator and other tokenizer functions
non-explicit would allow for less verbose construction of a tokenizer;
- the docs state that char_delimiters_separator is deprecated in favour of
char_separator. Are there any plans to actually drop the former and change the
default for the template parameter of tokenizer to the latter? I guess this
might be controversial because interfaces and semantics of the two differ, so
maybe providing a new template, say string_tokenizer, is a good idea?

Combining the first two points, the following:

    auto t = boost::tokenizer<boost::char_separator<char>>(
        str, boost::char_separator<char>("-;|"));

could be written more conveniently as:

    auto t = boost::tokenizer<boost::char_separator<>>(str, "-;|");

and adding the third point it would become:

    auto t = boost::string_tokenizer<>(str, "-;|");

which is IMO a significant improvement. Any thoughts?

Best regards,
Robert