Boost logo

Boost :

From: jbandela_at_[hidden]
Date: 2001-09-09 21:28:42

I was thinking about the following changes to Tokenizer. I would like
some feedback before I implement them.

1. char_delimiters_separator and offset_separator would take an extra
template parameter to specify whether the tokens are assigned using

tok+=*next; OR

(Thanks to Gennadiy E. Rozental for this idea)

2. escaped_list_sepator would allow specifying multiple characters
for escape, separator(c), and quote. This would solve the problem
that Johan Nilsson brought up about empty tokens

3. escaped_list_separator would take another template parameter, to
specify the functor to be used to process escapes. The default would
be tokenizer_default_escape_processor which would have the same
semantics as present. Also available would be a c_escape processor
that would process the C escapes, and a url_escape_processor which
would process url escapes ( ie + for space, and %hh where hh are hex
digits). The last would ease processing of cgi parameters.


John R. Bandela

Boost list run by bdawes at, gregod at, cpdaniel at, john at