From: Eddy L O Jansson (eddy_at_[hidden])
Date: 2004-10-05 22:08:57
Boost / Tokenizer / Escaped List Separator - empty_token_policy patch
This url/archive contains a trivial patch to Boost 1.31.0 which
extends the default escaped_list_separator<> tokenizer function with an
optional empty_token_policy. In addition, a patch to update the
documentation to reflect this change as well as a new example
program demonstrating its use, is included also.
The patch implements the safe default of boost::keep_empty_tokens
such that a patched version of this library behaves exactly as the
old one, unless specifically instructed not to. This is done by
setting empty_tokens to boost::drop_empty_tokens in the constructor.
This gives you the usual benefits of "Escaped List Separator" -- the ability
to specify and use escape and quote characters -- but without the downside
of always having to deal with empty tokens by hand. This simplifies client
The "empty tokens" policy is already available in the less sexy
char_separator<> implementation, I simply brought it forward to
It might be redundant.
I might have fubared the implementation.
-- "IBM clearly did contribute a lot of the Unix-related information into Linux. We just don't know what it is." -- Kevin McBride, legal mastermind. Eddy L O Jansson | http://gazonk.org/~eloj
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk