[Boost-users] Re: tokenizer question

From: Dennis Jones (djones_at_[hidden])
Date: 2005-06-02 18:44:35

"Pablo Aguilar" <pablo.aguilar_at_[hidden]> wrote in message
news:d7o1bi$d60$1_at_sea.gmane.org...
>
> While I haven't tried it myself, for the same reason (not available on VC6
> nor BCB), I do think it'd be easier to use than trying to customize
> tokenizer. Also, (and again, I haven't tried it) I believe Spirit 1.6.x
> (spirit.sourceforge.net) is available for older compilers; I know it's
> supposed to work for VC6, but I'm not sure about BCB.
>

While I can appreciate the usefulness of Spirit, I just don't see the need
for it here -- especially since it means bringing in an entire library just
for a single operation in my application.

In the event that tokenizer is incapable of handling my desired
requirements, I have come up with a reasonable work-around: I simply
pre-process the string, substituting the '+' operators with some
un-printable character (like '\x01', or any other character the user is
highly unlikely to type), while leaving any '+' symbols within quotes alone.
Then I use the tokenizer with the '\x01' as the token separator, and voila!

- Dennis