Boost logo

Boost :

From: jbandela_at_[hidden]
Date: 2001-06-01 18:23:48


> Code comments / questions:
> 1) I would have expected that the Tokenizer class would have
>
> template <class TokenizerFunc,
> class Token=std::string,
> class Iterator=std::string::const_iterator>
> class Tokenizer {...}
>
>
> Then the third example could be simplified from:
> typedef
tokenizer<string,string::const_iterator,punct_space_separator<char> >
> Tok;
> to
> typedef tokenizer<punct_space_separator<char> > Tok;
>
> In my mind this is much clearer.

I like your idea, as it will simplify the common case where a string
is being is being broken up.

> 2) Is it possible to use a raw cstring instead of std::string as
the input? I
> experimented a bit, but I was unsuccessful in getting this to
work. I have a
> use for this in an application which needs to tokenize data from a
socket
> connection which is returned as a cstring and I wouldn't want the
overhead of an
> extra string construction / string copy.
>

Yes. Use const char* instead of string::const_iterator as the
iterator argument. Leave all the other parameters the same. It will
take the cstring and break it up into a series of strings.

Thanks for your comments,

John R. Bandela


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk