Re: [Boost-users] tokenize string delimiter

Subject: Re: [Boost-users] tokenize string delimiter
From: OvermindDL1 (overminddl1_at_[hidden])
Date: 2010-02-11 17:22:53

On Thu, Feb 11, 2010 at 2:09 PM, Jeff Flinn
<TriumphSprint2000_at_[hidden]> wrote:
> Matthias Vallentin wrote:
>>
>> On Wed, Feb 10, 2010 at 04:57:41PM -0800, Marshall Clow wrote:
>>>
>>> Can anyone tell me why iter_split (and split, etc) requires a
>>> vector<string> to hold the results? (or, more generally, a container)
>>
>> In general, the split function receives a string as input and returns an
>> array. (BTW, this is consistent with the majority of scripting
>> languages, such as Ruby, Python, etc.) Because the split function a
>> priori does not know in how many parts it will chop the input string,
>> and it is likely to be more than one, a vector fits naturally for this
>> task.
>
> I think what Marshall is saying is that an output iterator for example as
> provided by std::back_inserter is a more natural, less limiting fit. If the
> final destination is not a vector, why copy to a vector, just to then copy
> from the vector to the final destination.

You can do that off the bat using Boost.Spirit2.1+ like this:

Let str be the string to split and delim your delimiter. Then,

    // Pick any of these result types, they all work, along with
anything else that support push_back and a few other things
    std::vector<std::string> result;
    std::string result;
    myCustomClassThatSupports_push_back result;
    parse(str.begin(),str.end(), raw[+~char_(delim)]%lit(delim), result);

Or case insensitive:

    parse(str.begin(),str.end(),
nocase[raw[+~char_(delim)]%lit(delim)], result);

And there is *SO*MUCH* more you can do with spirit too, and it
executes faster then the tokanizer and such as well, while working on
input iterators, etc...