Re: [Boost-users] tokenize string delimiter

Subject: Re: [Boost-users] tokenize string delimiter
From: Pavol Droba (droba_at_[hidden])
Date: 2010-02-18 15:42:51

Hi,

On 11. 2. 2010 22:09, Jeff Flinn wrote:
> Matthias Vallentin wrote:
>> On Wed, Feb 10, 2010 at 04:57:41PM -0800, Marshall Clow wrote:
>>> Can anyone tell me why iter_split (and split, etc) requires a
>>> vector<string> to hold the results? (or, more generally, a container)
>>
>> In general, the split function receives a string as input and returns an
>> array. (BTW, this is consistent with the majority of scripting
>> languages, such as Ruby, Python, etc.) Because the split function a
>> priori does not know in how many parts it will chop the input string,
>> and it is likely to be more than one, a vector fits naturally for this
>> task.
> 
> I think what Marshall is saying is that an output iterator for example
> as provided by std::back_inserter is a more natural, less limiting fit.
> If the final destination is not a vector, why copy to a vector, just to
> then copy from the vector to the final destination.
> 

If you would like to see iterator based tokenization, you can use
find_iterator and split_iterator that is an underlying engine below
iter_split.

There are examples in the documentation.

In addition you can use reference trick vector<iterator_range> that will
not copy string just stores the references to the found matches.

In summary, it is up to the user to decide what is more appropriate.
vector<string> is more convenient an easier to user, find/split_iterator
provides less overhead.

Best Regards,
Pavol.