[Boost-users] boost_tokenizer gives unexpected result

Subject: [Boost-users] boost_tokenizer gives unexpected result
From: andrew.marlow_at_[hidden]
Date: 2010-06-17 08:28:10

I am getting a result from the boost_tokenizer that I do not expect. I 
wonder if anyone could explain.

I have a line which only contains one occurence of the delimiter and it 
happens to be the first character. I would expect tokenization to yield 
two tokens,  The first would be the empty string, the next would be the 
word that occurs after the delimiter. What actually happens is there is 
only one token, the word I am after. I do not get the expected empty 
string first. This seems wrong to me. Can anyone comment please? Here is 
the (simplified) code:

boost::char_separator<char> emSeparator("M");
boost_tokenizer emTokenizer(line, emSeparator);
boost_tokenizer::const_iterator it2 = emTokenizer.begin();
std::cout << "first token = [" << *it2 << "]" << std::endl;

I am using boost version 1.42.0 on Windows-XP SP3, compiling with VS 2005 
(aka vc8).

Regards,

Andrew Marlow

___________________________________________________________
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/information/legal_information.asp?Code=ECAS-845C5H  for additional disclosures.