Re: [Boost-users] Help with Tokenizer

From: David Klein (d.klein_at_[hidden])
Date: 2006-06-09 04:39:58

boost-users-bounces_at_[hidden] wrote:
> I have a string that is delimited with '|' that I want to
> parse with boost::tokenizer.
> 
> ie: "label 1|label2||label       3"
> 
> It all parses ok except that the whitespace within the label
> is removed.
> 
> Is there a way to configure the tokenizer to leave whitespace alone?
> 
> Thanks
> Frank
> 
> 
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users

hi frank,
the following snippet works for me (winxp, vc71, boost 1.33.1):

typedef boost::char_separator<char> separator_type;
string s("label 1|label2||label        3");
boost::tokenizer<separator_type> t(s, separator_type("|"));
copy(t.begin(), t.end(), ostream_iterator<string>(cout, "\n"));

output:
label 1
label2
label        3

HTH dave