Re: [Boost-users] [boost-users] tokenizer vs string algorithm split.

From: Bill Buklis (boostuser_at_[hidden])
Date: 2007-12-13 12:48:42

This may not matter for the CSV file you're parsing, but at least for a more
general solution for CSV processing, you'd also have to handle fields that
are surrounded by quotes and may even contain embedded commas. I don't know
if split or tokenizer can handle that.

 

-- Bill --

 

  _____  

From: Larry [mailto:lknain_at_[hidden]] 
Sent: Thursday, December 13, 2007 8:08 AM
To: boost-users_at_[hidden]
Subject: Re: [Boost-users] [boost-users] tokenizer vs string algorithm
split.

 

My limited experience is that tokenizer is faster. I have tried it several
times in different schemes but the tokenizer always seems to come out faster
by more than a little. I would prefer the split() scheme but I haven't found
the way to make it go faster.

 

Larry

----- Original Message ----- 

From: chun ping wang <mailto:cablepuff_at_[hidden]>  

Newsgroups: gmane.comp.lib.boost.user

To: boost-users_at_[hidden] 

Sent: Wednesday, December 12, 2007 10:56 PM

Subject: [boost-users] tokenizer vs string algorithm split.

 

Hi I was wondering which one is better and faster to split a file of csv
value of number and put it into container of double. 
1.) Which option is better. 
             // method 1. 
             std::vector<std::string> split_string; 
             boost::algorithm::trim(flist);
             boost::algorithm::split(split_string, flist,
boost::algorithm::is_any_of(","));
             std::vector<double> elements;
             BOOST_FOREACH(std::string s, split_string) 
             {
                 elements += boost::lexical_cast<double>(s);
              }

              // method 2. 
               boost::char_separator<char> sep(",");
               boost::tokenizer<boost::char_separator<char> > tokens(flist,
sep); 
               std::vector<double> elements;
               BOOST_FOREACH(std::string token, tokens)
               {
                   elements += boost::lexical_cast<double>(token);
               } 

2.) When is it better to use string algorithm split instead of tokenizer and
vice versa. 

  _____  

_______________________________________________
Boost-users mailing list
Boost-users_at_[hidden]
http://lists.boost.org/mailman/listinfo.cgi/boost-users