Re: [boost] [tokenizer] tokenizing by strings rather than chars

Subject: Re: [boost] [tokenizer] tokenizing by strings rather than chars
From: Stewart, Robert (Robert.Stewart_at_[hidden])
Date: 2011-05-31 12:53:07

Andrzej Krzemienski wrote:
> Rob Stewart wrote:
> > Andrzej Krzemienski wrote:
>
> > Spirit.Qi would do that handily.  Something like this should
> > work:
> > namespace qi = boost::spirit::qi;
> > std::vector<int> data;
> > qi::phrase_parse(input.begin(), input.end(), *qi::int_,
> >    qi::eol, data);
>
> Thanks for the reply. And could I ask a similar question? I
> wanted to simplify my real task to make it shorter and clearer
> to understand, and it look like I changed it to a different
> one. My apologies.

NP

> My real task is to check if the data in Windows clip-board is a
> column of data (copied from a grid or table). Such a column is
> just text separated with "\n\r" sequences. So, for example
> these are valid data:
> "12" - a column of height 1 with data "12",
> " a\n\r b" - a column of height 2 with two with data " a" and "
> b"
> "-1\n\r\n\r3" - a column of height 3 with datum "-1", empty
> cell, and datum "3".
>
> The spirit solution above will skip the empty cell in my third
> example. On the other hand, I do need the information about
> empty cells too.

In that case, you'd use qi::parse(), because the qi::eol characters become significant (you don't want them skipped -- ignored).  However, there's an issue: you need to know whether you actually got a value since you need to account for empty cells.  Presuming that there's a sentinel value to use to indicate no value, such as std::numeric_limits<int>::min(), then, this should work:

qi::parse(input.begin(), input.end(),
   *(
      (
         qi::int_
         | qi::attr((std::numeric_limits<int>::min)())
      )
      >> qi::eol
   ),
   data);

If there is no sentinel value, then you could use a std::vector<boost::optional<int>> and qi::attr(boost::optional<int>()).

HTH

_____
Rob Stewart                           robert.stewart_at_[hidden]
Software Engineer                     using std::disclaimer;
Dev Tools & Components
Susquehanna International Group, LLP  http://www.sig.com

IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.