Re: [Boost-users] tokenize string delimiter

Subject: Re: [Boost-users] tokenize string delimiter
From: Matthias Vallentin (vallentin_at_[hidden])
Date: 2010-02-12 19:01:21

On Fri, Feb 12, 2010 at 08:57:53AM +0000, Brian O'Kennedy wrote:
> The first argument to qi::parse is a non-const reference (which fails on the
> temporary you're passing in).

That makes sense (and fixed it).

> Try:
> 
>   std::string::iterator ii = str.begin();
>   boost::spirit::qi::parse(ii, str.end(),
>            raw[+~string(delim)] % lit(delim), result);

This grammar definition does not work for me, I get an assertion failing
with: subject_is_not_negatable, which makes sense as the complement of
string is not really defined. 

When we parse into std::vector<std::string>, this seems to be a
straight-forward solution:

    using namespace boost::spirit::qi;
    using namespace boost::spirit::karma;

    std::string str("foo---bar---baz");
    std::string delim("---");
    std::vector<std::string> result;
    
    std::string::iterator i = str.begin();
    parse(i, str.end(), 
            +alpha % lit(delim), result);
    
    std::cout << format(stream % ", ", result) << std::endl;

How do we have to modify the above when result is of type std::string?
The example below merely puts "fbb" in the result string:

    std::string::iterator i = str.begin();
    parse(i, str.end(), 
            raw[+alpha] % lit(delim), result);

According to attribute composition rules, I would assume the grammar
attribute is equivalent to vector<iterator_range<I>>. How does qi::parse
determine which function to use for the result? Is it merely using
push_back, and hence only the first character is appended? 

> Spirit uses the first parameter to return how far it got in the parsing of
> your string.

Why is a const_iterator insufficient? Shouldn't it also work to report
the position of the parser?

   Matthias
-- 
Matthias Vallentin
vallentin_at_[hidden]
http://www.icir.org/matthias