Re: [Boost-users] [spirit] how to get_position() from a multi_pass<position_iterator <const char*> >

Subject: Re: [Boost-users] [spirit] how to get_position() from a multi_pass<position_iterator <const char*> >
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2009-08-23 09:27:48

> >> I read something about a "base()" method for wrapping iterators, but
> >> could not find one in the multi_pass sources.
> >
> > Ok, I understand. But I think your whole approach is not viable.
> Regex_p requires a bi-directional iterator, while multi_pass exposes an
> input_iterator Only. So generally, even if it compiled it wouldn't work
> in the end.
> 
> I see, obviously I did not read the documentation closely enough, I
> just found out that it states regex_p requires a bidirectional
> iterator and multi_pass exposes a forward iterator.

Yeah, sorry, I meant to say forward iterator as well...

> Nevertheless the combination multi_pass/regex_p compiled and worked
> fine for me.

I think it depends on the regex you used. Not all regex require
bidirectional support from the iterator. 

> regex_p even worked fine with only the position_iterator (until you
> changed the position_iterator iteration type) which made it no longer
> compile with regex_p.

IFAIR, the position_iterator was always a forward iterator only as well (at
least starting from Boost 1.31, i.e. since it's using V2 of the iterator
library). Making the position iterator bidirectional should be possible, but
for this you need to add backwards position tracking, which isn't
implemented.

> (I used regex_p in the first place because lexeme_p included leading
> whitespace (which is really a nuisance to me), regex_p does not create
> this issue).

Yeah, you seem to fight this error in the tree code by inventing all kind of
sophisticated workarounds. I'm sorry for the trouble. But as said before, I
doubt somebody will find the time to get back to this. At least not before
Spirit 2.1 is finished.

> > But to answer your original question, multi_pass doesn't allow to
> access the base iterator, which might be an oversight. I'll have a
> closer look and add it to the multi_pass iterator if possible.
> 
> Thank you for this.
> 
> I wonder if it would be possible to create some multi_pass like
> iterator which allows regex_p to be used with a position_iterator
> (both are very handy tools and it is quite of sad that they cannot be
> used together).
> 
> Is there a solution to this in Spirit V2.1?

Well, for one Spirit 2.1 doesn't expose this bug you're working around. But
even if the multi_pass there has been rewritten (is faster and has a smaller
footprint) it exposes still a forward iterator only. That's mainly because
this is sufficient for Spirit parsers. The position_iterator has been
rewritten as well, but it's not finished yet. If you look through the Spirit
maoling list archives you should be able to find it. It's not bidirectional
still, but if you like you can add this functionality yourself and
contribute it back.

Spirit V2.1 has no regex component yet, and I think we won't add it again.
Spirit now has a full-fledged lexer you can use to tokenize the input stream
using regular expressions.

> Nevertheless thank you a lot for your work on spirit, it is great! :)

You're welcome!
Regards Hartmut

PS: Could you post further questions with wrt Spirit on the Spirit mailing
list? This way other people might be able to join the discussions.