Boost logo

Boost Users :

Subject: Re: [Boost-users] regex iterator question
From: John Maddock (john_at_[hidden])
Date: 2009-03-30 11:47:51

> I've got a need for regex and I would like to use it to extract tokens
> matching a regular expresssion from a file stream. Seems like this would
> be a common desire.
> So first shot is
> boost::regex_token_iterator<some_input_iterator>
> This doesn't work since regex_token_iterator requires a bidirectional
> iterator. Seems reasonable enough.
> So next I comb through boost and find
> #include <boost/spirit/iterator/multi_pass.hpp>
> and try
> boost::regex_token_iterator<
> boost::spirit::multi_pass<some_input_iterator>
> I'm thinking this is veeeeeery cool - maybe a 1000 lines of free code
> included for the price of one.
> But, doesn't work. multi_pass is a forward_trasversal iterator while
> regex_token_iterator requires a bidirectional_trasversal_iterator. A huge
> disappoint to come soooo close.
> Thinking about it, this problem must come very often. How is it usually
> addressed? There must be a simple bridge across this. In a pinch, I'll
> just have to load the whole file into some sort collection, but I prefer
> the ultimate unlimited file size solution.

If you check the regex examples there are some "load_file" routines than
dump a files contents into memory, but I agree it's not an ideal solution.
I did experiment with some adapters to solve this issue in the early days of
regex but never got a really good solution, and folks weren't demanding it
so it got dropped :-(

But.. how about a memory mapped file? Boost.Interprocess has support for
although I admit it's not quite a one liner...

HTH, John.

Boost-users list run by williamkempf at, kalb at, bjorn.karlsson at, gregod at, wekempf at