Re: Reminder: Tokenizer formal review

From: jbandela_at_[hidden]
Date: 2001-06-01 18:10:44

> For the templated function make_token_iterator, would it be 
reasonable to
> deduce the template argument "Type" from
> iterator_traits<Iterator>::value_type ?
> 
> so rather than having to specify it as in your example code as
> 
> const string test_string = "...";
> Iter i = make_token_iterator<string>(test_string.begin(), 
test_string.end()
> ... );
> 

I think the value_type of a string iterator is char, whereas we want 
string. In addition, the template parameter (string in this case) 
specifies the return type when the iterator is dereferenced. It is 
possible to write a parsing functor that gives  back a different type 
from string. For example, you could write a functor that would read 
columns of numbers and return double's instead of string.

> I was also wondering if this class could be used with the regex 
library, to
> describe the parsed tokens. (I haven't used that library either, so 
its more
> of a have you tried this question.)
> 

I do not have much experience with regex. However, I think it would  
be possible to write a functor that uses regex to do the parsing.

Thanks for your time and comments,

John R. Bandela