Re: [boost] [tokenizer] is exception unaware

From: Janek Kozicki (janek_listy_at_[hidden])
Date: 2006-07-18 09:21:15

Sebastian Redl said:     (by the date of Tue, 18 Jul 2006 14:06:59 +0200)

> Janek Kozicki wrote:
> 
> >	boost::tokenizer<>::iterator pos=tok.begin();
> >- is this IMO bad design intentional?
> >  
> >
> Boost.Tokenizer is written as to model an iterator. Have you even seen 
> an iterator throw an exception on reaching the end of the sequence?

The rationale for container iterators is that iterating over containers
should not be slowed down by exceptions. Those iterators don't even
check for the iterator's validity! And trying to access beyond the
container is not detected by the program, but simply causes segmentation
fault. That's good for usual containers.

> So yes, the design is probably intentional.

However tokenizer::iterator is not a typical container nor typical
iterator. It is even unusual that it checks for the iterator's validity
(because trying to access beyond the container causes assert(valid_) to fail ).

The rationale from typical container iterators makes no sense when
applied to tokenizer::iterator, because iterating over tokenizer items
is not fast - it requires parsing of the content.

There are two solutions to correct this:

- remove assert(valid_) check from tokenizer, so it follows more
precisely the standard, and causes segmantation fault on failure.

- do opposite, and accept the fact that tokenizer is not a typical
container, and replace assert_fail with throw.

-- 
Janek Kozicki                                                         |