Re: [boost] [tokenizer] is exception unaware

From: Martin Bonner (martin.bonner_at_[hidden])
Date: 2006-07-18 09:33:36

----Original Message----
From: boost-bounces_at_[hidden]
[mailto:boost-bounces_at_[hidden]] On Behalf Of Janek Kozicki Sent:
18 July 2006 14:21 To: boost_at_[hidden]
Subject: Re: [boost] [tokenizer] is exception unaware

> Sebastian Redl said:     (by the date of Tue, 18 Jul 2006 14:06:59
> +0200) 
> 
>> Janek Kozicki wrote:
>> 
>>> 	boost::tokenizer<>::iterator pos=tok.begin();
>>> - is this IMO bad design intentional?
>>> 
>>> 
>> Boost.Tokenizer is written as to model an iterator. Have you even
>> seen an iterator throw an exception on reaching the end of the
>> sequence? 
> 
> The rationale for container iterators is that iterating over
> containers should not be slowed down by exceptions. Those iterators
> don't even check for the iterator's validity! 
That depends on the implementation of the standard library.  Some
standard libraries /do/ check for the iterator's validity.

> And trying to access beyond the container is not detected by
> the program, but simply causes segmentation fault. That's good 
> for usual containers.

I'd say it causes undefined behaviour, and an assertion failure, a
segmentation fault, and silently returning the wrong value are all
acceptable (although the first two are preferred).

>> So yes, the design is probably intentional.
> 
> However tokenizer::iterator is not a typical container nor typical
> iterator. It is even unusual that it checks for the iterator's
> validity (because trying to access beyond the container causes
> assert(valid_) to fail ). 
> 
> The rationale from typical container iterators makes no sense when
> applied to tokenizer::iterator, because iterating over tokenizer items
> is not fast - it requires parsing of the content.
> 
> There are two solutions to correct this:
> 
> - remove assert(valid_) check from tokenizer, so it follows more
> precisely the standard, and causes segmantation fault on failure.

I don't see how a segmentation fault follows the standard any more than
an assertion failure.

> - do opposite, and accept the fact that tokenizer is not a typical
> container, and replace assert_fail with throw.

-- 
Martin Bonner
Martin.Bonner_at_[hidden]
Pi Technology, Milton Hall, Ely Road, Milton, Cambridge, CB4 6WZ,
ENGLAND Tel: [phone]