Re: [boost] Re: Boost.Test token_iterator_test failure

From: Jonathan Wakely (cow_at_[hidden])
Date: 2004-09-29 04:30:49

On Tue, Sep 28, 2004 at 01:26:32PM -0400, Gennadiy Rozental wrote:

> > For GCC 3.4.x the debug mode reports this:
> > error: attempt to copy-construct an iterator from a singular iterator.
> >
> > Are you sure you're passing l.end(), or is that a simplification?
> 
> I does simplified example a bit. I run test under gdb and found it fails in
> line similar to one above.
> 
> > Loads of iterators in token_iterator_test.cpp are singular, so when
> > passed to BOOST_CHECK_EQUAL_COLLECTIONS (which takes it's args by value)
> > a singular iterator is copied - which is illegal
> > e.g.
> > test_custom_compare()
> 
> What means singular in this context? I do not see any problems with
> BOOST_CHECK_EQUAL_COLLECTIONS calls.

http://www.sgi.com/tech/stl/trivial.html says:

    A Trivial Iterator may have a singular value, meaning that the
    results of most operations, including comparison for equality, are
    undefined. The only operation that a is guaranteed to be supported
    is assigning a nonsingular iterator to a singular iterator.

Specifically, copying a singular iterator is undefined (but works for
lots of stdlibs - libstdc++ debug mode is much stricter)

Default constructed iterators are often singular, a bit like an
uninitialised pointer - they don't point at anything and can't be used.
Although AFAIK it's legal to copy an uninitialised pointer, it's
undefined to try and do the same with a singular iterator.

> Here is actual line which causes problems:
> 
> typedef utf::range_token_iterator<std::list<char>::iterator>
> my_token_iterator;
> my_token_iterator tit( l.begin(), l.end() );
> 
> range_token_iterator implementation tries to build first token and use
> method assign for string with two iterators. Second iterator wouldn't be end
> but does it matter?

No, that sounds OK to me. That's not where I see an error though.

This ctor is the problem:

     explicit    range_token_iterator() {}

This default constructs the two iterator members, m_begin and m_end,
which may then be singular (depending on the type Iter).

If I compile the token_iterator_test with debug mode and run it in GDB I
see an exception being thrown because of line 121:

117:    my_token_iterator tit( l.begin(), l.end() );
118:    char const* res[] = { "a", "bc", ",", "cd" };
119:
120:    my_token_iterator end;
121:    BOOST_CHECK_EQUAL_COLLECTIONS( tit, end, res );

For e.g. std::list<char>, a default constructed iterator is singular, so
"end" is singular, and BOOST_CHECK_EQUAL_COLLECTIONS takes its arguments
by value - this copies a singular iterator - bang, you're dead.

If the default ctor is used then the iterator must not be used for
anything, other than assigning a non-singular iterator to it. Copying
it is not allowed, comparing it is not allowed, dereferencing it is not
allowed etc. etc.

This is even worse, this constructor might create a singular iterator,
which is then copied right away:

     explicit    range_token_iterator( Iter begin, Iter end = Iter() )
     : m_begin( begin ), m_end( end )
     {
         this->init();
     }

Same here:

    template<typename Iter>
    inline range_token_iterator<Iter>
    make_range_token_iterator( Iter begin, Iter end = Iter() )
    {
        return range_token_iterator<Iter>( begin, end );
    }

Bang, you're dead.

The basic_string_token_iterator seems to have exactly the same problem
with its constructors.

> > Also, why is test_wide() commented out for GCC (without STLPort) ?
> > I thought the point of the \nnn octal escapes I worked out was that GCC
> > won't accept the raw characters, but is happy with the escapes?
> > Did that not work? Why should it work with STLPort, it's the compiler
> > that rejects those chars, not the stdlib, isn't it?
> > And it wasn't all versions of GCC that were affected, was it?
> 
> Just glitch from hurrying to fix it and making couple things together.

Sorry for the tone of my questions - I was having a bad day yesterday and
that came across quite badly.

jon

-- 
"A well-written program is its own heaven
 A poorly written program is its own hell"
	- The Tao of Programming