Boost logo

Boost :

Subject: Re: [boost] [utf8_codecvt_facet]
From: Marshall Clow (mclow.lists_at_[hidden])
Date: 2012-12-11 09:32:55

On Dec 10, 2012, at 9:27 PM, Robert Ramey <ramey_at_[hidden]> wrote:

> change Revision: 81616
> Author: marshall
> Date: 11:12:42 AM, Wednesday, November 28, 2012
> Message:
> Fixed incorrect 'const' param for utf8_codecvt_facet::do_length
> ----
> Modified : /trunk/boost/detail/utf8_codecvt_facet.hpp
> This change causes the a failure in the serialization library test
> suite with "test_utf8_codecvt" on all platforms which use
> the gcc standard library.

Ok, I'm seeing that too - the test doesn't "fail" :-(, it just hangs.

> I'm not sure what motivated the change or what it addressed
> but I'm pretty sure it's causing this (and a couple other)
> test failures. Note that all the failures are related to usage
> of wchar_t - perhaps that helps.

Back on June 28th, I posted a message to the list titled "[Survey] utf8_codecvt_facet::do_length parameter" which described the problem:

> Marshall Clow <mclow.lists_at_[hidden]>
> To: boost_at_[hidden] List <boost_at_[hidden]>
> [Survey] utf8_codecvt_facet::do_length parameter
> There's some confusion in boost (and in various standard libraries) as to the declaration of
> std::codecvt<>::do_length
> The '03 and '11 standards say:
>> template <class internT, class externT, class stateT>
>> class codecvt : public locale::facet, public codecvt_base {
>> // blah blah blah
>> virtual int do_length(stateT&, const externT* from, const externT* end, size_t max) const;
>> };
> But some standard libraries (notably older Dinkumware libraries, and Visual C++ before Visual Studio11) say:
>> virtual int do_length(const stateT&, const externT* from, const externT* end, size_t max) const;
> (the first parameter is a const reference)
> There's some preprocessor stuff to deal with this in boost/detail/utf8_codecvt_facet.hpp, but it's wrong:
>> #if !defined(__MSL_CPP__) && !defined(__LIBCOMO__) \
>> && !defined(_LIBCPP_VERSION)
>> #else
>> #endif
>> virtual int do_length(
>> BOOST_CODECVT_DO_LENGTH_CONST std::mbstate_t &,
>> const char * from,
>> const char * from_end,
>> std::size_t max_limit
>> ) const throw();
>> #else
>> ) const;
>> #endif
> I'm happy to fix this, but I'd like to be sure which systems actually have const parameters.
> I've checked gcc and libc++, and they match the standard.
> STL assures me that all versions of Visual Studio before VS11 do not match, but VS11 does.
> The #defines above suggest to me that LibCOMO has this problem as well.
> Does anyone have access to such an install?
> And if anyone's standard library (that I haven't listed) does define the first parameter to std::codecvt<>::do_length as a const reference, please let me know.

FWIW, I got zero responses.

Then Microsoft shipped Visual Studio 12, which also defined the parameter as "not const".
I looked at LibComo, and it too defines the parameter as non-const.

So, after discussing this with Beman (since it's his library), I changed the code to look like this:

// Only do CONST on older Dinkumware libraries.
#if defined(_CPPLIB_VER) && (_CPPLIB_VER < 540)

If that's causing problems for you, I apologize, and I'll be happy to work with you to fix it - though I don't understand _how_ it's causing problems.
I tried changing the macro to always be "const", and the test still hung for me (on both clang and gcc)

Looking into this a bit more, it appears that Boost.IOStreams has it's own #define for this. Sigh.

-- Marshall

Marshall Clow Idio Software <mailto:mclow.lists_at_[hidden]>

A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait).
        -- Yu Suzuki

Boost list run by bdawes at, gregod at, cpdaniel at, john at