[boost] Re: Re: std::string <-> std::wstring conversion

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2004-12-17 23:31:34

Beman Dawes wrote:
> At 05:01 PM 12/17/2004, Jonathan Turkanis wrote:
>
>  >The Dinkumware CoreX library also contains a component,
>  wstring_convert, >for this purpose. Since it doesn't make a detour
> through streams, it
> might
>  >be more efficient.
>
> Dinkumware is proposing part of their library for standardization. See
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1683.html
>
> Should Boost be looking at that proposal? Seems to me we should.

I'm aware of this proposal. It's what convinced me to do the current rewrite of
the code_converter component, making a codecvt the first template parameter. By
chance, I finally got around to it today.

boost::io::code_converter is a generalization of the wbuffer_convert template
from CoreX and n1683. While wbuffer_convert is a stream buffer adapter, taking a
narrow-character stream buffer and propoducing a wide-character stream buffer,
code_converter is a Device adapter, taking a narrow-character Device and
producing a wide-character Device. Since stream buffers are models of the Device
concept (http://tinyurl.com/53bvc), code_converter subsumes wbuffer_convert.

wbuffer_convert is declared as follows:

       template< class Codecvt,
              class Elem = wchar_t,
              class Tr = std::char_traits<Elem> >
       class wbuffer_convert : basic_streambuf<Elem, Tr> { ... }

code_converter (as I am rewriting it today) is declared as follows:

    template< typename Codecvt, typename Device,
              typename Alloc = std::allocator<char> >
    class code_converter { ... };

For a given Codecvt type Cvt, wbuffer_convert<Cvt> is essentially the same as

     boost::io::streambuf_facade <  // The adapted streambuf
        boost::io::code_converter< // The adapted Device.
            Cvt, // The code conversion policy
            basic_streambuf<typename Cvt::extern_type> // The underlying Device
        >
     >

(Here basic_streambuf<typename Cvt::extern_type> is the narrow-character device;
applying code_converter yields a wide-character device; finally, applying
streambuf_facade yields a wide-character stream buffer.)

> If we like it, we can support their proposal and perhaps do an
> independent implementation for Boost.

It's easy to implement as a wrapper around code_converter. Should I add it?

> If we think it needs changes, the sooner those get communicated to Dinkumware
> and/or the LWG, the better.

I'd rather standardize code_converter, since it's more general. ;-)
Unfortunately, the Boost Iostreams library hasn't been widely used enough yet to
propose even the core components for standarization. (While it hasn't been
officially released, I know people are already using it since they email me
frequently.)

Assuming the standard iostreams library isn't going to be expanded to
incorporate support for generic devices, I think wbuffer_convert is just about
right.

(One thing I don't understand is why the character type of wbuffer_convert is
allowed to be specified as the second template argument. It seems to me that the
character type should always be equal to Codevt::intern_type.)

> --Beman

Jonathan