[boost] Re: std::string <-> std::wstring conversion

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2004-12-17 22:36:53

Stefan Slapeta wrote:
> Robert Ramey wrote:
>>
>> Note that the implementation above is not dependent on codecvt nor on
>> streams.
>>
>>
>
> This is absolutely crucial. Imagine you have to read XML files with
> many thousands of strings...

> BTW, that's the same reason that lexical_cast isn't applicable for
> many
> real cases. Its catastrophic performance inhibits every usage whenever
> speed _could_ be an issue...

Streams aren't as slow as you think. For converting large amounts of data, a
stream-based approach can quite fast.

The most expensive aspect of using streams are:

 - initialization, if you have to create a new stream each time you want to
convert or format a small amount of data
 - the formatted i/o operations

The code_converter component doesn't incur either cost. A code_converter can be
constructed once and used many times. It can also convert between narrow and
wide characters without invoking any formatted i/o operations.

> I would like to see this conversion functions available at a more
> general place than the serialization library. But probably there is
> much
> more to do to improve internationalization support than only providing
> string/wstring conversion ;)

The code_converter component from the iostreams library is already quite
flexible. It might make sense to have a specialized component for converting
strings, but I'd like it to be general enough to handle any type of string.
Robert's approach, using iterator adaptors, might be a good way to go. But a
Codecvt should be a template parameter:

    template< typename Codecvt,
              typename Iterator
              ... >
    class converting_iterator;

> Stefan

Jonathan