Re: [boost] [gsoc] Request Feedback for Boost.Ustr Unicode String Adapter

Subject: Re: [boost] [gsoc] Request Feedback for Boost.Ustr Unicode String Adapter
From: Yakov Galka (ybungalobill_at_[hidden])
Date: 2011-08-16 12:18:56

On Fri, Aug 12, 2011 at 15:29, Daniel James <dnljms_at_[hidden]> wrote:

> [...]
> A part of the popularity of boost is because it works well with
> existing code. So we need to work with the strings we get at the
> command line, from streams etc.
>

It will work well with existing code. The user will just need to convert her
strings to UTF-8 if they aren't UTF-8 already.

 >> Unless a different solution can be found.
> >
> > Exactly.
>
> I meant a different solution to assuming that std::string is always
> UTF-8. It appears unlikely that your proposal will be accepted by
> boost, so what other possibilities are there? Perhaps a distinct
> string type, maybe some mechanism to specify what encoding strings are
> using, or something else entirely?
>

I assume that we all agree that we want to encourage UTF-8, as it's the only
way to handle Unicode on all systems except windows. Assuming we *do* want
to move to UTF-8, we have just one problem, compatibility. We cannot do
both, use UTF-8 and don't break or change any existing code. It's just
impossible. See (3) in my previous mail.

(4) I've already proposed another solution on another thread. We can add a
compile-time flag that, if set, makes all the narrow char and std::string
interfaces assume UTF-8 encoding. This flag can be off by default as long as
we feel comfortable. When more people find this useful we can make it the
default and deprecate the non-UTF-8 configuration. This is *the only way* to
kill two birds with one stone. Any other solution either silently breaks
existing code, or requires boilerplate code to be added even in code that
uses utf-8 std::strings.

I think this is the simplest, least painful way to switch to UTF-8. In fact,
if the experiment fails, we can just deprecate the feature, remove it later
and pretend it'd never happened.

Unfortunately, the community mostly ignored this proposal. The only reply
from a library author showed complete unwillingness to soil his holy code
with such unimportant things like portable Unicode support on windows. He
sent me to request UTF-8 support from microsoft itself.

-- 
Yakov