Re: [boost] [review] Review of Nowide (Unicode)

Subject: Re: [boost] [review] Review of Nowide (Unicode)
From: Artyom Beilis (artyom.beilis_at_[hidden])
Date: 2017-06-20 20:13:14

>
> I welcome the UTF-8 approach on Window. However, the fact that Nowide does
> not handle invalid UTF-16 transparently is an issue that has to be
> addressed.
>
> [snip]
>
>> - What is your evaluation of the potential usefulness of the library?
>>
>
> I already use this approach through a different implementation. Even though
> I wouldn't bring Boost just for that single library, for projects that
> already use Boost it might be a valuable addition.

Actually there is standalone version out of boost scope (see docs).

>
>> - Do you think the library should be accepted as a Boost library? Be
>> sure to say this explicitly so that your other comments don't obscure
>> your overall opinion.
>>
>
> Conditional acceptance, subject to transparent invalid UTF-16 resolution.
>
>
> --
> Yakov Galka
> http://stannum.co.il/
>

After long discussions on this list the following updated policy will
be applied to nowide:

1. Conversion will always lead to **valid UTF-8/UTF-16** regardless
validity of the source
    unlike the current status that returns error/creates error status.

2. Instead of failing the conversion and returning an error the
invalid characters will be replaced
with U-FFFD - Replacement Character - similar to behavior of WinAPI.

So you will not get Invalid UTF-16 <- Quazy UTF-8 -> Invalid UTF-16 path
but you will be able to complete the path as: Invalid UTF-16 <- Valid
UTF-8 with substitutions -> Valid UTF-16

I hope it complies with your needs

Artyom