Re: [boost] [review] Review of Nowide (Unicode) starts today

Subject: Re: [boost] [review] Review of Nowide (Unicode) starts today
From: Artyom Beilis (artyom.beilis_at_[hidden])
Date: 2017-06-13 15:42:43

On Tue, Jun 13, 2017 at 6:08 PM, Peter Dimov via Boost
<boost_at_[hidden]> wrote:
> Artyom Beilis wrote:
>
>> Note, under POSIX user takes strings as is and can't trust the source.
>> Under Windows it need to convert them using nowide which can give him
>> false assumption that it receives valid UTF-8.
>
>
> What use case are we discussing here?
>
> Is it the one where the user gets a path from an external source and passes
> it to Nowide? In which case Nowide does no validation on POSIX and strict
> UTF-8 validation on Windows? Giving the Windows user the false assumption
> that Nowide validates, which turns out to not happen in a later POSIX port.
>
> Or is it the one where the user receives a path from Nowide? Which currently
> doesn't occur anywhere, because the library does not return paths, it only
> takes them as arguments. (It would occur if you add `readdir`.)
>

For that purpose you have Boost.Filesystem that internally keeps wide strings
on Windows and narrow on POSIX platforms.

Nowide does not try to replace Boost.Filestem. However once you try to get
an std::string from Boost.Filesystem (using nowide integration) you will
get either error or valid UTF-8 string.

>> Once again I have no problem providing wtf8 to wide and other way around
>> functions when user EXPLICITLY says it.
>
>
> How would portable user code look like in this scenario?
>

Nowide does not try to provide solution to all possible portable API.
But it provides a toolset to make such portable solutions easier.

However if you convert some wide string from some Windows API you
will expect to get valid UTF-8 and not WTF-8.

Artyom