Re: [boost] [nowide] Library Updates and Boost's broken UTF-8 codecvt facet

Subject: Re: [boost] [nowide] Library Updates and Boost's broken UTF-8 codecvt facet
From: Artyom Beilis (artyomtnk_at_[hidden])
Date: 2015-10-08 11:40:36

Note it isn't really dependency just a shortcut - header only - if you don't include
one you don't get one.

Nowide is about application of utf8 everywhere policies (see utf8everywhere.org)

Note you can't do what I propose with std::experimental::path because there is no place
to imbue encoding.

[Issues with std::experimental::path]

std::experimental::path proposal has problem

std::experimantal::path("stuff.txt") on Windows
would use native narrow encoding - i.e. local code page
and you can't provide UTF-8 name on windows 
see: 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4100.pdf

i.e. you need to use fs::u8path to create it from UTF-8...

Page 16 - note 4 and 5 - which is contrary to what nowide porposes and

But using u8path is problematic on POSIX systems.

For POSIX based operating systems with the native narrow encoding not set to 
UTF-8, a conversion to UTF-32 occurs, followed by a conversion to the current 
native narrow encoding. Some Unicode characters may have no native character set 
representation

You can change "native" encoding on the fly per process on on runtime
It isn't something global, even if current locale LC_ALL=C the utf8 filenames
can still be valid u8path.

Actually on Linux it does not even have to be in specific encoding a file named
"\xFF\xFF.txt" is valid file name but not valid encoding.

I think this std needs good and deep changes.

[/Issues with std::experimental::path]

 Artyom Beilis

----- Original Message -----
> From: Ion Gaztañaga <igaztanaga_at_[hidden]>
> To: boost_at_[hidden]
> Cc: 
> Sent: Thursday, October 8, 2015 5:36 PM
> Subject: Re: [boost] [nowide] Library Updates and Boost's broken UTF-8	codecvt facet
> 
> On 07/10/2015 16:49, Artyom Beilis wrote:
> 
>>  Some updated regarding boost.nowide 1. Library moved to github and
>>  its format is converted to modular boost layout:
>>  https://github.com/artyom-beilis/nowide2. Fixed unsupported
>>  std::ios::ate flag by boost::nowide::fstream3. Added some C++11
>>  interfaces to boost::nowide::fstream4. Added integration
>>  functionality with boost::filesystem:
>> 
> https://github.com/artyom-beilis/nowide/blob/master/include/boost/nowide/integration/filesystem.hpp
> 
> Please don't make nowide dependent on Boost filesystem. It adds unneeded 
> dependencies and disallows other "path" types (like 
> std::experimental::path). Couldn't you template it on a "path" 
> class? 
> This will make the utility more general and avoids any unneeded dependency.
> 
> Best,
> 
> Ion
> 
> _______________________________________________
> Unsubscribe & other changes: 
> http://lists.boost.org/mailman/listinfo.cgi/boost
>