Re: [boost] [review] Review of Nowide (Unicode) starts today

Subject: Re: [boost] [review] Review of Nowide (Unicode) starts today
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2017-06-16 14:15:35

>> And you can safely concatenate two different strings and valid file
>> will be created. Even if dir is in ISO 8859-1 and file in UTF-8. The
>> file will be valid even if not representable in any encoding.
> 
> "POSIX" is not correct here. File names under Mac OS X are UTF-8. Always
> UTF-8, not merely by convention, and no mater what locale is set.

You were right but are now wrong I am afraid Peter:
https://mjtsai.com/blog/2017/03/24/apfss-bag-of-bytes-filenames/

Apple's shiny new APFS treats paths as a dumb sequence of bytes, and
they are retrofitting HFS+ to behave the same to match.

> (They're stored in UTF-16 in the filesystem, just like in NTFS. Unlike
> NTFS, it's _valid_ UTF-16.)

You should assume that invalid UTF will present on all filing systems
because almost all of them treat paths as dumb byte strings. It's the
only portable assumption. It's why AFIO v2 opts out of case
insensitivity on Windows which no doubt will surprise quite a few end
users, but it does present many security attacks at the AFIO layer (by
pushing the problem further up the stack).

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/