Re: [filesystem] becoming worse than the standard?

From: Gavin Lambert (boost_at_[hidden])
Date: 2022-08-16 03:15:32

On 16/08/2022 11:53, Vinnie Falco wrote:
> My experiences with std::filesystem and boost::filesystem have been
> nothing but negative. I think that the decision to make the character
> type different on Windows was a mistake. The need for locales and
> imbuements and global state and... really, it is just giving me a big
> headache.

Using wchar_t on Windows is actually the least painful option.  (And you 
don't have to worry about locales and imbuements etc if you never try to 
convert to not-wchar_t.)

For correct behaviour, you *must* only use the W variants of the native 
API methods, or wchar_t methods of standard library functions.

Inevitably, everything in the standard library that accepts 'char' 
params assumes that these are encoded in the ANSI code page, not UTF-8. 
This can't be "fixed" or it breaks all the legacy apps.

In practice, this means that unless you can absolutely guarantee that 
your paths only contain pure ASCII (and the instant you accept a path or 
filename from the user, you lose), it is *never* safe to use any of the 
non-wide library methods.

You *can* (and many do) store paths in other libraries and in the 
application in 'char'-encoded-as-UTF-8, but then you have to remember 
every single time you hit the standard library or direct WinAPI 
boundaries to convert your strings to wide before passing them across, 
or hilarity will ensue (without even a convenient compiler error).

Storing paths as wchar_t in the first place both avoids the cost of 
converting back and forth and potential corruption (often overlooked, 
unless you regularly test with unicode paths) from accidentally 
forgetting a conversion.

> (where is the signature of fopen that accepts a filesystem::path?)

Why are you using fopen in C++ in the first place?

Filesystem does provide 'path' overloads for fstreams, which you should 
have been using instead anyway.

> It should be utf-8 only, use Plain Old char (even on Windows), it should
> be completely portable, except that it requires that directories are
> possible and that the filesystem isn't weird (I don't really care
> about compatibility with grandpa's EPROMs that can hold 9-bit flat
> files).

In theory, the standard library (and other wrapper libraries around the 
WinAPI, including Filesystem) could start doing more sane things by 
using the C++20 'char8_t'/'u8string' types to disambiguate between UTF-8 
encoded paths and legacy idkwtf-'char'-encoded paths.  But this will 
take a very long time to percolate through the ecosystem, especially as 
there are a bunch of people who hate the very idea of it.  And it 
doesn't solve the conversion performance angle.

(Hopefully, Windows will eventually provide char8_t entrypoints and 
APIs, which will make it easier to interoperate with not-Windows.)

Although as Emil has already pointed out, it's valid in not-Windows to 
have arbitrary not-UTF-8 byte sequences in paths, so you can get into 
trouble in that direction as well.

That's another reason for using wchar_t in Windows and char in 
not-Windows: no conversions happen at all (at least where values are 
accepted natively from the OS), which has maximal compatibility for 
otherwise-invalid byte sequences that nevertheless exist.