Re: [boost] Making Boost.Filesystem work with GENERAL filenames with g++ in Windows (a solution)

Subject: Re: [boost] Making Boost.Filesystem work with GENERAL filenames with g++ in Windows (a solution)
From: Beman Dawes (bdawes_at_[hidden])
Date: 2011-10-26 16:13:54

On Tue, Oct 25, 2011 at 9:41 AM, Alf P. Steinbach
<alf.p.steinbach+usenet_at_[hidden]> wrote:
> First, apologies if I'm posting to the wrong group/list; if so then please
> redirect me.

This is the best place to post, although opening a ticket is best for
problems that aren't trivial to fix.

> IMHO access to files is a crucial part of Boost.Filesystem. However, with
> Boost 1.47, and using g++ 4.4.1 in Windows 7, boost::filesystem::ifstream
> etc. fail to open or create files with non-ANSI characters. It works fine
> with Visual C++; it FAILS with g++ 4.4.1, which is the one bundled with the
> Code::Blocks IDE.

Yes, although it is actually characters that are not covered by the
current file system codepage rather than non-ANSI characters, IIRC.
Surprisingly, no one has opened a ticket yet.

Until someone does open a ticket and the problem gets fixed, there are
a couple of workarounds:

(1) Use V2. Its fstream.hpp uses an implementation hack that works as
long as 8.3 filenames are enabled. (Some Windows users disable 8.3
filenames as an optimization.)
(2) V3 may work OK with the Microsoft 65001 UTF-8 codepage, although
I've never used it myself and you would have to pass in a UTF-8
encoded narrow character name.

> The failure probably has nothing to do with the g++ version: it's due to g++
> not offering the Visual C++ wchar_t oriented extensions to the standard
> iostreams (Boost.Filesystem uses these Visual C++ extensions).

Right. libstdc++ doesn't provide the wchar_t overloads.

> I stumbled onto this while I was writing about using Unicode in C++
> programming in Windows.
>
> I wrote up a technical solution in section 5, starting on page 16, of that
> work-in-progress document, available on Google Docs at:
>
>
> https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B2oiI2reHOh4ZjdkNmUyNDctNzI1Yi00NmJjLThiMzgtYmI3OGE0ZmE5MDg2&hl=en
>
> Essentially, the fix I ended up with, full source code given in the above
> doc, uses Windows short file names if (1) there is no wide character support
> and if furthermore (2) the filename can't be perfectly translated to ANSI.
> The C++ implementation's support for wide chars is automatically detected
> using C++98-compatible code.
>
> I do not know what to do with this.

If you care enough to open a ticket on the Boost bug tracker, I'll
move the V2 code to V3. But there is a big backlog of tickets, so no
guarantees as to when that will happen.

Another possibility is to try to talk the libstdc++ folks into
supporting the Dinkumware wchar_t extension. They will presumably want
to do that anyhow to support TR2 (or whatever it is going to be
called).

> But considering that Boost.Filesystem is slated for later inclusion in the
> C++ standard library (or at least into TR2), I think it would be nice if it
> is able to give access to all accessible files in Windows, also with g++, so
> that we don't end up with a file handling part of the standard library that
> can't handle files in general; hence this posting and plea for advice  --
>  what more should I do, if anything?

The libstdc++ folks are strong supporters of the standard library and
its TRs, so I suspect they will faithfully implement whatever comes
out of the standardization process.

Thanks,

--Beman