Re: [boost] Making Boost.Filesystem work with GENERAL filenames with g++ in Windows (a solution)

Subject: Re: [boost] Making Boost.Filesystem work with GENERAL filenames with g++ in Windows (a solution)
From: Yakov Galka (ybungalobill_at_[hidden])
Date: 2011-10-26 06:24:24

On Wed, Oct 26, 2011 at 11:56, Alf P. Steinbach <
alf.p.steinbach+usenet_at_[hidden]> wrote:

> On 26.10.2011 11:47, Artyom Beilis wrote:
>
>>
>>
>>
>> ----- Original Message -----
>>
>>> From: Alf P. Steinbach<alf.p.steinbach+**usenet_at_[hidden]<alf.p.steinbach%2Busenet_at_[hidden]>
>>> >
>>>
>>> First, apologies if I'm posting to the wrong group/list; if so then
>>> please
>>> redirect me.
>>>
>>> IMHO access to files is a crucial part of Boost.Filesystem. However, with
>>> Boost
>>> 1.47, and using g++ 4.4.1 in Windows 7, boost::filesystem::ifstream etc.
>>> fail to
>>> open or create files with non-ANSI characters. It works fine with Visual
>>> C++; it
>>> FAILS with g++ 4.4.1, which is the one bundled with the Code::Blocks IDE.
>>>
>>> The failure probably has nothing to do with the g++ version: it's due to
>>> g++
>>> not offering the Visual C++ wchar_t oriented extensions to the standard
>>> iostreams (Boost.Filesystem uses these Visual C++ extensions).
>>>
>>> I stumbled onto this while I was writing about using Unicode in C++
>>> programming
>>> in Windows.
>>>
>>>
>> Why just not to implement boost::filesystem::fstream over _wfopen and
>> custom
>> streambuf implementation?
>>
>> It is relatively simple.
>>
>> I have implemented such thing under booster namespace as part of cppcms
>> project
>>
>> http://cppcms.svn.sourceforge.**net/viewvc/cppcms/framework/**
>> trunk/booster/booster/nowide/<http://cppcms.svn.sourceforge.net/viewvc/cppcms/framework/trunk/booster/booster/nowide/>
>> http://cppcms.svn.sourceforge.**net/viewvc/cppcms/framework/**
>> trunk/booster/lib/nowide/src/<http://cppcms.svn.sourceforge.net/viewvc/cppcms/framework/trunk/booster/lib/nowide/src/>
>>
>> http://cppcms.svn.sourceforge.**net/viewvc/cppcms/framework/**
>> trunk/booster/booster/nowide/**fstream.h?revision=1967&view=**markup<http://cppcms.svn.sourceforge.net/viewvc/cppcms/framework/trunk/booster/booster/nowide/fstream.h?revision=1967&view=markup>
>>
>>
>>  I wrote up a technical solution in section 5, starting on page 16, of
>>> that
>>> work-in-progress document, available on Google Docs at:
>>>
>>>
>>> https://docs.google.com/**viewer?a=v&pid=explorer&**chrome=true&srcid=**
>>> 0B2oiI2reHOh4ZjdkNmUyNDctNzI1Y**i00NmJjLThiMzgtYmI3OGE0ZmE5MDg**2&hl=en<https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B2oiI2reHOh4ZjdkNmUyNDctNzI1Yi00NmJjLThiMzgtYmI3OGE0ZmE5MDg2&hl=en>
>>>
>>> Essentially, the fix I ended up with, full source code given in the above
>>> doc,
>>> uses Windows short file names if (1) there is no wide character support
>>> and if
>>> furthermore (2) the filename can't be perfectly translated to ANSI. The
>>> C++
>>> implementation's support for wide chars is automatically detected using
>>> C++98-compatible code.
>>>
>>> I do not know what to do with this.
>>>
>>>
>> That is what very good about stream buffers... You can implement anything
>> you need.
>>
>
> If the authors of Boost.Filesystem are happy to reimplement stream buffers
> from scratch, all the way, then that would indeed be good news.
>
> But do you *really* think that is realistic?
>
>
I think when someone gives you a working implementation, it's rather dumb to
ask if it's realistic or not.

>
>  Using short file names is no go for two reasons:
>>
>> 1. It works only when file exists (can't create new file)
>> 2. It is quite deprecated
>>
>
> Both points are incorrect.
>
> Regarding (1), did you notice that I wrote "full source code provided"...?
>

Yes, you provide a hackish workaround, that is vulnerable to a
race-condition btw. Even a more serious problem is that it doesn't work with
paths longer than MAX_PATH.

> I think when something demonstrably works, it's rather dumb to assert that
> it can't work, sorry.
>
>
>
>  But considering that Boost.Filesystem is slated for later inclusion in the
>>> C++
>>> standard library (or at least into TR2), I think it would be nice if it
>>> is able
>>> to give access to all accessible files in Windows, also with g++,
>>>
>>
>> This is a problem that can be fixed easily in Boost.Filesystem.
>>
>
> Yes, I think so, for Windows.
>
> However, more important, the existence of an actual fix (which I linked to)
> shows that the present interface does not prevent a fix for Windows.
>
> For some other OS-es it may not necessarily be easy to fix, however, and
> thus, for inclusion of Boost.Filesystem in the standard library I think that
> should be thoroughly investigated.

Personally I think that boost::filesystem::paths are a sad joke, it's a pity
they're heading to the standard. Although the OS-part is definitely good,
the way path class is design isn't suitable for paths outside the unix
world. Even if you fix the Unicode problems, you still cannot use long paths
on windows (longer than MAX_PATH), although they are supported by the OS.
Moreover, judging by the last fixes to the library, it looks like Beman
wants to shift the burden of this on the user of the library, instead of
implementing something that works transparently.

>  so that we
>>> don't end up with a file handling part of the standard library that
>>> can't handle files in general;
>>>
>>
>> This is a "bug" in Windows operating system... but this is other story.
>>
>
> Sorry, that's bullshit: there is no such bug in the OS.

I think what Artyom means is that the "bug" in windows is the whole UTF-16
crap.

-- 
Yakov