[boost] Re: [filesystem] i18n branch committed to CVS

From: Beman Dawes (bdawes_at_[hidden])
Date: 2005-06-08 21:48:39

"Vladimir Prus" <ghost_at_[hidden]> wrote in message 
news:200506031613.04544.ghost_at_cs.msu.su...
> On Thursday 19 May 2005 01:26, Beman Dawes wrote:
>
>>... Essentially, you are saying that there's
>>  >an operating system that performs some char->wchar and wchar->char
>>  >convertions in path operations, but does not provide any API to do the
>>  >same
>>  >convertion on plain char* and whar_t* pointers. I find this somewhat 
>> hard
>>  >to believe.
>>
>> Windows, for one.
>
> Could you be more specific? Which transformation done by the filesystem 
> can't
> be approximated with the call to MultiByteToWideChar or 
> WideCharToMultiByte?

It is the "approximated" that is the concern. An approximately correct path 
isn't good enough. Users expect exact rather than approximate behavior.
>
>> Although that is really beside the point. The worry is
>> the need for conversions when a path changes from wide to narrow, or visa
>> versa.
>
> Why is it a worry?

Some conversions are lossy. That is, they are not full reversible. So on 
unnecessary round trip can lose data.

>...
>>  >Then, the memory overhead is a single bool flag, telling if a path was
>>  >created from char or whar_t.
>>
>> The memory overhead I was worried about wasn't user space for the bool, 
>> but
>> the need to link in both narrow and wide versions of functions,
>> particularly on low memory embedded systems.
>
> Do you have the specifics? What OS/hardware do you have in mind? IIRC, 
> Windows
> converts all user-provided paths into internal representation anyway. And
> isn't Java, that uses single-string type, works on such low-memory devices 
> as
> mobile phones?

Phones that support Java are hardly "low-memory" devices. Regardless, as the 
cost of memory declines, what constitutes undue memory use changes. So you 
are correct to point out that this is a minor and declining concern.

>> A lot of people say they don't like the std::string design, but it is the
>> standard for C++. Perhaps someday another string design will become
>> popular, but that isn't even on the horizon AFAIKS.
>
> And if boost::path is accepted into standard as templated class, then
> any new string class will have even fewer chances. "Look, the path class 
> is
> also templates", everybody will say.

There is a proposal outstanding to add std::string overloads to many of the 
existing library classes. It isn't just the filesystem library than is 
further entrenching std::basic_string in the standard.

If you want to argue for a runtime polymorphic string class that can change 
it's internal representation as needed, that's fine. But the way to prove 
the point is to develop and popularize such a class.  Then a library like 
Boost.Filesystem would have a base to build on. But it doesn't seem to me 
that Boost.Filesystem is the place to do experiments with dynamically 
self-configuring strings.

>
>>  >>  >Also I note that there's no conversion from basic_path<char> to
>>  >>  >basic_path<wchar_t> or vice versa, as far as I can say. To recall 
>> my
>>  >>  >argument
>>  >>  >for conversion: say I have a library which exposes paths in the
>>  >>
>>  >> interface,
>>  >>
>>  >>  >should I use path or wpath in it? If I use path, then due to 
>> missing
>>  >>  >conversion, the library is unusable with other code that uses 
>> wpath.
>>
>> So
>>
>>  >>  >I need to use wpath.
>>
>> Yes. It is the same situation as with std::string vs std::wstring. If you
>> think your app may sometimes have to deal correctly with wide strings (or
>> paths) you should use std::wstring (and wpath).
>
> I keep on making the same argument over and over, but you don't hear it. 
> If
> I'm writing a library, I have no idea what kind of string the applications
> will pass to the library. And BTW, what if application's requirements 
> change
> over time?

I do very much hear that argument, and find it a very strong argument 
indeed. But I don't see that as a problem to be solved at the level of 
Boost.Filesystem.  Rather, a replacement for std::basic_string that offers 
runtime polymorthic self-configuration and interoperability. That is the 
place to start development IMO. Not in Boost.Filesystem. Boost.Filesystem is 
an innocent bystander that simply uses the std::basic_string compile-time 
polymorphism because that is the current standard.

>
>>  >Even if I provide both types in the interface, if there's no standard
>>  >path<->wpath conversion, I'll have to either:
>>  >
>>  >- write such convertion myself
>>  >- duplicate all code of the library -- for path and for wpath
>>
>> Partially in answer this very valid concern, I've exposed the 
>> wpath_traits
>> conversion interface. I'm not sure that is a complete solution, but at
>> least you wouldn't have to write the conversion code yourself.
>
> Well, I don't understand how to use it. Can you stetck the code code to
> convert path to wpath and vice versa? The wpath_traits code seem to deal
> with strings only.

There is already an example of a user defined path based on strings of longs 
(which was a real-world example mentioned by someone in the LWG).

I'll do another example, where the string type is std::wstring, but the 
external encoding is user supplied.

>> Please note that I'm not saying a single-path-type design is dumb or
>> anything like that. It is just that it would be too big a leap without a
>> lot of experimentation, trial use, etc.
>
> Then, probably it's too early to standardize boost::path. After it's in, 
> it
> won't be possible to add yet another path type.

People sometimes argue that it is premature to standardize some library 
component because a better one is just around the corner. The shared_ptr 
proposal had to fight that battle, for example. The LWG evaluates each 
proposal on its merits, as perceived at the time the proposal is considered. 
Once in a very great while, a proposal comes along that is good enough to 
cause removal of some component that was already accepted. The STL proposal 
was good enough to justify the remove of a now-forgotten dynamic vector 
called dynarray that was already voted in. If a much better string class 
comes along, well, then a filesystem library will be one of many libraries 
that will have to adapt to accommodate it.

--Beman