Re: [boost] New design proposal for boost::filesystem

From: Reece Dunn (msclrhd_at_[hidden])
Date: 2004-08-22 03:14:34

Carlo Wood wrote:
>On Sat, Aug 21, 2004 at 11:36:31AM +0100, John Maddock wrote:
> > > I propose the following design.  The aim of boost::filesystem should 
>be to
> > > support the following coding idiom:
> > >
> > > * The programmer should take care to only handle two types of paths
> > >   in his application:
> > >
> > >   1) Complete paths
> > >   2) Relative paths
> >
> > That is true now.
>
>No, the current implementation/design doesn't restrict anything at all
>in this regard.  You can just use fs::path to store paths - and that is
>it.  A path is not aware of a distinct difference between complete
>paths and relative paths (except that they are complete or relative
>of course) - definitely not in the same way as a design would that
>uses two different classes for the two.

A complete path is one that is fully qualified, thus:

   fs::path absolute = fs::path( "d:/devel/libraries/boost/1.31.0/libs" );

whereas a relative path is one that is "relative" to another. With a 
relative path, you must have an absolute path as the basis, e.g.:

   fs::path relative = fs::path( "../boost/mpl" );

and can turn the relative path into an absolute one by doing:

   fs::path qualified = absolute / relative;

so where is the problem?

>The most problematic difference is that it is possible to even store
>a third possibility (one that is neither complete nor relative).

??? Do you mean one that isn't relative because it is not based from an 
absoute path, or that the path does not exist? If so, you can do that in any 
file system, e.g.:

   fs::path foobar = fs::path( "../booster/foobar.foo" );
   fs::exists( absolute / foobar );

A path will either (in absolute form) refer to a file, a directory or be 
invalid.

> > > * The programmer will have to specifically tell the libary when a
> > constructed
> > >   path is 'native' and when not.  A native path is accepted according 
>to
> > native
> > >   rules and never gives any problem (exceptions) further on.
> > >   A non-native path is checked according to the existing rules, which
> > basically
> > >   means that the programmer can set a default check routine that will 
>in
> > effect
> > >   determine how portable the application will be.
> >
> > That is also true now, isn't it?
>
>No, because the 'native' that you can specify with the current design
>is only a check on the characters used in the directory components, and
>only related to the representation of a path - not *marking* the path
>as different.  The 'native' I am talking about is enforced for complete
>paths and reflexs also the root part of a directory ("C:\", "c:/cywgin/",
>"/" etc).  The essential difference is that a path that is once marked
>as 'native' must stay native.  It can never be converted to a portable
>path anymore.  Only relative paths that are portable from the start
>can stay portable after certain operations (like appending a directory, or
>cutting off a directory at the end).

The native you are referring to is an "absolute" path, correct? 
Boost.Filesystem stores the paths in a generic form. Thus, if your 
filesystem uses ':' as a directory separator, the library will map this 
correctly from it's internal state. If you enforce that all "native" Windows 
paths must have a directory, you are restricting the use of the library, 
e.g. "\\mycomp\devel\libraries" is not possible! Likewise, it would not be 
possible to store URLs.

NOTE: I am not sure if Boost.Filesystem currently supports URLs, but it 
should be possible to extend it to add that support. Also, URLs are not 
attached to a particular filesystem and are thus portable. It should be 
possible to handle both URLs and native paths, but limiting "native" paths 
will only make this harder.

> > > I propose two design changes:
> > >
> > > 1) 'native' is now not only a representation, but an *internal state* 
>of
> > fs::path.
> > >    (this has no effect on the representation as returned by
> > fs::path::string()).

This would make things too restrictive. I think you are confusing "native" 
with "absolute" paths, but how do you validate an absolute path? Is there a 
Windows and a POSIX function to validate an absolute path (e.g. 
IsPathAbsoute)? It would then be possible to have a fs::is_absolute( ... ) 
function.

> > > 2) All 'complete' paths are automatically marked as 'native'.

Likewise, how do you validate a complete path? What about a URL? If you have 
the URL "http://www.boost.org/people" and are using a system whereby the 
native directory separator is ':', the URL will be mapped to 
"http:::www.boost.org:people", corrupting the URL.

> > If I understand you correctly, you are suggesting that error checking is
> > turned off for native paths, I would support that, but other than that I
> > don't see how it differs.
>
>Heh - the current design does NOT have an internal state that marks
>a path as native or not.  Hence, it cannot do sensible error checks
>that need that knowledge.  Isn't that difference enough?

Native - as Boost.FileSystem uses it - is used to mark a path as using the 
native OS syntax for specifying paths. This is so you can do:

   fs::system_complete( fs::path( argv[1], fs::native ) );

and use the library on Windows, *nix, OpenVMS, MacOS, etc. without having to 
worry about the differences between how they specify their paths and thus 
what regular expressions need to be used on the string. The example from the 
documentation (using OpenVMS) is:

   DISK$SCRATCH:[GEORGE.PROJECT1.DAT]BIG_DATA_FILE.NTP;5

> > > Examples, the following code is legal:
> > >
> > > fs::path p1("C:\\foo\\a.exe", native); // As one might do on windows.
> > > fs::path p2("/usr/src/a.out, native); // As one might do on linux.
> > > std::cout << p1.native_file_string(); // ok, p1 is native.
> > >
> > > fs::path p3("foo/bar"); // Relative path, always succeeds.
> > > std::cout << p.string(); // ok
> > > std::cout << p.native_directory_string(); // Useless, but ok.
> >
> > Not useless at all.  And works now.
>   ^^^^^^^^^^^^^^^^
>     \__ this.
>
>That is not backupped.

Did you mean p to be the relative path p3? If so, then 
p3.native_directory_string() is perfectly valid and *not* useless. If the OS 
uses ':' as it's directory separator, then this will return "foo:bar". It is 
useful for all sorts of purposes, like debuging.

> > > fs::path p4(complete(p3)); // p4 is now "native", because now it is
> > complete.

Why does native ==> complete? You can have a native relative and/or complete 
path.

> > > std::cout << p4.string() // Not allowed because p4 is native 
>(complete).

Huh? IIUC path.string() returns the internal format of the path (e.g. 
"foo/bar") whereas path.native_directory_string() returns the path as 
represented by the native OS (e.g. "foo:bar").

>Right now you can call all the operations that actually pass a path to some
>system call - and UNLESS that path is complete - it will not be 
>non-portable.

Not true. "foo/bar" is relative *and* non-portable for OS's that use ':' as 
a directory separator.

>THAT means that boost::filesystem has a structural design error.  You are
>now saying that I can complete all paths just prior to calling any 
>operation
>that passes paths to system calls.  Would you mind even listing those for 
>me?
>Why not change boost::filesystem so that it takes care of completion
>automatically when needed?  That way you cannot make errors.

If you have an absolute path boost_dir and a relative path boost_build2, you 
can do:

   ::SetWorkingDirectory(( boost_dir / boost_build2 
).native_directory_string().c_str());

>As an example - I developed my application first on GNU/Linux with as only
>demand that it had to be portable to cygwin.  Therefore, I used UNIX paths.
>Testing if /usr/src/edragon/edragon/build/src/gui/gui.la existed on linux
>worked fine.  It did NOT work on cygwin.  Because I was stupid?  No, I 
>think
>that was because boost::filesystem is not activily supporting portable 
>programming.

Did you check that you have the file where you said on the cygwin system. 
Have you checked that "/usr/bin/ls.exe" exists? Also, have you checked that 
BOOST_POSIX is defined as per the note on using the cygwin platform in the 
docs?

Regards,
Reece

_________________________________________________________________
Express yourself with cool new emoticons http://www.msn.co.uk/specials/myemo