Re: [boost] boost::filesystem feedback

From: William E. Kempf (williamkempf_at_[hidden])
Date: 2002-07-19 14:54:42

----- Original Message -----
From: "Giovanni Bajo" <giovannibajo_at_[hidden]>
To: <boost_at_[hidden]>
Sent: Friday, July 19, 2002 12:15 PM
Subject: [boost] boost::filesystem feedback

(Note:  I've not looked at the library in depth, so some of my comments may
be off the mark according to specific design decisions there.)

> Hello,
>
> after having played a bit with the file system library, I have several
> feedbacks to share.
>
> What's the canonical form for "c:\windows"? The only way I found to setup
a
> path like this is fs::path("c:\\windows", fs::path::system_specific). Now,
I
> understand there are platforms where drives are mapped to directories. But
> it sounds strange that there is simply no way to specify a drive when
using
> the library under Windows. Actually, if you print a path constructed as
> above, it displays "c:/windows", but you can't construct the path with
this
> syntax. So, my point basically is that I want a way to select a driver
under
> Windows in canonical form. Suggestions are to accept either "c:/windows"
or
> "/c/windows" (changing the library so that the first directory in an
> absolute path is always the drive name - under Windows).

This prevents me from specifying simply "/windows".  Cygwin uses a different
syntax for this reason, "/c//windows".  I'm not sure if this syntax is worth
it or not, since Windows users won't be familiar with it (and at least
"c:/windows" is close to what they are used to).

BTW, you can map a drive to a directory in Windows (at least Win2K and
above) as well.  You can even have symbolic links, though the semantics are
different from POSIX.  This is all done through "junctionpoints".

I'm actually kind of still in favor of having a more POSIX-like structure
even under the Windows platform.  If you do this then maybe the better
syntax for the above would be "/drive/c/windows" or even your "/c/windows",
but all paths would be root relative instead of drive relative so there'd be
no "/windows".  (Well, actually, for Win32 I'd define "/windows" to point at
the windows directory for the system, which may well be d:\winnt instead of
c:\windows.)  The reason I'm thinking the POSIX form might be more
appropriate is that with the advent of 100G+ drives even MS is moving
towards a mount point type of system instead of drive mappings.  Further,
such syntax would allow us access to "special folders" that exist on Win32
platforms, such as My Documents and Desktop.  However, I don't know how
sensible such mappings would be for other non-Windows non-POSIX systems.

> I pretty much don't like "directory_iterator" in its current form. When I
> read the word iterator, I expect to have a container associated to it with
> its begin() and end() members. directory_iterator behaves as a STL
iterator
> for some things (full InputIterator semantic), but does not for others (it
> incapsulate its own 'container' to the user).
> My proposal is creating a new class called 'directory_list' which would
act
> as a container (I think it should match what it's now called
> directory_iterator_impl), and then defining directory_list::iterator used
to
> simply navigate through it as normal.
>
> Basically, 'directory_list' would be very very similar to a list<fs::path>
> (it might be its underlying implementation), with the list holding the
names
> of the files within the directory (and with lazy evaluation to be sure it
> gets filled only when the informations are requested by the user). It
would
> have a constructor getting a fs::path (throwing an exception if the path
> does not represent an existing directory). It would also implement most
> (all?) the member functions found in a list<fs::path> (like begin(),
end(),
> size(), erase(), etc). Giving it a full container semantic allows to write
> code like:

Theoretically, there's no need to store the data once accessed, and in fact
doing so could cause some issues.  Of course, lazy evaluation causes issues
as well, since the data the container wraps isn't owned by the container and
can be changed externally, even by other processes running on the system.
But for reasons others have pointed out, and because the current C APIs
leave us in the same quandry, I might be in favor of a fully lazy evaluated
directory_list (why "list"?, why not just "directory"?).  Of course, this
means the container doesn't fit full STL container requirements, and for
that reason I think maybe you should stick with just directory_iterators,
the same as with iostream_iterators.

> bool MatchCPP(fs::path fileName)
> {
>     return fileName.leaf().find(".cpp") != string::npos;
> }
>
> fs::directory_list dir = fs::path("/usr/whatever");
> fs::directory_list dir_only_cpp;
>
> std::copy_if(dir.begin(), dir.end(), back_inserter(dir_only_cpp),
MatchCPP);
>
> A further step would be to automatically implements filters like this and
> more features with overloaded constructors:

It's nicer if the directory_iterator itself could do the filtering.

> fs::directory_list dir(fs::path("/usr/whatever"), "*.cpp");  // construct
a
> directory_list only with .cpp files
> fs::directory_list dir("/usr/whatever", "*.cpp");  // as above, just
handier
> fs::directory_list dir("/usr/whatever", "*.cpp", true);  // automatically
> recurse subdirs
> fs::directory_list dir("/usr/whatever", regex(".+\\.(cpp|C)$"), true);  //
> matching with boost::regex instead of wildcards
>
> Filtering and recursing subdirectories are very important because, in my
> opinion, they are the two most common operations done while browsing a
> directory.
>
> One may ask why you need a directory_list, when you could simply have a
> make_directory_list() (with the overloads shown above for the constructor)
> and returning a list<fs::path>. The answer is that I really like lazy
> evaluation for directory browsing (especially when dealing with subdir
> recursion). A simple make_directory_list() would have to browse all the
> files to create a complete list<fs::path>, while a directory_list can be
> "constructed" on the fly, when the elements are requested by the user.

With the relevant race conditions.  I'm inclined to think such a container
is not such a good idea.

Bill Kempf