Re: [boost] Re: boost::filesystem - directory_iterator makesoptimizationhard

From: Peter Dimov (pdimov_at_[hidden])
Date: 2005-06-02 07:44:47

Beman Dawes wrote:

> * That is a lot of additional interface complexity to support an
> optimization that applies to Windows but not POSIX. Some of the other
> schemes (which involved additional overloads to specific operations
> functions) had less visible impact on the interface.
>
> * There have been no timings to indicate the inefficiency of the
> current interface actually impacts production applications.
>
> * AFAICS, there is nothing in the current interface which prevents
> caching of directory entries. Caching would probably aid more use cases 
> than
> proposed changes to interfaces. But both caching and user dirent
> storage introduce serious additional race conditions. Not a good
> thing. In fact a showstopper unless cache management is introduced,
> further complicating the interface.

As a data point, PHP does stat caching:

http://www.php.net/manual/en/function.is-dir.php

because of performance issues. Stat caching does apply to POSIX systems as 
well. Either a transparent caching scheme or is_* overloads taking a 
directory iterator may work.

The race conditions argument isn't very strong. The information returned by 
is_directory is inherently race-prone, caching or no caching. I won't be 
surprised if an OS does its own per-process stat caching, for instance.