[boost] Re: file_system prefomace problem (?)

From: Daniel Yerushalmi (daniel_zy_at_[hidden])
Date: 2003-01-07 14:30:51

"Beman Dawes" <bdawes_at_[hidden]> wrote in message
news:4.3.2.7.2.20030107125942.02368870_at_mailhost.esva.net...
> At 11:12 AM 1/5/2003, Daniel Yerushalmi wrote:
>  >Hi All
>  >
>  >I have code that iterate on directory tree (the recursive function)
>  >addFolder.  I had profiled it (MSVC 6.5)
>  >        Func          Func+Child           Hit
>  >        Time   %         Time      %      Count  Function
>  >---------------------------------------------------------
>  >      28.780  43.2       31.818  47.8      567
>  >boost::filesystem::is_directory(class boost::filesystem::path const &)
>  >(operations_posix_windows.obj)
>  >      10.136  15.2       22.666  34.0      151
>  >boost::filesystem::directory_iterator::directory_iterator(class
>  >boost::filesystem::path const &) (operations_posix_windows.obj)
>  >       7.249  10.9        7.249  10.9      592 `anonymous
>  >namespace'::find_next_file(void *,class boost::filesystem::path const
>  >&,struct _WIN32_FIND_DATAA &) (operations_posix_windows.obj)
>  >(operations_posix_windows.obj)
>  >
>  >As you can see most of the time is spent in the is_directory function. I
>  >know that (in Win32 - ::FindNextFileA  at least) the information for
>  >"is_directory" can be fetched in the "::find_next_file". I suggest to
add
>  >m_is_directory attribute to the path class (initial value "unknown") and
>  >have is_directory read from this attribute if possible. (For my
> application
>  >the is_directory time is swamped by other things but for other directory
>  >iterating applications this may be significant)
>
> Before answering the question "How can we cache file attributes between
> calls to filesystem function?" we need to answer the question "Should we
> cache file attributes between filesystem functions?"
>
> The problem is race-conditions. "c:/foo/bar" may have been a directory the
> last time we saw it, but now is a file. I'd want to think about that a bit
> more before doing any caching.
>
> I'd like the filesystem library to mature and see wider use before
worrying
> about optimizations, too.
>
> Thanks for the profiling,
>
> --Beman
I agree with you about the maturing (in fact in my application the iteration
does not take that much time so even speeding it by factor of 9 will not
help so much :( )

When you recursivly iterate on directories you implicitly cache the
is_directory information... (You use it to access the next level for this
path).
I recall that I have seen in boost (in the sandbox?) something called
recursive_directory_iterator...) maybe we should do the cache only for this
case (when more pepole will use this library) .

By the way - the speedup is only relevant to Win32 - the Posix API does not
fetch path attribute when iterating on a directory.

>
>
> _______________________________________________
> Unsubscribe & other changes:
http://lists.boost.org/mailman/listinfo.cgi/boost
>