Re: [boost] BUG: filesystem remove_all [PATCH]

From: Walter Landry (wlandry_at_[hidden])
Date: 2005-01-19 22:05:58

Beman Dawes <bdawes_at_[hidden]> wrote:
> At 10:58 PM 1/18/2005, Walter Landry wrote:
>  >Greetings,
>  >
>  >I have discovered a bug in the implementation of remove_all.
>  >remove_all works by reading through a directory and deleting entries
>  >along the way.  If it comes across a subdirectory, it immediately
>  >recurses into that subdirectory.  However, boost::filesystem uses
>  >readdir, which on some platforms (including OS X, apparently) uses a
>  >static data area that can be overwritten on each call to readdir.
>  >This messes up the original directory iteration, ending up with some
>  >entries not being deleted.
>  >
>  >A solution is to not recurse the directory immediately, but to make a
>  >list of them.  Once you have gone through all of the entries in a
>  >directory, then you recurse into the subdirectories.  A patch is
>  >below, although it could probably be improved.
> 
> Walter,
> 
> Thanks for the report. As Peter pointed out, that isn't supposed to happen 
> on a POSIX compliant system. Is it possible the bug is actually in the 
> remove_all code or your code?

I don't think so.  The same code works fine on NetBSD, Cygwin, and Linux
(various flavors).  It only breaks on OS X.

The particular case causing problems is a directory filled with about
560 subdirectories.  It may be that this is a sufficiently rare case
that no one has run into it.

Also, as a point of interest, there is an implementation of remove_all
in C at

  http://www.cs.utk.edu/~plank/plank/classes/cs360/360/oldtests/t1/2000/a2.html

It takes the same approach I gave, but also notes that you should
really call chmod on the directories before attempting to delete
anything in them.

> I've had a "to do" item for a month or so to change readdir to readdir_r on 
> systems which support it. My guess is that almost all modern POSIX 
> operating systems support readdir_r, but there is a configuration macro so 
> the code can adapt to what is available. I'm not sure my guesses about 
> POSIX are worth anything, however, since I would have guessed every modern 
> POSIX OS implements readdir correctly:-( Anyhow, readdir_r should solve 
> this problem so I'll accelerate that fix.

That would be great.

Cheers,
Walter Landry
wlandry_at_[hidden]