Re: [boost] [filesystem] How to remove specific files from a directory?

Subject: Re: [boost] [filesystem] How to remove specific files from a directory?
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2016-09-12 14:44:51

On 12 Sep 2016 at 7:51, eg wrote:

> > ... Windows may take until the next
> > reboot to delete a file, and until then it cannot be opened by
> > anybody.
> 
> This is news to me. Do you have any links to documentation on this?

Short answer: as the Win32 docs for DeleteFile() says:

"The DeleteFile function marks a file for deletion on close. 
Therefore, the file deletion does not occur until the last handle to 
the file is closed [in the system]. Subsequent calls to CreateFile to 
open the file fail with ERROR_ACCESS_DENIED."

https://msdn.microsoft.com/en-us/library/windows/desktop/aa363915%28v=
vs.85%29.aspx. [square brackets stuff added by me]

Longer answer: the Windows NT kernel was originally the next edition 
of the VAX VMS kernel, and was considered by many to have a set of 
superior, if more conservative, design choices to the Unixes of the 
day which ended up becoming POSIX. Many of the NT kernel APIs are 
very similar to those of VMS as a result. One interesting feature of 
the VMS filesystem was that when you deleted a file, the system went 
off and securely scrubbed the contents before doing the deletion, a 
process which could take considerable time. NTFS and Windows NT 
inherited that behaviour, and it was furthermore considered valuable 
for secure by design code to lock out use of a file being deleted 
because it makes inode swapping tricks and other such security holes 
on POSIX systems impossible on VMS and NT systems. NT absolutely 
allows you to explicitly opt into POSIX semantics by renaming a file 
before deletion as AFIO does, this leads to default more secure 
semantics than the POSIX default behaviour. Ultimately of course the 
ship has sailed, and POSIX is now the standard. NT reflects a 
continuing objection to many design failures in POSIX especially 
around the filesystem where POSIX has many deeply flawed design 
decisions.

As a result of the above behaviours, unfortunately the lion's share 
of code out there written for Windows which deals with the filesystem 
is simply wrong. It just happens to work most of the time and people 
are too ignorant and/or don't care that it is racy and will cause 
misoperation for some users. A lot of big famous open source projects 
indeed refuse to fix incorrect code after a bug report because they 
just don't believe it's a problem, mainly through not fully 
understanding what files that can't be unlinked means for correct 
filesystem code design. There is a group of dedicated AFIO repo 
followers who have tried logging bugs about these bad design patterns 
with various projects, and it's been quite amazing how little people 
care that their code will always fail under the right conditions. But 
in the end the file system has always been considered as unchanging 
when programmers write code for it, thus leading to a large number of 
race bugs and security holes caused by unconsidered concurrent third 
party induced changes.

AFIO is intended to help programmers to avoid these sorts of issue 
more easily than is possible with just the Boost and standard C++ 
library facilities where it is currently quite hard to write 
completely bug free filesystem code without resorting to proprietary 
APIs.

Those wanting lots more detail may find my conference presentations 
worth watching:

20150924 CppCon Racing the File System Workshop
https://www.youtube.com/watch?v=uhRWMGBjlO8

20160421 ACCU Distributed Mutual Exclusion using Proposed Boost.AFIO
https://www.youtube.com/watch?v=elegewDwm64

The third in the series is coming next week at CppCon.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/