[Boost-users] [filesystem] problem: is_regular_file and deduplified files (reparse+sparse)

Subject: [Boost-users] [filesystem] problem: is_regular_file and deduplified files (reparse+sparse)
From: Paul Harris (harris.pc_at_[hidden])
Date: 2015-07-22 20:56:21

Hi guys,

I'm seeing a lot of failures all of a sudden, its a serious problem.  I
imagine some of you may start seeing the same failures too very soon.

I'm seeing it with boost 1.58.0.
There is a related bug already entered here:
https://svn.boost.org/trac/boost/ticket/11057
I can't seem to log on and update this ticket.

But the problem extends further than that ticket.

Windows Server 2003 support has been dropped by Microsoft, and a lot of IT
depts around the world are switching over to the latest Windows Server.

With this server comes the new "dedup" feature, that can automatically
deduplify files.  This happens on a schedule, eg 2am saturday.  So suddenly
we are getting messages of failures of software from all over the place,
due to fs::is_regular_file()

Deduped files have the REPARSE and SPARSE flag set.
On the command line, you can run
FSUTIL REPARSEPOINT QUERY

and the "Reparse Tag Value" is 0x80000013

Which is a relatively new flag known as IO_REPARSE_TAG_DEDUP
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365740%28v=vs.85%29.aspx

These files act as normal files, you can fopen and fread them, so I assume
they should be treated almost like symlink by boost... perhaps not quite a
symlink because I assume the "lstat" link properties are identical to the
file's stat properties.

Typically, I iterate over directories and only process files if
fs::is_regular_file(filename) is true.

I wrote some code to check what the properties were on these files, and its
not any of the possible enums detected by file_status::type().

ideas?

Best regards,
Paul