Re: [Boost-users] [filesystem] problem: is_regular_file and deduplified files (reparse+sparse)

Subject: Re: [Boost-users] [filesystem] problem: is_regular_file and deduplified files (reparse+sparse)
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-07-22 21:54:13

On 23 Jul 2015 at 8:56, Paul Harris wrote:

> With this server comes the new "dedup" feature, that can automatically
> deduplify files.  This happens on a schedule, eg 2am saturday.  So suddenly
> we are getting messages of failures of software from all over the place,
> due to fs::is_regular_file()
> 
> Deduped files have the REPARSE and SPARSE flag set.
> On the command line, you can run
> FSUTIL REPARSEPOINT QUERY
> 
> and the "Reparse Tag Value" is 0x80000013
> 
> Which is a relatively new flag known as IO_REPARSE_TAG_DEDUP
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa365740%28v=vs.85%29.aspx
> 
> These files act as normal files, you can fopen and fread them, so I assume
> they should be treated almost like symlink by boost... perhaps not quite a
> symlink because I assume the "lstat" link properties are identical to the
> file's stat properties.
> 
> 
> Typically, I iterate over directories and only process files if
> fs::is_regular_file(filename) is true.
> 
> I wrote some code to check what the properties were on these files, and its
> not any of the possible enums detected by file_status::type().
> 
> ideas?

Proposed Boost.AFIO doesn't support IO_REPARSE_TAG_DEDUP because I 
have no access to any system to test the support upon.

However, if AFIO were to support IO_REPARSE_TAG_DEDUP, it would treat 
it identically to a symlink/junction point.

I'd suggest Boost.Filesystem do the same, and treat pseudo-symlinks 
as symlinks. That probably means adding full symlink support for 
Filesystem on Windows. Here are some links to example implementation 
code:

Reading a symlink target: 
https://github.com/BoostGSoC13/boost.afio/blob/master/include/boost/af
io/v2/detail/impl/afio_iocp.ipp#L511

Writing a symlink: 
https://github.com/BoostGSoC13/boost.afio/blob/master/include/boost/af
io/v2/detail/impl/afio_iocp.ipp#L848

Obviously best not allow rewriting a pseudo-symlink like 
IO_REPARSE_TAG_DEDUP, make it read only.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/