Re: [Boost-users] [filesystem] problem: is_regular_file and deduplified files (reparse+sparse)

Subject: Re: [Boost-users] [filesystem] problem: is_regular_file and deduplified files (reparse+sparse)
From: Paul Harris (harris.pc_at_[hidden])
Date: 2015-07-23 01:57:35

Hi Niall, you can use the Azure to test this sort of thing... I think.  I'm
trying it out now.

http://blogs.technet.com/b/tommypatterson/p/azureservertrial.aspx

On 23 July 2015 at 09:54, Niall Douglas <s_sourceforge_at_[hidden]> wrote:

> On 23 Jul 2015 at 8:56, Paul Harris wrote:
>
> > With this server comes the new "dedup" feature, that can automatically
> > deduplify files.  This happens on a schedule, eg 2am saturday.  So
> suddenly
> > we are getting messages of failures of software from all over the place,
> > due to fs::is_regular_file()
> >
> > Deduped files have the REPARSE and SPARSE flag set.
> > On the command line, you can run
> > FSUTIL REPARSEPOINT QUERY
> >
> > and the "Reparse Tag Value" is 0x80000013
> >
> > Which is a relatively new flag known as IO_REPARSE_TAG_DEDUP
> >
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa365740%28v=vs.85%29.aspx
> >
> > These files act as normal files, you can fopen and fread them, so I
> assume
> > they should be treated almost like symlink by boost... perhaps not quite
> a
> > symlink because I assume the "lstat" link properties are identical to the
> > file's stat properties.
> >
> >
> > Typically, I iterate over directories and only process files if
> > fs::is_regular_file(filename) is true.
> >
> > I wrote some code to check what the properties were on these files, and
> its
> > not any of the possible enums detected by file_status::type().
> >
> > ideas?
>
> Proposed Boost.AFIO doesn't support IO_REPARSE_TAG_DEDUP because I
> have no access to any system to test the support upon.
>
> However, if AFIO were to support IO_REPARSE_TAG_DEDUP, it would treat
> it identically to a symlink/junction point.
>
> I'd suggest Boost.Filesystem do the same, and treat pseudo-symlinks
> as symlinks. That probably means adding full symlink support for
> Filesystem on Windows. Here are some links to example implementation
> code:
>
> Reading a symlink target:
> https://github.com/BoostGSoC13/boost.afio/blob/master/include/boost/af
> io/v2/detail/impl/afio_iocp.ipp#L511
>
> Writing a symlink:
> https://github.com/BoostGSoC13/boost.afio/blob/master/include/boost/af
> io/v2/detail/impl/afio_iocp.ipp#L848
>
> Obviously best not allow rewriting a pseudo-symlink like
> IO_REPARSE_TAG_DEDUP, make it read only.
>
> Niall
>
> --
> ned Productions Limited Consulting
> http://www.nedproductions.biz/
> http://ie.linkedin.com/in/nialldouglas/
>
>
>
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>