Re: [boost] [AFIO] Formal review

Subject: Re: [boost] [AFIO] Formal review
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2015-09-02 17:54:36

> > > Limiting the discussion to just the problem of race free filesystem
> > > on POSIX, imagine the problem of opening the sibling of a file in a
> > > directory whose path is constantly changing. POSIX does not provide
> > > an API for opening the parent directory of an open file descriptor.
> > > To work around this, you must first get the canonical current path of
> > > the open file descriptor, strip off the end, open that directory and
> > > then use that as a base directory for opening a file with the same
> > > leafname as your original file. You then loop all that if you fail to
> > > open that leafname, or the leafname opened has a different inode.
> > > Once you have the correct parent directory, you can open the sibling.
> > > This is an example of where caching the stat_t of handle during open
> > > saves syscalls and branches in more performance important APIs later
> > > on.
> >
> > Sorry for asking a question again (and I'm sure Niall will ignore it
> because
> > he judges it as 'disrespectful, petty, or juvenile', but perhaps
> somebody
> > else can answer this):
> 
> You could treat me with the respect due to someone working in this
> field for several years the same as the respect I treat you for your
> ample experience in your domain. If I treated you the way you treat
> me, you would explode as we have seen before with you shouting at
> other people on this list who you felt were being disrespectful to
> you.
> 
> I am not incompetent. If I chose a design, it was for a reason. It
> may have been a suboptimal choice, but it is not due to lack of
> experience nor a lack of prior research. I have been programming for
> over twenty years, and more than fifteen commercially. I have a very
> different style and approach to you, but that does not make it
> inferior or to be mocked or attacked just because I am imprecise in
> your opinion.
> 
> > In another mail you said AFIO wouldn't cache anything, thus must do a
> lot of
> > syscalls over and over again, which is the reason for it to be slow.
> 
> We don't second guess the OS. If the OS lies to us - and it does,
> frequently - we pass through those lies exactly. Similarly, if you
> loop requesting the length of a file, you'll get back exactly what
> the OS tells us each time. We always ask the OS for the length if you
> ask AFIO for the length. No caching.
> 
> However the semantics emulation and workaround code still needs to
> correctly emulate any missing semantics on some platform in order to
> provide a universal file system model to the AFIO user. AFIO provides
> race-free sibling lookup which works just fine on Windows, but
> requires some extra work on POSIX, so on POSIX it runs the algorithm
> described above.
> 
> The difference is not unlike the difference between FreeBSD and Linux
> with respect to POSIX. POSIX defines a universal portable standard,
> and both Linux and FreeBSD implement most of that standard. FreeBSD
> implements POSIX far more accurately than Linux, and has
> correspondingly lower peak performance in everything from mutexs to
> opening files.
> 
> However, once one includes the cost of the workarounds for Linux's
> divergences from POSIX in application code, much of that performance
> gap is eliminated. FreeBSD also scales out much better than Linux
> because the lower peak performance in a 1-4 CPU machine is really a
> situation with a higher fixed cost but a less steep slope on the
> variable cost curve, so as you ramp up the hardware FreeBSD scales
> better.
> 
> AFIO is similar to POSIX here. It provides a universal standard
> model. All platforms require workarounds to match that model, but the
> workarounds vary between even Linux and FreeBSD. If AFIO must execute
> extra syscalls on some particular platform in some particular case,
> you should accept that as a sunk cost on that platform and not worth
> worrying about in the bigger picture as the simplified application
> code with fewer workarounds is likely to perform better, be easier to
> maintain, and transparently improves as OS facilities improve. Your
> application code is also much more portable.

Thanks!

Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu