Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11

Subject: Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11
From: Lee Clagett (forum_at_[hidden])
Date: 2017-03-30 14:06:30

On Thu, 30 Mar 2017 08:34:07 -0400
Lee Clagett <forum_at_[hidden]> wrote:
> On Wed, 29 Mar 2017 17:18:23 -0400
> Vinnie Falco via Boost <boost_at_[hidden]> wrote:
> > On Wed, Mar 29, 2017 at 5:04 PM, Niall Douglas via Boost
> > <boost_at_[hidden]> wrote:  
> > > as that paper Lee linked to points out, everybody writing storage
[...snip...]
> 
> This still has implementation/system defined behavior, but overwriting
> a single sector is more likely to be "atomic" from the perspective of
> the filesystem (but not necessarily the hard-drive). And it didn't
> require massive structural changes. Writing out a cryptographic hash
> of the header would leave a single assumption - fsync is a proper
> write barrier in the OS/filesystem and in the hard-drive. Niall has
> been particularly harsh on fsync, but I do not think its all bad.
> With the exception of OSX, it seems that many filesystems implement
> it properly (might regret saying this), and a user can purchase an
> "enterprise" hard-drive that is not trying to artificial boost
> benchmarks stats. At the very least the number of assumptions has
> been decreased.

I forgot that even with the cryptographic hash this algorithm is
assuming that the filesystem does not change sectors during a single
sector overwrite. Otherwise it could point back to a prior log header
that was never actually overwritten. Crap. So a rare but still somewhat
crummy non-portable issue.

Lee