Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11

Subject: Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11
From: Lee Clagett (forum_at_[hidden])
Date: 2017-03-29 16:32:55

On Wed, 29 Mar 2017 12:07:11 -0400
Vinnie Falco via Boost <boost_at_[hidden]> wrote:

> On Wed, Mar 29, 2017 at 12:03 PM, Niall Douglas via Boost
> <boost_at_[hidden]> wrote:
> > If he rewrote those two paragraphs above with the below I would
> > have no objection apart from that his design is flawed and should
> > not enter Boost without being substantially redesigned.  
> 
> I could take this seriously if you provide code in the form of a
> regression test which shows the scenario in which a power loss harms
> NuDB database integrity. Because, as they say, "talk is cheap."
> 

Read this [paper on crash-consistent applications][0]. Table 1 on page 5
should be of particular interest. I _think_ the bucket portion of
NuDB's log has no size constraint, so its algorithm is either going to
be "single sector append", "single block append", or "multi-block
append/writes" depending on the total size of the buckets. The
algorithm is always problematic when metadata journaling is disabled.
Your assumptions of fsync have not been violated to achieve those
inconsistencies.

A test case could be writing completely random data to the bucket
portion of the log to see what happens. The behavior should be
indeterminate, because its the equivalent of reading uninitialized
memory.

Lee

[0]https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-pillai.pdf