Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11

Subject: Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11
From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2017-03-26 22:20:31

On Sun, Mar 26, 2017 at 6:07 PM, Niall Douglas via Boost
<boost_at_[hidden]> wrote:
> ...That includes
> making use of collisions in weak hashes to direct your power loss
> recovery implementation to sabotage and destroy additional data after
> the power loss.

No idea what you're going on about here, NuDB's recovery process
doesn't recalculate hashes.

> NuDB is not using cryptographically strong hashing, and is therefore
> subject to collision induced post power loss data loss.

Again no idea what you're going on about here, the presence of
collisions in the hash function in no way harms the integrity of a
NuDB database. When two keys have the same hash value, they simply
occupy adjacent slots. The fetch() algorithm iterates all entries in
the bucket which have the same hash value:
https://github.com/vinniefalco/NuDB/blob/master/include/nudb/impl/basic_store.ipp#L362

The database constants were chosen to make a collision incredibly
unlikely, and the implementation has code to explicitly handle the
case where a collision happens.

As I said before, NuDB was designed for adversarial inputs, each
database has a randomly calculated "salt" which permutes the hash
function. An attacker would have to compromise the machine and extract
the salt in order to reliably produce keys which hash to the same
value. NuDB assumes attackers do not have root access to the machine
upon which it runs (or else anything is possible really).

> If NuDB clearly said in its docs "no durability guarantees except on
> this list of filing systems and mount options: ..." I'd be happy.

There are definitely filing systems and mount options which are
unsuitable for NuDB. All I can reliably say is that it works for some
subset of all possible systems, without explicitly mentioning what
those systems are - only because I am just a humble peon from a small
podunk who doesn't know the way of things.

However, if there was someone out there who was worldly in their
travels with much knowledge and experience across a broad spectrum of
filesystems, operating systems, mount options, and hardware, who
perhaps has dedicated considerable time to developing file libraries
of their own - maybe that person could leverage their immense wisdom
to helping me compile such a list of filing systems and mount options?

Thanks!