Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11

Subject: Re: [boost] NuDB: A fast key/value insert-only database for SSD drives in C++11
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2017-03-28 13:27:25

> I think there is a point where handling difficult filesystems and
> hardware is out of scope for this library.

I agree. Just don't claim any guarantees about reliability or data
safety and I'm totally happy. I would then advise that if you're not
implementing durability, you might as well remove the inefficiency of
keeping write journals etc and use a design which goes even faster.

I mentioned some time ago that even with AFIO v2 today in its very alpha
state, I reckon in a few hundred lines of code I can meet or beat NuDB
precisely because I would assume no data integrity guarantees and that
lets me use a single file for everything. That, in turn, allows
concurrent inserts, and I suspect I can also allow deletion. Simpler is
better.

I would of course not recommend that anyone use AFIO v2 in its present
state. It's also currently not compiling on POSIX, it awaits some TLC
from me after the ACCU conference next month.

>> COW filing systems generally offer much stronger guarantees than
>> non-COW filing systems. You are correct that if you are operating on
>> one of those, you can skip a ton of work to implement durability.
>> This is why AFIO v2 has "storage profiles" where ZFS, ReFS and BtrFS
>> are all top of tree in terms of disabling work done by AFIO and its
>> clients. FAT32, meanwhile, sits at the very bottom.
> 
> Getting the NuDB file concept to work with AFIOv2 seems like it could
> be very useful then. Does v2 have a method for specifying dependency
> order on writes (I couldn't find any)? I thought v1 had this feature -
> does v2 drop it?

The AFIO v1 review indicated that the community did not want AFIO to
implement a mechanism for dependency ordering. They wanted just a
barebones file i/o abstraction layer, so you get a completion handler
callback and that's all.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/