Subject: Re: [boost] [btree] Status report
From: Cory Nelson (phrosty_at_[hidden])
Date: 2011-05-21 07:17:07
On Thu, May 19, 2011 at 11:07 AM, Beman Dawes <bdawes_at_[hidden]> wrote:
> On Thu, May 19, 2011 at 3:24 AM, Rutger ter Borg <rutger_at_[hidden]> wrote:
>> 3) It would be nice if it would work with an asynchronous file-like
>> services, like, e.g., Asio's RandomAccessHandleService, see
> I'd need stronger motivation that "it would be nice"! What would the benefit be?
I've got a fuzzy full-text search running on top of my own async
b-tree code right now.
Searching needs to hit the b-tree several times depending on the
length of the query. Being able to launch all these b-tree queries in
parallel is very important for performance, but launching one thread
for each query is tremendously sub-optimal. In a server environment
that processes hundreds of these full-text searches at once, it might
mean asking for thousands of threads! With async I can do all of this
very efficiently using only a single thread (or any number of threads
of my choosing).
This said, async will significantly increase the amount of code needed
to implement this library. I wouldn't want it to be delayed for it,
especially when most people won't need async.
If it could be designed in a way that decouples the core bits from I/O
as much as possible so that it would be easy to move over later, that
would be cool. ie. implement operations with simple I/O-agnostic
state machines -- instead of doing any potentially blocking operation
like filling buffers directly, it requests the user of the state
machine to fill the buffer for it and resume processing afterward. A
blocking lower_bound() might be implemented something like:
while((r = sm.run()) != done)
if(r == ...) ...
if(r == need_block) fill_buffer(sm.buf, sm.buf_length);
-- Cory Nelson http://int64.org