Re: [boost] Re: Re: STL applied to disk

From: David Abrahams (dave_at_[hidden])
Date: 2002-11-19 10:38:27

"Wesley W. Terpstra" <terpstra_at_[hidden]> writes:

> Sorry for the latency; was ill.
>
> A status update (since I was working without internet while sick).
>
> I have a working vector<T> wrapper as a prototype. It supports every
> operation and type specificied on the SGI site except i->foo(), and
> vector<T>::pointer is void*. The iterators are random access, if mutable
> then *x = t; works and *x is convertable to T.
>
> One interesting detail was this code fragment:
>
> 	typedef JFA::VectorDB<IntTraits>	DB;
> 	typedef DB::Session			Session;
> 	
> 	JFA::Environment env = JFA::Guess("/tmp");
> 	DB db(env, "test");
>
> 	JFA_BEGIN_TRANSACTION(t, env) // opens retry block
> 	Session s = t(db);
> 	
> 	int k = 0;
> 	Session::const_iterator i;
> 	Session::iterator e = s.end(); // getting iterators is expensive
> 	for (i = s.begin(); i != e; ++i) k += *i;
>
> 	JFA_END_TRANSACTION(t) // commits and closes block
>
> The interesting thing to note is that s.begin() and s.end() are MUTABLE
> iterators. Accessing these requires write-locking the sector they are on.
> Yes I could require ((const)s).begin(), but I think that this is bulky.
>
> However, I found that by returning a proxy object, I could delay the locking
> till *i. This means that the above code does not write-lock at all.
> (It also reads 40M in under half a second on my budget hardware :-)
>
> So... I am beginning to lean towards the "don't do that" approach where I
> simply don't allow the user to call member methods on items in the
> container. (And not let them take pointers) This allows at least the above
> optimization and a few others (like *i = *j; -- no deserialize&serialize)
> and probably more I don't forsee yet.

I haven't been paying attention, but IIUC what you're proposing, these
things are no longer conforming iterators.

The way to make random access iterators over disk storage is to build
an iterator which stores its value_type internally. You can even
arrange for it to construct the value_type in its internal storage on
demand, so that it doesn't store anything until it is dereferenced.

-- 
                       David Abrahams
   dave_at_[hidden] * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution