Re:Re: [boost] Relational templates

From: Greg Colvin (greg_at_[hidden])
Date: 2002-04-30 12:45:00

At 08:16 PM 04/29/2002, Vertleyb wrote:
>Mohammed,
> 
>> All what u said is good. What about if I want to store a huge number of
>> records in a file? Is it possible to retrieve data quickly based on a
>> certain creteria? What r the limitation of data stored and retrieved?
>
>Efficient processing of huge amount of data located on the disk is not our
>current goal.
>We target applications that process small amount of data that all can fit
>into RAM.
>We believe for such applications SQL/ODBC is not the best choice.

With gigabytes of RAM available "small" is a lot larger than it used to
be, and with virtual memory and memory mapping of files the disk versus
memory distinction is less relevant than it used to be.

What that means is that it pays to pay attention to how memory maps
onto disk blocks to avoid thrashing.  I don't know that your relational
templates should do this directly,but it should be possible to separate
your algorithms from how the data is stored.

Also, some relational operations can be much more efficient if indexes
are available, so you should support optional indexing.  That is, your
operations should work with just sequential access to data, but should
take advantage of random access and associative access where possible.