From: Manfred Doudar (manfred.doudar_at_[hidden])
Date: 2005-11-24 21:30:03
Robert Ramey wrote:
> Ian McCulloch wrote:
>>Besides, is the boost iostreams library really much slower than a
> I'm pretty confident that its much, much slower, but this will remain
> in dispute until someone runs the code with a profiler.
>>Anyway, this is a side issue. The main point is:
>>David Abrahams wrote:
>>> | For many archive formats and common datatypes there exist APIs
>>> | that can quickly read or write contiguous sequences of those
>>> types | all at once (**). Reading or writing such a sequence by
>>> | separately reading or writing each element (as the serialization
>>> | library currently does) can be an order of magnitude more
>>> | expensive.
> Sorry - that's NOT the main point.
> The main point is - do enhancement for special cases have to be
> incorporated into the the core code so that everybody else
> is obligated to use it? What are the advantages and
> disadvantages of doing so?
Umm, I don't follow - how are others going to be obligated to use those
special cases that don't necessarily apply to them?
> No one is disputing that it desireable to be able to extend
> the library for these special circumstances.
Actually it's *very* desirable.
>>If there is to be any possibility of targetting an archive to this
>>format, then array support is crucial.
> Then just make an archive which does it- what's stopping you?
True - but the suggestion is that it's common enough to want as part
of the core library.
Here we use netCDF (the format brought up by Ian) and HDF-5 data files,
and run with MPI on a mosix cluster ...
the point being that these data types are integral to industries like
ours - where large volumes of data are the norm - and array access is
indeed crucial - we see code here that runs in the order of seconds to
minutes, or hours depending on how data is read and written - but as a
concesion, that is also dependent on the underlying libraries we inherit
from 3rd parties (aka netCDF, HDF-5, et al).
I've been loosly following this thread and I understand your reluctance
to this - but it's a matter of perception as to how needful, and
widespread such a need is.
... back to lurker mode
-- Manfred Doudar MetOcean Engineers www.metoceanengineers.com
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk