Re: [boost] Random Access to Files (RAF) - any interest ?

From: Tomas Puverle (Tomas.Puverle_at_[hidden])
Date: 2006-05-25 15:53:34

> > > More realistic feature request would be support
> > > for sparse files.
> > >
> > > The empty areas could be used for appending
> > > new data efficiently, normal operations on RAF
> > > should ignore then.
> > >
> > > Support for sparse files is available in Win32 via
> > > DeviceIoControl(FSCTL_QUERY_ALLOCATED_RANGES).
> > >
> > > I have vague feeling Linux has something like that too.
> >
> > ZFS on Solaris 10 has an extension to the the Posix lseek() flags, 
SEEK_DATA
> > and SEEK_HOLE.  It makes it easy to find the holes in your files.  I may be
> > wrong here but I don't think Linux supports this.
> >
> Sparce file handling was described in Kernighan book more then 20
> years ago. I think that most UNIX-systems produces "holes" by default
> (Linux do this on ext2/ext3 and XFS for sure). It is not a problem to
> RAF. It just ask a system to allocate space and it is not important
> whether the data allocated or "hole" created. when writing in the
> "hole" system will allocate blocks for a file. There may be a problem
> that there is no more space on device, exactly such case is descried
> in MSDN (using memory-maped file on sparce or comprressed file). We
> will just check for an error in such cases.

Oh, I think you misunderstood what I was trying to say.  The current Posix 
lseek() interface supports 3 flags:  SEEK_BEG, SEEK_CUR and SEEK_END.  
However, ZFS adds 2 more flags:  SEEK_HOLE and SEEK_DATA.  They allow you to 
discover the holes in your files, which is not possible under Unix.  Under 
normal Unix semantics, just like you said, a read of a "hole" will just return 
zeros, and even the on-disk size will display the "right" size, even though 
the date isn't actually present.  However, with ZFS, an application can, for 
example, ignore holes.  That could be advantageous for many programs, e.g. 
compression, tar, etc...