Re: [Boost-users] memory mapped io and boost

From: peter_foelsche_at_[hidden]
Date: 2008-05-02 14:17:14

> > Dear All,
> > 
> > from what it looks like the memory mapped io classes in 
> > boost::interprocess cannot deal with readonly files.
> 
> Which memory mapped classes? Managed or not?

managed

> For managed_mapped_file you can't use read-only, because there are 
> mutexes built in the mapped file that have to be locked in case one 
> process opens creates new elements.

I think this (synchronization) is overkill.
Most people want memory mapped io to write/read from files.
Synchronization is not necessary.
Of course creating new elements in a readonly file would fail.

> So I guess you are referring to managed classes, right?
> a)The problem with allowing read-only is that another process can open
>  
> it for writting and you would have problems.

this may cause a crash. 
The file needs to be opened in a way which prevents other openings for writing.
Problem is that UNIXs don't allow for locking of files.
But at least on windows this can be done.

> 
> b)The internal structure should be changed to avoid using mutexes and 
> that can be difficult because internal structures and user-defined 
> algorithms and indexes should avoid that (and also user-defined object
> s 
> that write in some read-operations).
> 
> You also say that disk-space is being allocated when growing a file. 
> That's the idea. Otherwise, it would be the internal "offset_ptr" 
> pointers would become useless unless we map all file portions 
> contiguously and try to manage several mapped regions (which might be 
> a 
> bit tricky). Anyway, I don't see why growing a file should be slower 
> than creating a new one. The OS looks for free disk sectors so file 
> contents are not physically contiguous although you get a single file.
>  
> Why is making the file sparse an advantage?

in a sparse file (the default on UNIXs) diskspace is only allocated when needed.
So if you create a new file and do
        fd = open("filename", O_CREAT | O_TRUNC, 0600);
        seek(fd, 4*1024*1024*1024, SEEK_SET); 
        write(fd, "", 1);
only a single sector is being allocated.
You can have a huge file on disk (according to the filesize) which in fact takes nearly no disk space.
Only when writing into it (e.g. via memory mapped io) disk space is being allocated.
If you do this (create an empty big file) on Windows the default is to allocate disk space -- thus the matching system call may be very slow. 
Only if you set some file attribute this is avoided.
Problem is when the system runs out of disk space while writing into memory_mapped memory.
On UNIXs you will get a useless signal which gives you the choice of terminating the process.
I don't know about Windows but I guess you get a structured exception which can be caught in C++.

Peter

> 
> Regards,
> 
> Ion
> 
> 
> 
> > 
> > This is a big problem.
> > 
> > I think opening readonly files should be possible but all write 
> > operations to the file should fail.
> > 
> > Of course mapping a readonly file should be done with the matching f
> lags 
> > so that the mapping can be shared and that the matching address spac
> e is 
> > readonly.
> > 
> > Also another drawback is that at least on Windows disk-space is bein
> g 
> > allocated when growing a file.
> > 
> > This is slow and it can be avoided by making the file sparse.
> > 
> > On UNIXs this is the default.
> > 
> >  
> > 
> > Thanks
> > 
> > Peter
> > 
> > 
> > --------------------------------------------------------------------
> ----
> > 
> > _______________________________________________
> > Boost-users mailing list
> > Boost-users_at_[hidden]
> > http://lists.boost.org/mailman/listinfo.cgi/boost-users
> 
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
> 
>