Re: [Boost-users] [iostream] Problems opening large mapped_file

Subject: Re: [Boost-users] [iostream] Problems opening large mapped_file
From: Heiko Klein (Heiko.Klein_at_[hidden])
Date: 2012-08-07 02:58:38

On 2012-08-07 06:38, Vitaly Budovski wrote:
> On 7 August 2012 02:43, Heiko Klein <Heiko.Klein_at_[hidden]> wrote:
>> Hi,
>>
>> I wanted to test the mapped_file_source iostream device. My code is working
>> well with a normal file_source device and files >= 6GB. When I change the
>> constructor to a mapped_file_source, it works still well for smaller files
>> (e.g. 150MB), but for files > 2GB I get (tested with 2.5GB):
>>
>> exception occured: failed getting file size: Value too large for defined
>> data type
>>
>>
>> And for files > 4GB (tested with 6GB) I get:
>>
>> exception occured: failed opening file: Bad file descriptor
>>
>>
>>
>> My constructor is as follows:
>>
>>          using namespace boost::iostreams;
>>          typedef stream<mapped_file_source> mmStream;
>>          mmStream* mmistream = new mmStream();
>>          mmistream->open(mapped_file_source(fileName));
>>
>>
>>
>> I've tested against boost 1.44 and a 32bit linux (Ubuntu 10.04).
>>
>>
>> Are these general mmap problems, or am I missing some flags for
>> mapped_file_source?
>>
>> Best regards,
>>
>> Heiko
>
> Hello,
>
> 32bit systems have only approximately 3GB of user addressable memory.
> You should upgrade to a 64bit system.
>
>
> Regards,
>
> Vitaly

Vitaly,

thanks for your explanation. This makes sense for the problems with the 
6GB file. But it does not explain the 'failed getting file size: Value 
too large for defined data type' with the 2.5GB file. Do you have an 
idea why this operation fails?

Since I cannot upgrade to 64bit everywhere, I catch now the exception 
and open the stream again as file_source.

Heiko