Re: [Boost-users] Can a mapped_file larger than 2GB be random seeked?

Subject: Re: [Boost-users] Can a mapped_file larger than 2GB be random seeked?
From: Narcélio Filho (narcelio_at_[hidden])
Date: 2015-05-05 02:57:12

Hi Michael!

>> I'm compiling under Ubuntu 14.10 (amd64):
>>
>> $ g++ boost-mmap.cpp -lboost_iostreams
>> $ ./a.out
>> 0
>> 2147493647
>> Segmentation fault (core dumped)
>>
>> Everything is 64-bits:

> You are certain that 'everything' is 64-bits? Without diving too deep
> into any code, this sounds vaguely, suspiciously like a 32-bit
> addressing issue. It could even be something as 'simple' as underlying
> file API, or other, code using shorts, ints, instead of longs, for
> legacy purposes. Or layers built on said underlying API (i.e. boost
> wrappers).
> 
>> $ file a.out
>> a.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically
>> linked (uses shared libs), for GNU/Linux 2.6.32,
>> BuildID[sha1]=d39e3dc0c99e666631747e496bf798b92e4d4c71, not stripped

Yes, I'm pretty sure that all my system and boost libs are all 64 bits:

$ dpkg --get-selections | grep boost-iostream
libboost-iostreams-dev:amd64                    install
libboost-iostreams1.55-dev:amd64                install
libboost-iostreams1.55.0:amd64                  install

$ file /usr/lib/x86_64-linux-gnu/libboost_iostreams.so.1.55.0
/usr/lib/x86_64-linux-gnu/libboost_iostreams.so.1.55.0: ELF 64-bit LSB
shared object, x86-64, version 1 (SYSV), dynamically linked,
BuildID[sha1]=a4839c7904aba3cb68a66739aa801e4d5d50c14f, stripped

I'm using just boost API, that calls mmap(). Is this a boost bug?

Using mmap() directly, it works as expected:

#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>

#include <iostream>
#include <fstream>
#include <cassert>

using namespace std;

int main(int argc, char *argv[])
{
    const size_t GIGA = 1024l*1024l*1024l;
    const size_t FILESIZE = 2l * GIGA + 10000;

    ofstream block("file", ios::binary | ios::out);
    assert(block.good());

    block.seekp(FILESIZE);
    block.write("", 1);
    block.close();

    auto fd = open("file", O_RDWR);

    if (fd == -1) {
        perror("Error opening file for reading");
        exit(EXIT_FAILURE);
    }

    char *map = (char*)mmap(0, FILESIZE, PROT_WRITE, MAP_SHARED, fd, 0);

    if (map == MAP_FAILED) {
        close(fd);
        perror("Error mmapping file");
        exit(EXIT_FAILURE);
    }

    char buffer[10] = {0};

    cout << 0 << endl;
    lseek(fd, 0, 0);
    write(fd, buffer, 1);

    cout << FILESIZE - 2 << endl;
    lseek(fd, FILESIZE - 2, 0);
    write(fd, buffer, 1);

    cout << 1l*GIGA << endl;
    lseek(fd, 1l*GIGA, 0);
    write(fd, buffer, 1);

    if (munmap(map, FILESIZE) == -1) {
        perror("Error un-mmapping the file");
    }

    close(fd);

    return 0;
}

$ g++ --std=c++11 ex.cpp
$ ./a.out
0
2147493646
1073741824

--
[]s, Narcélio.