Re: [boost] Boost Serialization: Serializing large objects

Subject: Re: [boost] Boost Serialization: Serializing large objects
From: Chris Glover (c.d.glover_at_[hidden])
Date: 2017-10-11 18:46:10

>
>
> Thanks Robert. Does Boost use operator<< to add data to the stringstream?
> If not, how is data added?
> I would like to know so that I can look into why using that method we get
> an error.
>
> Adlai
>

You say the object is 700 megs. Is that 700 megs in memory or in text? The
text representation is almost certainly bigger than the in memory
representation so you might just be running out of memory.

Alternatively, you could just be running out of address space if
stringstream requires a contiguous buffer. That's an implementation detail,
but I would not be surprised if stringstream allocates a contiguous buffer,
in which case you almost certainly would be bumping into a memory
fragmentation issue due to the 32bit address space.

If you serialize directly to a file instead of stringstream, as Robert
suggested, the problem might go away.

-- chris