Re: [boost] Serialisation: Is is_trivial<T> a sufficient precondition to bypass serialisation?

Subject: Re: [boost] Serialisation: Is is_trivial<T> a sufficient precondition to bypass serialisation?
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-02-04 19:55:10

On 4 Feb 2015 at 21:46, Thomas Heller wrote:

> > Basically, AFIO v1.3 has new pre-serialisation metaprogramming in
> > response to Robert's comments on his Incubator about it being too
> > hard to serialise and deserialise data with AFIO. AFIO will now
> > consume any STL container and will implicitly auto expand to ASIO
> > scatter gather buffers any:
> >
> > 1. Trivial type T
> > 2. C array of trivial type T
> > 3. STL container of trivial type T, including initializer_list.
> 
> So you generate a list of N scatter/gather buffers for a container with the
> size of N? That sounds a tad too much. Doesn't that create a immense
> overhead for the network interface?

Firstly AFIO does file i/o, not network i/o. You'd use ASIO for 
network i/o.

If the user feeds std::list<int>(1000) to AFIO write(), I don't think 
you get a choice here: a gather write of 1000 individual items is 
requested and that is what is issued.

If the user feeds std::vector<int>(1000) to AFIO write(), there is 
already a specialisation of to_asio_buffers() for vector, array, 
string and C arrays which recognises the fact they guarantee 
contiguous storage of their contents. In that situation AFIO issues a 
single gather buffer for the entire contents at once.

If the user feeds std::vector<std:vector<int>(1000)>(100), then AFIO 
will create an ASIO gather buffer sequence of 100 buffers, each 
pointing at a region of 1000 ints. You can nest your STL containers 
to any arbitrary length - AFIO understands. AFIO also understands you 
can gather write a const value_type container like unordered_map, but 
cannot scatter read into a const value_type and a static assert is 
thrown if you try. In other words, const containers or value_types 
always turn into asio::const_buffer, as indeed they should.

As with network sockets, most DMA engines for disk i/o have hard 
limits on scatter gather buffer size. That isn't AFIO's problem. 
POSIX also imposes a limit of IOV_MAX scatter gather buffers which is 
AFIO's problem, and AFIO correctly issues blocks of IOV_MAX until the 
input is fully dispatched.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/