[boost] [gil io_new review] Reading images from in-memory sources

Subject: [boost] [gil io_new review] Reading images from in-memory sources
From: Phil Endecott (spam_from_boost_dev_at_[hidden])
Date: 2010-12-05 15:46:34

> review of Christian Henning's extensions to the Boost Generic Image Library
> starts on December 1st and lasts until December 10th, 2010.

I've been looking at how to read images from in-memory data.  For 
example, I have various places where I mmap() something that contains 
encoded images (e.g. a large map made up from many tiles in a single 
file).  I can also imagine retrieving an encoded image from a database, 
for example.

According to io.qbk:

[h2 Reading And Writing In-Memory Buffers]
Reading and writing in-mempory buffers are supported as well. See as follows:

     ``
     // 1. Read an image.
     ifstream in( "test.tif", ios::binary );

     rgb8_image_t img;
     read_image( in, img, tiff_tag() );

     // 2. Write image to in-memory buffer.
     stringstream out_buffer( ios_base::out | ios_base::binary );

     rgb8_image_t src;
     write_view( out_buffer, view( src ), tiff_tag() );

     // 3. Copy in-memory buffer to another.
     stringstream in_buffer( ios_base::in | ios_base::binary );
     in_buffer << out_buffer.rdbuf();

     // 4. Read in-memory buffer to gil image
     rgb8_image_t dst;
     read_image( in_buffer, dst, tag_t() );

In the cases that I described above, using a stringstream would require 
that all of the data is copied at least once.  I guess that if I knew 
more about how streambufs work I would know how to avoid that.  Can 
anyone help me to implement this:

void read_mem_jpeg( some_gil_image_tpye& dest, const char* 
jpeg_mem_begin, const char* jpeg_mem_end) { ... }

in a way that doesn't copy all of the encoded data?

Looking at formats/jpeg/read.hpp, it seems that irrespective of that 
the data will be copied again (into 'buffer') before being read by libjpeg.

On the subject of copying, while reading PNGs it looks like the data is 
always copied into a temporary buffer in read_rows<>() even when no 
format conversion is needed (or is there some specialisation that I 
have missed?).  And when reading JPEGs, it seems to always read single 
lines, yet libjeg advises that you should get at least 
rec_outbuf_height lines on each call to avoid extra copying within the library.

Regards,  Phil.