Re: [boost] [gil] New IO release

Subject: Re: [boost] [gil] New IO release
From: Domagoj Saric (dsaritz_at_[hidden])
Date: 2010-10-10 21:26:21

From: "Christian Henning" <chhenning_at_[hidden]>
Sent: Friday, October 01, 2010 5:14 PM
> Hi there, a new gil IO extension is ready for download. Here is the link:
>
> http://code.google.com/p/gil-contributions/downloads/detail?name=boost_review.zip&can=2&q=#makechanges
>
> This is the final version that will be used for the boost review.

Hi, a while back I wrote about using alternative GIL.IO backend
implementations that would use native libraries/functionality (e.g. GDI+ on
Windows, QuickTime on Mac...) and provided a GDI+ implementation for the
original/current/official GIL.IO. That version used a configuration macro to
select backend implementation and you remarked that I should rather use a
different type to differentiate backends. That was a very good point as it
pointed :) in the right direction, where/how should the GIL.IO interface
change: to use objects instead of free functions (as was already requested
by other users) however you chose not to go that way but to only refine the
existing interface...because I did not agree both with this design decision
as well as the many implementation decisions (such as using std::streams for
in-memory image loading) I 'sat down' to write my own proposal...
Since "io_new" (allow me to dub your proposal that way for the purpose of
this post, while "io" will be the original "io" and "io2" will be my
proposal) is out the door I have cleaned up and updated my version at:
https://svn.boost.org/svn/boost/sandbox/gil
to a working state. It provides 5 backends with full read and write
capabilities:
- libjpeg_image (LibJPEG)
- libpng_image (LibPNG)
- libtiff_image (LibTIFF)
- gp_image (GDI+ (WinXP SP2+))
- wic_image (WIC (WinXP SP3+)).

xxx The design/interface difference xxx
--------------------------------------------

Unlike io and io_new it uses objects that represent on-disk images
("formatted images" in io2-speak). This has several advantages:
- you do not have to open an image twice in order to first query its
properties (if you need to do so for whatever reason) and then to actually
read it...which is both cumbersome and inefficient...
- you do not have to open an image many times (and seek thru it over and
over again) when reading an image in smaller pieces/ROIs...
- an object based design allowed for a CRTP based design where most of the
shared boiler plate code could have been extracted into a single base class
which in turn allows for greater maintainability as well as easier
extensibility
- it provides access to the underlying backend to the user for maximum
flexibility. With the current design, if the GIL.IO wrapper does not provide
access to a backend feature users are helpless: for example, users had to
wait for months for the library maintainer to add support for TIFF directory
selection by adding yet another global function or adding more (defaulted)
parameters to the existing one...With io2 this would not be necessary, the
user could simply say:
    libtiff_image my_tiff( "my_tiff.tiff" );
    ::TIFFSetDirectory( &my_tiff.lib_object(), <a directory number> );
    my_tiff.copy_to(....);
- it allows for easy/direct selection of the preferred backend and/or using
several different backends...
All in all the issue of easier writing/adding and selecting backend wrappers
is very important because someone might want to use FreeImage or LodePNG or
GDI+ or might simply be forced into using a specific backend by a 3rd party
library (like a GUI framework)...

Furthermore IO2 also provides moving ROI capabilities, meaning you can read
a large image in pieces by repeatedly calling an appropriate method on the
image object thereby avoiding the need to reopen the image an seek through
it. It will also try to be smart and read/decode as little as possible of
unwanted data when skipping through an image.

IO2 also tries to be very metaprograming friendly, for example the various
backends provide introspective information either directly or thru a traits
class:
 - mpl typelist of supported GIL image/pixel formats
 - mpl typelist of supported on-disk-image formats
 - mpl typelist of supported source types (e.g. all backends support char
const *, FILE & and a memory-range, while wic_image, for example, also
supports wchar_t const *, HANDLE and IStream &)
 - its native ROI and offset types
 - a metafunction that can be queried whether a pixel format is natively
supported
 - the desired allocation alignment
 - whether it has builtin conversion...

It can read from in-memory images which it models using a plain
boost::iterator_range<unsigned char const *>. So if you have a static image
like:
unsigned char const my_static_png[] = {...}
io2 will read it directly without the overhead of an intermediate
std::stream object...

Various policies for configuring the reading process are also provided,
e.g.:
{
    typedef libpng_image::reader_for<memory_chunk_t const &>::type reader_t;
    typedef image<bgr8_pixel_t, false> image_t;

    image_t test_image;
    reader_t reader( my_static_png );
    reader.copy_to_image( test_image, synchronize_dimensions(),
assert_formats_match() );

    wic_image::writer_for<char const *>::type( "test.jpg", test_image._view,
jpeg ).write_default();
}

xxx The implementation difference xxx
--------------------------------------------

IO2 is also much more efficient and less bloated (striving for zero-overhead
comparing to direct usage of the backend library). For example, the
following code:

int main( int /*argc*/, char * /*argv*/[] )
{
    using namespace boost::gil;

    typedef image<rgb8_pixel_t, false> test_image_t;

    test_image_t jpeg_test_image;
    test_image_t png_test_image ;

    for ( unsigned int i( 0 ); i < 10000; ++i )
    {
        #if IO
            jpeg_read_and_convert_image( "stlab2007.jpg", jpeg_test_image );
            png_read_and_convert_image( "boost.png"     , png_test_image  );
        #elif IO_NEW
            read_and_convert_image( "stlab2007.jpg", jpeg_test_image,
jpeg_tag() );
            read_and_convert_image( "boost.png"    , png_test_image ,
png_tag () );
        #elif IO2
            libjpeg_image::read( "stlab2007.jpg", jpeg_test_image );
            libpng_image ::read( "boost.png"    , png_test_image  );
        #endif
    }

    return 0;
}

gave the following results:

io:
exe size: 296.448 bytes
execution time: 10,3 seconds

io_new
exe size: 319.488 bytes
execution time: 12,1 seconds

io2 (using LibJPEG and LibPNG)
exe size: 244.224 bytes
execution time: 8,8 seconds

io2 (using WIC)
exe size: 42.496 bytes
execution time: 9,3 seconds
(GDI+ is similar in size but quite slower)

In this simple example io_new shows a regression both in terms of code bloat
and execution speed, being ~30% larger and ~37,5% slower than io2...

notes:
- the starting overhead of static linking with the CRT and of constructing
and resizing a GIL image<> is 40.448 bytes
- all libraries and test code were built using MSVC++ 10 with the following
relevant options:
/Oxt /Ob2 /Oi /Oy /GL /D "NDEBUG" /GF /MT /GS- /Gy /arch:SSE /fp:fast
/fp:except- and /LTCG for the linker...
- LibJPEG was additionally compiled with the NO_GETENV macro and LibPNG with
the PNG_NO_CONSOLE_IO (additionaly for io2 it was built with PNG_NO_STDIO,
PNG_NO_WARNINGS, PNG_NO_ERROR_TEXT macros, io and io_new would not link with
those macros)
- I don't know if I missed something or did something seriously wrong but
the io_new TIFF reader seems broken in the sense that it does not do any
pixel/image format conversion and thus works properly only when the
destination image/view is in the same format as the image on disk....?
- unfortunately io2 currently builds only with MSVC++...

There are many many things io2 does (or does not do) to achieve these
results, some of them are:
- it has special support/handling/code-paths for in-memory/"basic" views for
which it decodes directly to the target view avoiding any intermediate
buffers (and inherent memory allocation and copying. Even if the backend
does not support the target view's format it can sometimes still decode
directly to target view memory space and then do an in-place transformation
- it maximally uses functionality provided by the backends, so if a backend
provides conversion capabilities and the user only specified the
'synchronize_formats' policy without explicitly specifying a colour
converter the backend's builtin conversion will be used
- it completely avoids expensive/bloating STL and/or boost constructs, it
does not use a single std::vector (if required a scoped_ptr does just fine),
std::string (unlike io_new which, for example, creates and destroys 8
useless std::strings before it even calls jpeg_start_decompress(), or 15
std::strings before the first call to TIFFReadScanline() and then one
std::string for every subsequent TIFFReadScanline() call), shared_ptrs and
of course no streams...it also avoids iterator_facade abstractions as I've
seen them producing code several times larger than the loop they wrap...
- uses streamlined/minimized/out-of-the-main-code-path-as-possible error
detection and handling
- uses custom error handling and input-output routines that avoid the
use/inclusion of stdio and printf family of functions (and possibly allow
the linker to remove the big error/warning message tables provided by some
of the libraries), use memory mapped files for input, use MSVC capability to
throw through C code (avoiding the use of setjmp)
...

ps. The libjpeg_image::read( ... ) interface used in the test code is a
simple utility static member function provided by the base CRTP class (thus
available in all backends automatically) that simply wraps the default
...reader_for<>::...copy_to(...) code...

pps. There is one more thing I developed along with/for io2 and that is the
ability to fully configure the way 3rd party libraries (used by the
backends) are linked and initialised (although this is currently only
implemented for GDI+ and WIC). So if you have an image-centric application
you will probably want the backend lib to be linked statically and
initialised once on startup , if on the other hand you use GIL.IO only for
things like loading a skin a application startup and then no longer need
image IO functionality you'll want to load and initialise the library
on-demand once, do the work and then release it...and there are all possible
combinations in between (e.g. static linking can be with a .lib/.a or a
.dll/.so, dynamic linking can be 'only delayed but one-time loading' or can
be repeated loading, unloading and reloading of a .dll etc...)...This is
configured globaly with a macro, for example:
#define BOOST_GIL_EXTERNAL_LIB ( BOOST_LIB_LINK_LOADTIME_OR_STATIC,
BOOST_LIB_LOADING_STATIC, BOOST_LIB_INIT_ASSUME )
According to the BOOST_GIL_EXTERNAL_LIB macro individual library backends
will make assumptions as to in which state the 3rd party library is in when
they are constructed and what else they need to do to fully initialise
it...To ensure the 'contract' specified with the BOOST_GIL_EXTERNAL_LIB
macro each backend provides a public nested type called "guard" intended for
the user to instantiate it before using the backend. For example:
    gp_image::guard const lib_guard;
    ...
    do something
    ...
    GDI+ automatically cleaned up at end of scope...

--
"What Huxley teaches is that in the age of advanced technology, spiritual
devastation is more likely to come from an enemy with a smiling face than
from one whose countenance exudes suspicion and hate."
Neil Postman