Re: [boost] [gil] New IO release

Subject: Re: [boost] [gil] New IO release
From: Christian Henning (chhenning_at_[hidden])
Date: 2010-10-11 11:27:12

Hi Domagoj, thanks for hard work. I haven't had a chance to look at it
but if everything that you say is true that's great! There is quite a
bit to answer here. See below:

> Hi, a while back I wrote about using alternative GIL.IO backend
> implementations that would use native libraries/functionality (e.g. GDI+ on
> Windows, QuickTime on Mac...) and provided a GDI+ implementation for the
> original/current/official GIL.IO. That version used a configuration macro to
> select backend implementation and you remarked that I should rather use a
> different type to differentiate backends. That was a very good point as it
> pointed :) in the right direction, where/how should the GIL.IO interface
> change: to use objects instead of free functions (as was already requested
> by other users) however you chose not to go that way but to only refine the
> existing interface...because I did not agree both with this design decision
> as well as the many implementation decisions (such as using std::streams for
> in-memory image loading) I 'sat down' to write my own proposal...

I'm a little lost here. What objects and free functions do you mean?
And what way did I not choose to go? I apologize, my memory is a
little spotty.

Why not use std::streams and what are the alternatives? In the end I
understand std::streams as providing an interface though other
implementations can be used, as well.

> - wic_image (WIC (WinXP SP3+)).

WIC?

> xxx The design/interface difference xxx
> --------------------------------------------
>
> Unlike io and io_new it uses objects that represent on-disk images
> ("formatted images" in io2-speak). This has several advantages:

Could you go into more details? What are "formatted images"?

> - you do not have to open an image twice in order to first query its
> properties (if you need to do so for whatever reason) and then to actually
> read it...which is both cumbersome and inefficient...

Agreed. In io_new you can reuse a std::ifstream as often as you need.
Same for FILE*.

> - you do not have to open an image many times (and seek thru it over and
> over again) when reading an image in smaller pieces/ROIs...

That's the same statement as the first one, correct? ROI is Region Of Interest?

> - an object based design allowed for a CRTP based design where most of the
> shared boiler plate code could have been extracted into a single base class
> which in turn allows for greater maintainability as well as easier
> extensibility

CRTP?

> - it provides access to the underlying backend to the user for maximum
> flexibility. With the current design, if the GIL.IO wrapper does not provide
> access to a backend feature users are helpless: for example, users had to
> wait for months for the library maintainer to add support for TIFF directory
> selection by adding yet another global function or adding more (defaulted)
> parameters to the existing one...With io2 this would not be necessary, the
> user could simply say:
>     libtiff_image my_tiff( "my_tiff.tiff" );
>     ::TIFFSetDirectory( &my_tiff.lib_object(), <a directory number> );
>     my_tiff.copy_to(....);

That's a neat feature that I like to have too.

> - it allows for easy/direct selection of the preferred backend and/or using
> several different backends...
> All in all the issue of easier writing/adding and selecting backend wrappers
> is very important because someone might want to use FreeImage or LodePNG or
> GDI+ or might simply be forced into using a specific backend by a 3rd party
> library (like a GUI framework)...

Again, that's good to have. Are you providing a jamfile that has such
flexibility?

>
> Furthermore IO2 also provides moving ROI capabilities, meaning you can read
> a large image in pieces by repeatedly calling an appropriate method on the
> image object thereby avoiding the need to reopen the image an seek through
> it. It will also try to be smart and read/decode as little as possible of
> unwanted data when skipping through an image.

I would love to know how you seek through a image using a 3rd party
lib, for instance with libjpeg. Right now I'm just reading and
discarding unwanted regions. Not the most ideal solution to say the
least.

>
> IO2 also tries to be very metaprograming friendly, for example the various
> backends provide introspective information either directly or thru a traits
> class:
>  - mpl typelist of supported GIL image/pixel formats
>  - mpl typelist of supported on-disk-image formats
>  - mpl typelist of supported source types (e.g. all backends support char
> const *, FILE & and a memory-range, while wic_image, for example, also
> supports wchar_t const *, HANDLE and IStream &)
>  - its native ROI and offset types
>  - a metafunction that can be queried whether a pixel format is natively
> supported
>  - the desired allocation alignment
>  - whether it has builtin conversion...

Sounds all good if I understand correctly. I believe io_new does some
of these things, as well.

>
> It can read from in-memory images which it models using a plain
> boost::iterator_range<unsigned char const *>. So if you have a static image
> like:
> unsigned char const my_static_png[] = {...}
> io2 will read it directly without the overhead of an intermediate
> std::stream object...

io_new tries to have an extensible framework though supporting such
plain byte arrays can be added.

>
> Various policies for configuring the reading process are also provided,
> e.g.:
> {
>     typedef libpng_image::reader_for<memory_chunk_t const &>::type reader_t;
>     typedef image<bgr8_pixel_t, false> image_t;
>
>     image_t test_image;
>     reader_t reader( my_static_png );
>     reader.copy_to_image( test_image, synchronize_dimensions(),
> assert_formats_match() );
>
>     wic_image::writer_for<char const *>::type( "test.jpg", test_image._view,
> jpeg ).write_default();
> }

Not sure what synchronize_dimensions and assert_formats_match stands
for. In io_new you will get an assertion when you try to read an image
( without conversion ) that is incompatible
with the user provided image. For instance in case a file is rgb8 a
user can read it with bgr8 image.

> xxx The implementation difference xxx
> --------------------------------------------

[snip]

> notes:
> - the starting overhead of static linking with the CRT and of constructing
> and resizing a GIL image<> is 40.448 bytes

What do you mean?

> - LibJPEG was additionally compiled with the NO_GETENV macro and LibPNG with
> the PNG_NO_CONSOLE_IO (additionaly for io2 it was built with PNG_NO_STDIO,
> PNG_NO_WARNINGS, PNG_NO_ERROR_TEXT macros, io and io_new would not link with
> those macros)

When using these compiler symbols do you experience significant speed ups?

> - I don't know if I missed something or did something seriously wrong but
> the io_new TIFF reader seems broken in the sense that it does not do any
> pixel/image format conversion and thus works properly only when the
> destination image/view is in the same format as the image on disk....?

I'll investigate.

> - unfortunately io2 currently builds only with MSVC++...

Shouldn't be too hard to make changes to compile with gcc. I don't
have gcc but I remember what to do. Maybe you wanna try MinGW and
code_blocks IDE.

>
> There are many many things io2 does (or does not do) to achieve these
> results, some of them are:
> - it has special support/handling/code-paths for in-memory/"basic" views for
> which it decodes directly to the target view avoiding any intermediate
> buffers (and inherent memory allocation and copying. Even if the backend
> does not support the target view's format it can sometimes still decode
> directly to target view memory space and then do an in-place transformation

A basic view is byte array? Not sure what you mean.

> - it maximally uses functionality provided by the backends, so if a backend
> provides conversion capabilities and the user only specified the
> 'synchronize_formats' policy without explicitly specifying a colour
> converter the backend's builtin conversion will be used

That sounds great. I have to look at how you deal with certain
functionalities, like reading ROI in a jpeg image.

> - it completely avoids expensive/bloating STL and/or boost constructs, it
> does not use a single std::vector (if required a scoped_ptr does just fine),
> std::string (unlike io_new which, for example, creates and destroys 8
> useless std::strings before it even calls jpeg_start_decompress(), or 15
> std::strings before the first call to TIFFReadScanline() and then one
> std::string for every subsequent TIFFReadScanline() call), shared_ptrs and
> of course no streams...it also avoids iterator_facade abstractions as I've
> seen them producing code several times larger than the loop they wrap...

As far as I remember I use references when passing strings along.
Could you tell where in my code you see that useless string creation
destroying problem?

> - uses streamlined/minimized/out-of-the-main-code-path-as-possible error
> detection and handling

Not sure what you mean.

> - uses custom error handling and input-output routines that avoid the
> use/inclusion of stdio and printf family of functions (and possibly allow
> the linker to remove the big error/warning message tables provided by some
> of the libraries), use memory mapped files for input, use MSVC capability to
> throw through C code (avoiding the use of setjmp)

I have some sort of error handling implemented that users can
customize. Not for all supported formats but for some.

You do understand that io2 has to have portable source code in order
to become a boost lib? Any idea of avoiding setjmp in a portable
manner?

> ps. The libjpeg_image::read( ... ) interface used in the test code is a
> simple utility static member function provided by the base CRTP class (thus
> available in all backends automatically) that simply wraps the default
> ...reader_for<>::...copy_to(...) code...

I really don't understand.

>
> pps. There is one more thing I developed along with/for io2 and that is the
> ability to fully configure the way 3rd party libraries (used by the
> backends) are linked and initialised (although this is currently only
> implemented for GDI+ and WIC). So if you have an image-centric application
> you will probably want the backend lib to be linked statically and
> initialised once on startup , if on the other hand you use GIL.IO only for
> things like loading a skin a application startup and then no longer need
> image IO functionality you'll want to load and initialise the library
> on-demand once, do the work and then release it...and there are all possible
> combinations in between (e.g. static linking can be with a .lib/.a or a
> .dll/.so, dynamic linking can be 'only delayed but one-time loading' or can
> be repeated loading, unloading and reloading of a .dll etc...)...This is
> configured globaly with a macro, for example:
> #define BOOST_GIL_EXTERNAL_LIB ( BOOST_LIB_LINK_LOADTIME_OR_STATIC,
> BOOST_LIB_LOADING_STATIC, BOOST_LIB_INIT_ASSUME )
> According to the BOOST_GIL_EXTERNAL_LIB macro individual library backends
> will make assumptions as to in which state the 3rd party library is in when
> they are constructed and what else they need to do to fully initialise
> it...To ensure the 'contract' specified with the BOOST_GIL_EXTERNAL_LIB
> macro each backend provides a public nested type called "guard" intended for
> the user to instantiate it before using the backend. For example:
>     gp_image::guard const lib_guard;
>     ...
>     do something
>     ...
>     GDI+ automatically cleaned up at end of scope...
>

That sounds great and certainly is a nice-to-have feature.

In conclusion, I must say I'm impressed with your io2. I think there
are a lot of things we both can reuse and add into our projects. Maybe
a merge could be possible. One of my highest goals is to support as
many image formats as possible. For instance, a lot of work went into
the tiff extension since it's to most flexible. Here, all bit image
types are supported, for instance rgb7_14_12 in a tiled or strip
manner.

Lets try to make to most useful gil.io extension possible.

Regards,
Christian