Subject: Re: [boost] [gil] New IO release
From: Domagoj Saric (dsaritz_at_[hidden])
Date: 2010-10-17 19:00:25
"Christian Henning" <chhenning_at_[hidden]> wrote in message
> I'm a little lost here. What objects and free functions do you mean?
Your interface uses free (global) functions (like
xxx_read_and_convert_image()...) while my interface uses objects/classes and
corresponding member functions...
> And what way did I not choose to go? I apologize, my memory is a
> little spotty.
You chose not to use/switch to an object base interface but to stick with
the original free (global) function based interface.
> Why not use std::streams
Streams are _evil_ ...
> and what are the alternatives?
In my book anything is an alternative to screams :D
OTOH, only providing a streams based interface as an additional option is
usually just fine (as opposed to forcing a streams only based interface)
>> Unlike io and io_new it uses objects that represent on-disk images
>> ("formatted images" in io2-speak). This has several advantages:
> Could you go into more details? What are "formatted images"?
Well, images in different 'formats' like JPEG, PNG, GIF...obviously not a
very clever name that I just came up with to somehow distinguish them from
'raw'/in-memory images (like gil::image<>)...you are more than welcome to
come up with a more intuitive name ;)
>> - you do not have to open an image twice in order to first query its
>> properties (if you need to do so for whatever reason) and then to
>> read it...which is both cumbersome and inefficient...
> Agreed. In io_new you can reuse a std::ifstream as often as you need.
> Same for FILE*.
(AFAICT) You can reuse the low level file object but not the backend library
object, which is what I was talking about and what is prerequisite for, for
example, efficient 'moving ROI'/sequential-in-blocks image reading...
>> - you do not have to open an image many times (and seek thru it over and
>> over again) when reading an image in smaller pieces/ROIs...
> That's the same statement as the first one, correct? ROI is Region Of
Yes, or a very related one. Yes.
>> libtiff_image my_tiff( "my_tiff.tiff" );
>> ::TIFFSetDirectory( &my_tiff.lib_object(), <a directory number> );
> That's a neat feature that I like to have too.
But I don't see how you can have that with free functions...
>> - it allows for easy/direct selection of the preferred backend and/or
>> several different backends...
>> All in all the issue of easier writing/adding and selecting backend
>> is very important because someone might want to use FreeImage or LodePNG
>> GDI+ or might simply be forced into using a specific backend by a 3rd
>> library (like a GUI framework)...
> Again, that's good to have. Are you providing a jamfile that has such
No jamfile yet (Boost build/bjam are definitely not one of my favourite
subjects or skills for that matter :)
OTOH, I'm not quite sure (with my limited bjam knowledge) why/how would
different backends influence the library's jamfile (atleast as long as it is
> I would love to know how you seek through a image using a 3rd party
> lib, for instance with libjpeg. Right now I'm just reading and
> discarding unwanted regions. Not the most ideal solution to say the
With LibJPEG you cannot really/literally skip unwanted data but you can make
the library read the unwanted data using faster/less precise methods which
is what I currently do (and admittedly use a bit of LibJPEG's internal
implementation detail knowledge for that)...
> Not sure what synchronize_dimensions and assert_formats_match stands
synchronize_dimensions is a policy that causes the target image to be
resized to fit the source image (it is obviously not valid for views),
alternatively assert_dimensions_match and ensure_dimensions_match (throws in
case of mismatch) can be used...
For formats there are the corresponding assert_formats_match,
ensure_formats_match and synchronize_formats (with the latter performing a
builtin conversion if supported by the backend otherwise it uses GIL's
default colour converter) with the addition that a user specified colour
converter can be passed instead of the three mentioned predefined
>> - the starting overhead of static linking with the CRT and of
>> and resizing a GIL image<> is 40.448 bytes
> What do you mean?
that a statically linked program like:
int main( int /*argc*/, char * /*argv*/ )
boost::gil::image<rgb8_pixel_t, false> image;
image.recreate( 1, 1 );
takes 40.448 bytes...
>> - LibJPEG was additionally compiled with the NO_GETENV macro and LibPNG
>> the PNG_NO_CONSOLE_IO (additionaly for io2 it was built with
>> PNG_NO_WARNINGS, PNG_NO_ERROR_TEXT macros, io and io_new would not link
>> those macros)
> When using these compiler symbols do you experience significant speed ups?
Not (direct) speedups but smaller binaries (when coupled with custom error
and IO handling routines) because of removal of stdio code and error message
tables from the binary...
>> - I don't know if I missed something or did something seriously wrong but
>> the io_new TIFF reader seems broken in the sense that it does not do any
>> pixel/image format conversion and thus works properly only when the
>> destination image/view is in the same format as the image on disk....?
> I'll investigate.
Have you perhaps had the chance?
>> - unfortunately io2 currently builds only with MSVC++...
> Shouldn't be too hard to make changes to compile with gcc. I don't
> have gcc but I remember what to do. Maybe you wanna try MinGW and
> code_blocks IDE.
Thanks but I already have a pretty good idea of what needs to be
changed/fixed...it's nothing serious (things like wrapping __decltype and
__assume statements into macros and typing in a few hundred missing
ps. I'll be/am using GCC (and possibly Clang) on OSX...
>> There are many many things io2 does (or does not do) to achieve these
>> results, some of them are:
>> - it has special support/handling/code-paths for in-memory/"basic" views
>> which it decodes directly to the target view avoiding any intermediate
>> buffers (and inherent memory allocation and copying. Even if the backend
>> does not support the target view's format it can sometimes still decode
>> directly to target view memory space and then do an in-place
> A basic view is byte array? Not sure what you mean.
See the GIL view_is_basic<> metafunction...
> That sounds great. I have to look at how you deal with certain
> functionalities, like reading ROI in a jpeg image.
Actually, I forgot to mention, with the LibX (JPEG, PNG, TIFF) backends I do
not provide 'full'/2D/rectangular ROI capability but only 'vertical' ROIs
(i.e. you can specify from which to which row/scanline to read but whole
rows/scanlines are always read...) because the backends do not support
reading partial rows/scanlines so adding full/proper ROIs for those backends
would require emulation which in turn would cause code bloat/complication
and (possible) redundant data copying...For now it seems to me that the user
should handle 'full' ROI support in such cases (if required)...
LibTIFF is a special case here because it does support 'full' ROIs (although
not arbitrarily aligned/positioned) with tiled images. However this is not
something one can detect at compile-time so a different approach would have
to be devised if this 'possible/dynamic full ROI' support is to be added to
> As far as I remember I use references when passing strings along.
> Could you tell where in my code you see that useless string creation
> destroying problem?
Place a breakpoint in the std::string constructor...it is mostly due to the
change you made to the io_error() helper function (to take a std::string
const & instead of char const *)...
>> - uses streamlined/minimized/out-of-the-main-code-path-as-possible error
>> detection and handling
> Not sure what you mean.
For example not performing a full check-and-throw on every library call but
accumulating multiple results and throwing in one place/out of a loop...
Error reporting 'level'/complexity should ideally be configurable by the
user (which is something I'd like to see on a global Boost level) in the
sense that the user can choose whether errors should be reported with a
simple std::exception with a single/simple hardcoded message (like "GIL IO
failure") or a with a full std::ios_base::failure filled with detailed and
formatted error messages provided by the backend library or with something
> You do understand that io2 has to have portable source code in order
> to become a boost lib?
Of course...as said earlier the current MSVC specifics should be relatively
easy to cleanup...
> Any idea of avoiding setjmp in a portable manner?
It cannot be avoided using only standard C and C++ because/if the libraries
use setjmp. My code also uses/works with setjmp unless
BOOST_GIL_THROW_THROUGH_C_SUPPORTED is defined (which is automatically
defined for MSVC++)...
>> ps. The libjpeg_image::read( ... ) interface used in the test code is a
>> simple utility static member function provided by the base CRTP class
>> available in all backends automatically) that simply wraps the default
>> ...reader_for<>::...copy_to(...) code...
> I really don't understand.
Sorry, this was just an attempt at a quick explanation as to why and how an
apparently a different interface was used in the different code examples I
gave (at one place I used the full object based ...reader_for<>... interface
and in the mentioned example the 'oneliner' x_image::read() interface was
If you look at the formatted_image<>::read<>() static member function you'll
see what I mean...
> In conclusion, I must say I'm impressed with your io2. I think there
> are a lot of things we both can reuse and add into our projects. Maybe
> a merge could be possible. One of my highest goals is to support as
> many image formats as possible. For instance, a lot of work went into
> the tiff extension since it's to most flexible. Here, all bit image
> types are supported, for instance rgb7_14_12 in a tiled or strip
Well yes, your proposal goes further than just IO with for example the
toolbox extension...although I had an idea, since the beginning of my
dealing with GIL IO, that should probably go to the toolbox extension and
that is support/adapters for image classes from various GUI frameworks that
would make them conform the the GIL image concept. One more thing I forgot
to mention is that I already made some steps in that direction with the GDI+
backend that make a gp_image behave directly like a gil::any_image<>
(without the need to call copy_to...(...) on a gp_image object to read its
The massive work you did with the wrapping of all the image/reading/writing
properties that the individual backends provide can also prove to be
However a (relatively simple) merge does not seem possible because of the
radically different interfaces (as well as implementations)...
-- "What Huxley teaches is that in the age of advanced technology, spiritual devastation is more likely to come from an enemy with a smiling face than from one whose countenance exudes suspicion and hate." Neil Postman
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk