Re: [boost] gil::io "non-review" (was: [gil] Can not open test.jpg)

Subject: Re: [boost] gil::io "non-review" (was: [gil] Can not open test.jpg)
From: Christian Henning (chhenning_at_[hidden])
Date: 2010-03-23 11:53:08

Hi Phil,

>> I would be interested in a pointer to details of Partial Image
>> Decoding. Depending on the amount of effort I'll either add it now or
>> after the review. My time frame is kinda small right now.
>
> I think the situation was that you were decoding and discarding the stuff
> after the area of interest, which was obviously wasteful and fixable.  The
> more difficult issue was whether it is possible to reduce the work done
> prior to the area of interest.  I don't have a good enough understanding of
> libjpeg to know the answer to that.  Presumably there are similar issues
> with the other formats.

For now, I consider this an optimization. I someone can show me how to
do that I would be willing to add.

>
> So for my problem of chopping up the TIFF into lots of tiles, I think I have
> two choices:
>
> (1) Decode the whole TIFF into one gil image.  For each tile, create a gil
> view and create and save a PNG.

Depending on the size of your TIFF you might need to use the BigTiff
lib. You probably know that. If you are working in x64 why not load
the whole thing? RAM is cheap. Unless you have your image is several
TBs.

Like this one: http://www.aperio.com/bigtiff/#Sample_Images

>
> (2) For each tile, partially decode the TIFF into a gil image, and create
> and save a PNG.

Tile-based reading is being worked on right now. But as far as I know
a tiff is either scanline or tile_based. You cannot read scanlines in
a tile_based image and vice versa. I might be wrong, but that's my
understanding right now.

>
> The first choice uses lots of RAM.  The second choice uses lots of time
> because the partial decoding has to decode and discard all of the content
> that precedes the area of interest.  Is that true?

Yes, I believe you're correct.

>>> I also recall some concerns about how you were handling errors from one
>>> of
>>> the libraries. ?I think you had a setjmp() that was bound to fail, or
>>> something like that.
>>
>> I'm using setjmp in the jpeg and png specific code.
>
> In jpeg/read.hpp you have a namespace-scope static jmp_buf.  I immediately
> worry about thread-safety.  Even in a single-threaded app I can't see how
> this works if you have more than one jpeg file open.  Then you setjmp() in
> the constructor.  The jmp_buf is invalid as soon as that scope returns, so
> if libjpeg calls your error_exit() from after the ctor has finished (e.g. in
> jpeg_read_scanlines()) it will crash.  (Have I grossly misunderstood how
> this works?)

Let's forget multi-thread safety for a bit. During the construction of
the my jpeg_decompress_mgr I tell libjpeg's jpeg_error_mgr to use my
own error handler ( static member function called error_exit() ).
After this, I save the current environment into the marker ( int[16]
buffer ) using the setjmp() function. The marker is a static global
inside the gil::detail namespace. Once an error occurs error_exit() is
called which leads to a longjmp which then jumps back to the setjmp
point. Next we clean up and issue an exception. Hope this makes sense.

In a single-treaded application how would this procedure be a problem?
I mean how can you have more than one jpeg file open?

For multi-threaded usage the above is problematic since the marker can
be overridden. In case the user reads several jpeg files at one time
this can lead to undefined behavior. One solution would be to create a
marker for each read operation. Do you thing that's a valid approach?

Regards,
Christian