Re: [boost] gil::io "non-review" (was: [gil] Can not open test.jpg)

Subject: Re: [boost] gil::io "non-review" (was: [gil] Can not open test.jpg)
From: Mateusz Loskot (mateusz_at_[hidden])
Date: 2010-03-23 12:04:24

Christian Henning wrote:
>>> I would be interested in a pointer to details of Partial Image
>>> Decoding. Depending on the amount of effort I'll either add it now or
>>> after the review. My time frame is kinda small right now.
>> I think the situation was that you were decoding and discarding the stuff
>> after the area of interest, which was obviously wasteful and fixable.  The
>> more difficult issue was whether it is possible to reduce the work done
>> prior to the area of interest.  I don't have a good enough understanding of
>> libjpeg to know the answer to that.  Presumably there are similar issues
>> with the other formats.
> 
> For now, I consider this an optimization. I someone can show me how to
> do that I would be willing to add.
> 
>> So for my problem of chopping up the TIFF into lots of tiles, I think I have
>> two choices:
>>
>> (1) Decode the whole TIFF into one gil image.  For each tile, create a gil
>> view and create and save a PNG.
> 
> Depending on the size of your TIFF you might need to use the BigTiff
> lib. You probably know that. If you are working in x64 why not load
> the whole thing? RAM is cheap. Unless you have your image is several
> TBs.

I assume Boost.GIL as general purpose library and not specific to 
particular environment, available memory, etc. configurations.
Then, I think it's not an option. I often works with rasters of N GBs 
(not even TB) and swinging large monoliths of memory around is fragile.

What if I have 4GB TIFF image with raster sample size of 1 BIT?
Would Boost.GIL handle it? How? Would it it get 8 times larger in memory 
as each sample is loaded to basic unit of 1 byte, for instance?

I think that the Boost.GIL I/O interface should be designed in a way 
that it doesn't force any limitations and at least allow highly 
optimised implementation of I/O specific to format.

I'd be truly interested in discussing various aspects of handling
large imagery in Boost.GIL.

> Like this one: http://www.aperio.com/bigtiff/#Sample_Images
> 
>> (2) For each tile, partially decode the TIFF into a gil image, and create
>> and save a PNG.
> 
> Tile-based reading is being worked on right now. But as far as I know
> a tiff is either scanline or tile_based. You cannot read scanlines in
> a tile_based image and vice versa. I might be wrong, but that's my
> understanding right now.

Or srtip-based - a chunk of scanlines.

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net