Re: [boost] GIL - Generic Image Library Review - Begins October 5

From: Lubomir Bourdev (lbourdev_at_[hidden])
Date: 2006-10-11 14:52:21

Andy,

Yes, I agree with you we can have a single concept for Colour, and that
its representation in memory (number of bits per channel, ordering of
the channels, etc.) could be models of that concept. At first glance, it
does seem appealing to separate color out. Here is where we disagree. I
believe that:
 - Dealing with color cannot be easily separated out of GIL without
splitting the library in half
 - It doesn't make sense to separate color out, because it will require
moving out of GIL lots of other concepts which are very much related to
images.

Let us consider carefully what this would involve in practice.

First, lets settle on terminology. A color value consists of an ordered
sequence of channel values, along with a color space that provides
interpretation for the channel values. For example, the color Red can be
represented in RGB color space with 8-bit unsigned integral channels as
[255 0 0] (that is, 100% Red, 0% Green, 0% Blue). The same color Red can
be represented in floating point BGR as [0.0f 0.0f 1.0f]. Color space
domains do not overlap completely. For example the same color Red cannot
be accurately represented in CMYK color space, i.e. using a mixture of
Cyan, Magenta, Yellow and Black inks. One common operation we would like
to do is convert between colors of different color spaces.
In GIL the construct that holds the value of a color in a given color
space is called a Pixel.

To support all this, we need:

1. A concept of a color channel
- support for conversion between channels of different representations
- channel traits to define the range of a channel
- channel iterators and reference types, to allow for channels whose
size is less than a byte
- low-level operations such as multiplying two channels, inverting a
channel, etc. These may require performance specializations for each
channel model
- metafunctions to determine if two channels are compatible (i.e. if
there is a lossless conversion between them)

2. A concept of a color space.
- properties of a color space, such as the number of channels, presence
of alpha
- compatibility between color spaces
- ordering of the channels

3. A concept of a Pixel (i.e. the holder of the channels)
- metafunctions determining things like compatibility between pixels
- various pixel-level operations, such as fill_channels,
transform_channels, for_each_channel. Implementation of these that is
fast (i.e. no explicit loop), supports pixels of any arity, supports
heterogeneous pixels, and pairs channels semantically (Red to Red) not
based on their ordering in memory
- metafunctions to get, say, the type of the K-th channel of a pixel.
- types for reference and iterators of pixels, for the same reasons we
have them for channels

4. Support for color conversion
- convertibility properties of an ordered pair of pixels
- color conversion implementations for ordered pairs of pixel types,
possibly with performance specializations
- ability to replace the default color converter, or provide overrides
for some combinations. For example, perform a high quality color
conversion using color profiles

To do fast color conversion some systems may want to perform color
conversion on multiple pixels simultaneously. This necessitates defining
pixel iterators, the associated iterator traits...

GIL core files that deal with exclusively with the above:
- cmyk.hpp
- device_n.hpp
- gray.hpp
- hsb.hpp
- lab.hpp
- rgb.hpp
- rgba.hpp
- channel.hpp
- color_convert.hpp
- pixel.hpp
- pixel_algorithm.hpp
- pixel_iterator.hpp
- pixel_iterator_traits.hpp
- planar_ptr.hpp
- planar_ref.hpp

GIL core files that deal in part with the above:
- gil_concept.hpp
- gil_config.hpp
- metafunctions.hpp
- typedefs.hpp
- gil_all.hpp

Of the 27 files in GIL core, 20 are necessary to support just basic
color conversion. You characterized Color as something that is easy to
separate out of GIL. I hope that this convinces you otherwise.

Now, as for the second statement, whether it makes sense to separate
color out of GIL. In my opinion, certainly not! Yes, there are cases
where people would need just color and color conversion and won't have
to deal with images directly. But there are also cases where people will
need vectors and vector operations and won't need matrices. Does that
mean we should split a linear algebra library into a vector-only and
image-only libraries? Color support is tightly integrated with the rest
of GIL, and this is with good reason: The list above is long not because
dealing with color is complex on a conceptual level, but because of the
vast variability in how color can be represented and this variability
needs to be handled by the image processing library.

You may argue that you need color to define the color of your dialog box
or your 3D mesh, objects seemingly unrelated to images. Ultimately,
however, everything gets rasterized into an image to be displayed on the
screen. That's why it seems natural for the color to be part of an image
library.

If you are still not convinced, look of all the other image libraries.
Does Cairo separate color out? Does Vigra do so? Does Anti-Grain do so?
How about OpenCV?
VXL have a set of eight core libraries to do imaging:
http://paine.wiau.man.ac.uk/pub/doc_vxl/index.html

Why do you think they deal with color inside their "Core Image" library?
http://paine.wiau.man.ac.uk/pub/doc_vxl/core/vil/html/annotated.html

Or do you think that every imaging library developer must have gotten it
wrong?

Lubomir