Boost logo

Boost :

From: Mateusz Loskot (mateusz_at_[hidden])
Date: 2019-07-23 19:16:10


FYI, although this is related to my recent thread about "Managing input images
for tests" [1], which is more about test images for core tests; oferring
"PNM as GIL built-in dependency-less human-friendly easy-debuggable formats"
always supported by core IO and useful for testing, issues reporting, etc. :-)
This thread here is more specific to test images for test/extension/io stuff.

I'm slowly preparing for I/O tests overhaul with these major goals:

1. Improve structure of test/extension/io tests, physical and logical
   (i.e. targets, in both, Boost.Build and CMake)
2. Improve distinction between basic, light and quick VS thorough, fat and
   slow tests; make clear what is tested on CI and what is tested locally.
3. Get rid of tweaking tests content with compile-time macros:
   Shortly, most of this terrible complexity displayed by
   the "Compiler Symbols" table [2] must go!
4. Maintain collection of test images for I/O tests.

If you know about any other issues/annoyances related to this, please share.

First we need to decide how to approach hosting of test images.
So, let's consider some feasible options and their pros and cons:

1. Keep test images required for tests inside GIL repo
    + simplifies overall maintenance
    + makes it easier for contributors to run completely tests coverage
    - affects size of repository and Boost distribution
    - requires maintenance of .gitattributes
    - may require use of Git LFS

    Examples: OpenCV,

2. Keep test images in external repository or archive
    + keeps repository and Boost distribution size smaller
    - extra maintenance to keep tests and data in sync
    - extra scripts to download and deploy images for tests on CI and locally

    Examples: (see below)

3. Identify public sets of standard test images vailable online (e.g. [3], [4])
    + keeps repository and Boost distribution size smaller
    - extra scripts to download and deploy images for tests on CI and locally

    Examples: none found

Keep in mind, collection of test images may grow, in files number and disk size.
For example, addressing a GitHub issue may bring in new test image(s) specific to
testing a bug fix. For example, see my PR with tests for PNG tRNS [5] which
addressed issues reported by FreeOrion project.

Have I missed any other options, any pros/cons? Please, complete my list.

Note on JuliaImages, which is an intersting case, which I think is based on
the option 2. The developers of the image processing packages for Julia language
maintain dedicated repository with test images [6]. The repository is used as Julia
package loadable at run-time offering API to find and load image by name.

If we choose the option 2., likely, we will want to achieve something similar, but
implemented as a hybrid semi-automatic solution:

1. User or CI runs script to deploy test images in known location

   libs/gil/test/get-test-images.{sh|ps1} --target-dir libs/gil/test/images/

2. C++ test cases load images from known location using dedicated fixture

   rgb8_image_t i = fixture::load_test_image('lena/lena_gray_16bit.png');

In details, the known default location can be overriden by user for both, the
script and the C++ test cases using dedicated command line argument (also
passed via CMake, if configured, etc.). Things can be tweaked for convenience,
but that is implementation detail, not important at this stage.

Question: what is our preferred way to host test images?

Although it is a bit more work, I am inclined towards the option 2. with
JuliaImages-like approach outlined above.


Best regards,

Mateusz Loskot,
Fingerprint=C081 EA1B 4AFB 7C19 38BA  9C88 928D 7C2A BB2A C1F2

Boost list run by Boost-Gil-Owners