Re: [Boost-users] [boost.test] parsing runtime arguments

Subject: Re: [Boost-users] [boost.test] parsing runtime arguments
From: Olaf Peter (ope-devel_at_[hidden])
Date: 2018-08-27 11:47:08

Hello Raffi,

> What about using the new API for fixtures and accessing argc/argv from 
> the "setup" member function?
> 
> https://www.boost.org/doc/libs/1_68_0/libs/test/doc/html/boost_test/tests_organization/fixtures/models.html 
> 
> 
> If you are constructing a global fixture, the class is instanciated 
> before the framework starts, hence the observed behaviour.

Ok, so the constructor isn't the right place for this, thanks for 
pointing out.

At this time I did some experiments with your suggestions using a global 
fixture.

So, what I finally like to have as behaviour is shown below (actually 
using quick&dirty CLI11, but this detail is information only):

$ bin\testrunner_a.exe  -- --help
My Test Module/Suite Name
Usage: bin\testrunner_a.exe [OPTIONS]

Options:
   -h,--help                   Print this help message and exit
   -s,--source-dir DIR         directory to the test case sources.
   --input-extension TEXT      file extension at read time.
   -d,--destination-dir DIR    directory where to write the test results.
   --output-extension TEXT     file extension at write time.
$ bin\testrunner_b.exe  -- --help
...
$ bin\testrunner_a.exe -- -s ../source/testsuite/
....

with a re-usable (maybe) fixture for the CLI parameter where these 
parameters are accessible later.

What I have is:

- a class implementing the delayed dataset API 
(https://www.boost.org/doc/libs/1_68_0/boost/test/data/monomorphic/delayed.hpp) 
where the test data (input and reference/gold) is loaded at runtime 
given by CLI args (--source-dir and --input-extension). It's obvious 
that --source-dir is required.
- a failure_diagnostic_fixture saving data in case of test failure, 
where the save path/file is also created at runtime from CLI args 
(--destination-dir and --output-extension).

At his time, each of these classes is re-usable for different test 
suites and have their own command line parser member functions and are 
used like:

---8<---
BOOST_DATA_TEST_CASE_F( failure_diagnostic_fixture,
        labels_ok,
     utf_data::make_delayed<dataset_loader>(
        "test_case/labels_ok" ),
     input, expected, test_case_name)
{
     btt::output_test_stream                     	os;

...
     // call failure_diagnostic_fixture's member and save
     // context in case of failure for later inspection
     failure_closure(test_case_name, input, os.str());
}
--->8---

Getting the command line options can be accomplished using a global 
fixture, than using a static member which can be used by the two helper 
mentioned above. But, this prevents to get the test_suite/module name 
for the help message (since isn't exist yet). Also, I didn't found a way 
to stop the further tests after --help requested.

Moving the argc/argv code into the two helper classes doesn't make the 
things easier, since I would have two help messages than.

I don't like thing more complicate than necessary, but I have some 
testuite_xyz which I want to run independent from cmake/ctest; so the
command line arguments are mandatory (even --help :-).

Thanks,
Olaf