Re: [Boost-bugs] [Boost C++ Libraries] #12953: access to master_test_suite().{argc, argv}

Subject: Re: [Boost-bugs] [Boost C++ Libraries] #12953: access to master_test_suite().{argc, argv}
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2018-06-26 12:54:58

#12953: access to master_test_suite().{argc, argv}
-------------------------------+-----------------------------
  Reporter:  ope-devel@…       |      Owner:  Raffi Enficiaud
      Type:  Feature Requests  |     Status:  assigned
 Milestone:  Boost 1.66.0      |  Component:  test
   Version:  Boost 1.63.0      |   Severity:  Problem
Resolution:                    |   Keywords:
-------------------------------+-----------------------------

Comment (by ope-devel@…):

 Hello Raffi,

 thank your for your effort! But I'm not sure if I can adapted theses
 solution 1:1. In the real project I use the dataset as aggregate of test
 files and informations, the 'expected' file and as 3rd field the test case
 file name:

 {{{
 class dataset_loader
 {
 public:
     typedef std::vector<fs::path>                       pathname_type;
     typedef std::vector<std::string>                    data_type;

 public:
     pathname_type const& test_case_name() const { return m_test_case; }
     data_type const& input() const { return m_input; }
     data_type const& expect() const { return m_expected; }

 public:
     dataset_loader(fs::path const& path,
         std::string const& relative_path,   // relative path from
 testsuite
         std::string const& input_extension  // test case file input file
 extension
     );

 private:
     void read_files(fs::path const& path);
     std::string read_file(fs::path const& file_path);

 private:
     fs::path                                            m_prefix_dir;
     pathname_type                                       m_test_case;
     data_type                                           m_input;
     data_type                                           m_expected;
     std::string                                         input_extension;
     std::string
 expected_extension;
 };
 }}}

 The class checks if there is for each 'input' file a complementary
 'expect' file, otherwise the input is discarded. This allows me to skip
 specific tests by only simply renaming the input file.
 Your approach creates in my use case for the test bench 2 (isn't it?)
 objects to load the files. At zip time I get an error if the element
 numbers doesn't match. Hence, I have to take care to rename the 'expect'
 file too - which is a burden and error prone.
 Further, as far I see, no way to get the case name (not right, I see a
 long path from boost.test  IIRC). The failure test output e.g. "/_0" gives
 an enumeration id, with test case name I can specify the concrete file
 name, see notes on bottom.

 Here is what happens under the hood (error handling not shown :-)

 {{{
 dataset_loader::dataset_loader(fs::path const& path,
     std::string const& relative_path,
     std::string const& input_extension_
 )
 : m_prefix_dir{ CMAKE_TESTSUITE_PREFIX_READ_PATH } // => argv [1]
 , input_extension{input_extension_} // => argv[2]
 , expected_extension{ ".expected" } // => argv[3]
 {
     BOOST_TEST_INFO("dataset_loader load test files from " << path);
     fs::path p = m_prefix_dir / fs::path(relative_path) / path;
     read_files(p);
     assert(m_input.size() == m_expected.size()
            && "dataset_loader test vector size mismatch");
 }

 void dataset_loader::read_files(fs::path const& path)
 {
     try {
         if(fs::exists(path) && fs::is_directory(path)) {

             std::vector<fs::path> dir_list { };
             std::copy(fs::directory_iterator(path),
                       fs::directory_iterator(),
                       std::back_inserter(dir_list));

             for(auto const& file : dir_list) {
                 if (fs::extension(file) == input_extension) {
                     m_test_case.emplace_back(
                         file.parent_path().filename() / file.stem()
                     );
                     fs::path const input_file  = file;
                     fs::path const expect_file =
 fs::change_extension(file, expected_extension);
                     m_input.emplace_back(   read_file(input_file ));
                     m_expected.emplace_back(read_file(expect_file));
                 }
             }
         }
         else { /* error message */ }
     }
     catch(std::exception const& e) { /* ... */ }
 }

 std::string dataset_loader::read_file(fs::path const& file_path)
 {
     fs::ifstream file{ file_path };
     std::ostringstream ss{};
     ss << file.rdbuf();
     return ss.str();
 }
 }}}

 where the test case directory structure is:

 {{{
 $ROOT
   /test_case
     /concrete_test
       case_a_000.{input,expected}
       case_a_001.{input,expected}
       case_b_000.{input,expected}
       case_b_001.{input,expected}
     /other_concrete_test
       ....
   .. ~ 150 files each of input,expected
 }}}

 and the test case self:

 {{{
 struct foo_dataset : public testsuite::dataset_loader
 {
     foo_dataset()
     : dataset_loader{ "test_case/parse_identifier",
                       "../parse/syntax",
                       ".pls" }
     { }
 } const foo_dataset;

 BOOST_DATA_TEST_CASE( basic_syntax,
       foo_dataset.input()
     ^ foo_dataset.expect()
     ^ foo_dataset.test_case_name(),
     input, expected, test_case_name)
 {
    ...
 }

 // ... and > 100 BOOST_DATA_TEST_CASE files with different path and
 relative_path
 // constructor args
 }}}

 Imo, using the solution you provided would result into 2 dataset_loader
 instances, loading 'input' and 'expected'. How to get the bullet-proof
 features (filter missing paired test cases at load time and test_case
 name) using it?

 Thank you,
 Olaf

-- 
Ticket URL: <https://svn.boost.org/trac10/ticket/12953#comment:10>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.