[test]portability of data test cases and build issues
Hi, I've been working on portability fixes for new features in boost.test. They now works fine with gcc 4.6 and 4.7 msvc-10.0 and msvc-11.0. My attempt with clang on linux lead to compiler crashing, but I see it passing in regression on darwin, so I am not sure on the status. I do still have some issues about portability and boost.build: 1. Boost.Build does not handle correctly target foo with sources residing in subdirectory foo under location of Jamfile. I had to rename directory boost/libs/test/test/test_datasets into test_datasets_src, because of that. 2. How do I tell that I want to pass compilation to gcc in toolset gcc-4.7 only? I've tried <toolset>gcc-4.7:<cxxflags>-std=c++11 and <toolset>gcc-4.7.2:<cxxflags>-std=c++11 and either one does not work. <toolset>gcc:<cxxflags>-std=c++11 does work, but I want to pass different options for different versions of gcc. 3. Question: Should I tell *unconditionally* in Boost.Test Jamfile in boost/libs/test/build that I want to pass -std=c++11/-std=c++0x depending on a version? If not, how do I test test_dataset target with this option? Also How do I disable for configurations which does not support it? 4. Issues with regression testers setup? If you look here: http://www.boost.org/development/tests/trunk/developer/test.html You will see that: a) clang linux columns are empty. Why is that? b) msvc-11.0 setup is broken because of: 'cl' is not recognized as an internal or external command, operable program or batch file. c) sun setup is broken because of: /bin/sh: line 2: CC: not found Can we fix these? 5. Portability for other compilers. I am not quite sure which is lowest version of gcc this feature works. Anyone who is interested in making it work with older/different compilers feel free to submit patches. Regards, Gennadiy
Le 04/11/12 10:17, Gennadiy Rozenal a écrit :
Hi,
I've been working on portability fixes for new features in boost.test. They now works fine with gcc 4.6 and 4.7 msvc-10.0 and msvc-11.0. My attempt with clang on linux lead to compiler crashing, but I see it passing in regression on darwin, so I am not sure on the status. I do still have some issues about portability and boost.build:
1. Boost.Build does not handle correctly target foo with sources residing in subdirectory foo under location of Jamfile. I had to rename directory boost/libs/test/test/test_datasets into test_datasets_src, because of that. Hi,
I you to post on the Boost.Build/User/Dev ML when you have issues such that ones. I use to do the following without any issue [ run ../example/a_test.cpp ] Is this what you needed?
2. How do I tell that I want to pass compilation to gcc in toolset gcc-4.7 only? I've tried <toolset>gcc-4.7:<cxxflags>-std=c++11 and <toolset>gcc-4.7.2:<cxxflags>-std=c++11 and either one does not work.
<toolset>gcc:<cxxflags>-std=c++11 does work, but I want to pass different options for different versions of gcc.
Wow, I'm really surprised you have troubles with this. You need to define your toolsets in the user_config.jam file. I have for example using gcc : 4.7.1 : /usr/gcc-4.7.1/bin/g++-4.7 ; using gcc : 4.7.1x : /usr/gcc-4.7.1/bin/g++-4.7 : <cxxflags>-std="c++11" ; Someone has been requesting the addition of a 'std' feature that gives the standard, but I don't think this has been implemented or even retained.
3. Question: Should I tell *unconditionally* in Boost.Test Jamfile in boost/libs/test/build that I want to pass -std=c++11/-std=c++0x depending on a version?
Do you want to provide several versions for the same Boost release? If yes, I think that each version needs a specific Jamfile. Take a look at what Spirit has done in the past. Anyway, No, I don't think this is the place. This is up to the user to compile with whatever compiler she wants. See above. If there is code that should not work if a specific c++11 feature is not available just try to emulate it, report an #error when the user is using this file, or just don't include specific code needing the specific feature.
If not, how do I test test_dataset target with this option? Also How do I disable for configurations which does not support it? I use to have a config.hpp file that gives some macro stating which features are available depending on the compiler/platform. In the c++ files I just check for these macros and I disable the code that would not work.
You can also define a new bjam feature and define the sources you include depending on this feature. This is a little bit more complex, but allows you to be more precise. You can take a look at the Jamfile in thread/build which defines a feature threadapi.
4. Issues with regression testers setup? If you look here: http://www.boost.org/development/tests/trunk/developer/test.html You will see that: a) clang linux columns are empty. Why is that?
There are some of them for Boost.Thread when the program needs to fail. I don't know why.
b) msvc-11.0 setup is broken because of: 'cl' is not recognized as an internal or external command, operable program or batch file. Well this is just another broken tester. Don't worry about it, or just contact the tester directly or iusing the Boost.Testing ML. c) sun setup is broken because of: /bin/sh: line 2: CC: not found This is anew tester that has not yet reached to have any good results. Can we fix these? As I said above, there is a specific list for this purpose. Boost.Testing.
5. Portability for other compilers. I am not quite sure which is lowest version of gcc this feature works. Anyone who is interested in making it work with older/different compilers feel free to submit patches. You have not yet said if the new features/files work with c++98 compilers?
For what I can see on the regression page it seems that test_datasets and test_tools_test are not quite portable. IMHO, I think that you are using without any check C++11 features. There are specific macros in Boost.Config that you can use to check if specific features are available. Some of them are already available in a protable way in Boost, e.g. to avoid ../boost/test/data/monomorphic/fwd.hpp:44:33: error: expected class name struct is_std_collection : std::false_type {}; you can use struct is_std_collection : boost::false_type {}; including <boost/type_traits.hpp> instead of <type_traits> But I guess you know already all that. I suggest you to use a test file for each feature provided by your library. This will let you have a better granularity of what specifically is not working with each compiler/platform. HTH, Vicente
Vicente J. Botet Escriba <vicente.botet <at> wanadoo.fr> writes:
I use to do the following without any issue [ run ../example/a_test.cpp ]
I already do something like this (in my case [ glob test_datasets/*.cpp ], but if target name is thew same as directory name it will fail to build on linux.
2. How do I tell that I want to pass compilation to gcc in toolset gcc-4.7 only? I've tried <toolset>gcc-4.7:<cxxflags>-std=c++11 and <toolset>gcc-4.7.2:<cxxflags>-std=c++11 and either one does not work.
<toolset>gcc:<cxxflags>-std=c++11 does work, but I want to pass different options for different versions of gcc. Wow, I'm really surprised you have troubles with this. You need to define your toolsets in the user_config.jam file. I have for example
using gcc : 4.7.1 : /usr/gcc-4.7.1/bin/g++-4.7 ; using gcc : 4.7.1x : /usr/gcc-4.7.1/bin/g++-4.7 : <cxxflags>-std="c++11" ;
Someone has been requesting the addition of a 'std' feature that gives the standard, but I don't think this has been implemented or even retained.
Why do I need to introduce new toolsets? toolset gcc, gcc-4.7, gcc-4.6 already works fine. Moreover the main point is to make it work on regression tester box, not mine. I need to eliminate failures in regression table. I want to pass std option (different one) to compilers which support it and ignore test on compilers which do not have it. I can probably get the same result by ifdefing the code, but I'd rather have it marked unsupported in regression table instead of fake pass. Gennadiy
Gennadiy Rozenal wrote
Vicente J. Botet Escriba <vicente.botet <at> wanadoo.fr> writes:
I use to do the following without any issue [ run ../example/a_test.cpp ]
I already do something like this (in my case [ glob test_datasets/*.cpp ], but if target name is thew same as directory name it will fail to build on linux.
2. How do I tell that I want to pass compilation to gcc in toolset gcc-4.7 only? I've tried
<toolset> gcc-4.7: <cxxflags> -std=c++11
and
<toolset> gcc-4.7.2: <cxxflags> -std=c++11
and either one does not work.
<toolset> gcc: <cxxflags> -std=c++11 does work, but I want to pass
different options for different versions of gcc. Wow, I'm really surprised you have troubles with this. You need to define your toolsets in the user_config.jam file. I have for example
using gcc : 4.7.1 : /usr/gcc-4.7.1/bin/g++-4.7 ; using gcc : 4.7.1x : /usr/gcc-4.7.1/bin/g++-4.7 : <cxxflags> -std="c++11" ;
Someone has been requesting the addition of a 'std' feature that gives the standard, but I don't think this has been implemented or even retained.
Why do I need to introduce new toolsets? toolset gcc, gcc-4.7, gcc-4.6 already works fine. Moreover the main point is to make it work on regression tester box, not mine. I need to eliminate failures in regression table. I want to pass std option (different one) to compilers which support it and ignore test on compilers which do not have it. I can probably get the same result by ifdefing the code, but I'd rather have it marked unsupported in regression table instead of fake pass.
Have you tried filtering the targets with <toolset>gcc <toolset-gcc:version>4.7.1? Best, Vicente -- View this message in context: http://boost.2283326.n4.nabble.com/test-portability-of-data-test-cases-and-b... Sent from the Boost - Dev mailing list archive at Nabble.com.
Hi Gennadiy, On Monday, 5. November 2012 12:00:16 Gennadiy Rozental wrote:
Vicente J. Botet Escriba <vicente.botet <at> wanadoo.fr> writes:
I use to do the following without any issue
[ run ../example/a_test.cpp ]
I already do something like this (in my case [ glob test_datasets/*.cpp ], but if target name is thew same as directory name it will fail to build on linux.
Interesting. Bug or feature. Can you post a minimal testcase to the Boost.Build ML ?
Wow, I'm really surprised you have troubles with this. You need to define your toolsets in the user_config.jam file. I have for example
using gcc : 4.7.1 : /usr/gcc-4.7.1/bin/g++-4.7 ; using gcc : 4.7.1x : /usr/gcc-4.7.1/bin/g++-4.7 : <cxxflags>-std="c++11" ;
Someone has been requesting the addition of a 'std' feature that gives the standard, but I don't think this has been implemented or even retained.
See comments below. Right now using c++11 is a real mess.
Why do I need to introduce new toolsets? toolset gcc, gcc-4.7, gcc-4.6 already works fine. Moreover the main point is to make it work on regression tester box, not mine. I need to eliminate failures in regression table. I want to pass std option (different one) to compilers which support it and ignore test on compilers which do not have it.
That is definitely the wrong direction. This breaks as soon as the compiled library API depends on the c++ standard.
I can probably get the same result by ifdefing the code, but I'd rather have it marked unsupported in regression table instead of fake pass.
That is the way the go. Unfortunately, there are only two ways to archive this: 1) special toolsets 2) an extra feature for different standards. Option 1) is a no-go as we can't enforce uniform toolset names. Even the dependency on the toolset name is bogus, as people can use _anything_ for version identifiers. Not to mention strange cross-platfrom toolset names... 2) would enable you to add something like <std>c++11 into the requirements of your tests. Those would then be skipped for older compilers. And this should result in an empty entry in the test matrix. Just for the record: I've tried to inject "<build>no" requirements into the dependency chain for builds missing the <cxxflags>-std=gnu++11 (c++11 with gnu extensions used here) but this is to late in the evaluation process. Injecting "<cxxflags>- std=gnu++11" does not work either. Raw experimental patch attached for critique and/or enhancement. I think we need feature std : c++98 c++03 c++11 : propagated ; and then some lofig for at least gcc which sets the appropiate options depending on the real compiler version queried by "gcc -v" (already done for darwin). For msvc, one could do feature.set-default std : c++11 ; for msvc 11.0 and above. Perhaps even msvc-10.0, I have not used this one. The gcc users (especially test runners) could then use "std=c++11" in order to get a clean separate build tree. And library test could detect via requirements. Further comments appreciated. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
Jürgen Hunold <jhunold <at> gmx.eu> writes:
Hi Gennadiy,
On Monday, 5. November 2012 12:00:16 Gennadiy Rozental wrote:
Vicente J. Botet Escriba <vicente.botet <at> wanadoo.fr> writes:
I use to do the following without any issue
[ run ../example/a_test.cpp ]
I already do something like this (in my case [ glob test_datasets/*.cpp ], but if target name is thew same as directory name it will fail to build on linux.
Interesting. Bug or feature. Can you post a minimal testcase to the Boost.Build ML ?
That's easy. Rename test_datasets_src into test_datasets in boost.test testing directory change Jamfile accordingly and try to build it.
I can probably get the same result by ifdefing the code, but I'd rather have it marked unsupported in regression table instead of fake pass.
That is the way the go.
Unfortunately, there are only two ways to archive this: 1) special toolsets 2) an extra feature for different standards.
I am not sure how is this relevant. All I want is to say This is expected to fail if we are building on configuration which does not have this and this feature. Nothing to do with build system. Should I just tweak expected failures file after that and that's it? Gennadiy
Hi Gennadiy, On Tuesday, 6. November 2012 13:36:11 Gennadiy Rozental wrote:
Jürgen Hunold <jhunold <at> gmx.eu> writes:
Interesting. Bug or feature. Can you post a minimal testcase to the Boost.Build ML ?
That's easy. Rename test_datasets_src into test_datasets in boost.test testing directory change Jamfile accordingly and try to build it.
Noted. Thanks.
I can probably get the same result by ifdefing the code, but I'd rather have it marked unsupported in regression table instead of fake pass.
That is the way the go.
Unfortunately, there are only two ways to archive this: 1) special toolsets 2) an extra feature for different standards.
I am not sure how is this relevant. All I want is to say This is expected to fail if we are building on configuration which does not have this and this feature. Nothing to do with build system. Should I just tweak expected failures file after that and that's it?
Well, that is always an option, but should be the last. A clean Boost.Build based solution should work better in the long term. Especially if c++17 et. al come into play. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
Hi Gennadiy, On Sunday, 4. November 2012 09:17:31 Gennadiy Rozenal wrote:
4. Issues with regression testers setup? If you look here: http://www.boost.org/development/tests/trunk/developer/test.html You will see that: a) clang linux columns are empty. Why is that? b) msvc-11.0 setup is broken because of: 'cl' is not recognized as an internal or external command, operable program or batch file. c) sun setup is broken because of: /bin/sh: line 2: CC: not found
Can we fix these?
Issues with specific testers should always go directly to boost-testing@ ML. Cross posted my reply so that testers can read it. Yours, Jürgen -- * Dipl.-Math. Jürgen Hunold ! * voice: ++49 4257 300 ! Fährstraße 1 * fax : ++49 4257 300 ! 31609 Balge/Sebbenhausen * jhunold@gmx.eu ! Germany
participants (5)
-
Gennadiy Rozenal -
Gennadiy Rozental
-
Jürgen Hunold -
Vicente Botet -
Vicente J. Botet Escriba