Re: [boost] [cmake] Minimum viable cmakeification for Boost
Am 20.06.2017 5:08 nachm. schrieb "Niall Douglas via Boost" < boost@lists.boost.org>:
Niall says that using the global BUILD_SHARED_LIBS is a cmake2ism, and that explicit targets are preferred nowadays.
Niall, can you point me to a reference stating this? Preferably some cmake documentation telling me what the right(tm) thing to do is.
Any cmake construct which requires you to write an if() in a non-root-level CMakeLists is a cmake2-ism. There are a very few places in cmake3 remaining where no good alternative to global variables exist. BUILD_SHARED_LIBS is definitely not one of those. I'll ask again: do you have any reference to those statements? The official cmake docs seem to disagree about this specific case. I would even call it non sensical alltogether. Why should the root cmake be responsible for configuring a specific sub module. This is asking for cluttering everything into one single entity in a (almost) totally unrelated place. Why not keep it local to where it should be used. One concrete example: I have networking layer which supports different transports, say tcp, libfabric, MPI and infiniband verbs. The user of my library doesn't really care about which one is used, so ideally, I would give him a compiled version of my library with the internal settings I believe are best for him. In that case, my network layer module would be configured with one of those options. Naturally, I would think, those options are defined within this very sub project. The root project really doesn't care, nor do I want that this implementation detail leaks into the users CMakeLists.txt. Would you consider that bad practice to begin with? How would you solve this? There are tons of other use cases very similar to that. Having everything clobbering up in the root Cmakelists.txt doesn't sound appealing to me. Shared libraries usually have different settings to static libs and again to header only libs. They have different relationships to their dependencies, different usage requirements for consumers. You can use generator expressions to encode those differences, but then you've hard coded them so external cmake no longer can easily override them. You have just made your non-root CMakeLists hard to reuse by cmake you didn't write nor design. Generator expressions are also hard to write and debug, are overwhelmingly confusing to read, and randomly don't work in some places depending on which build system generator you are using. So best avoid all of that complexity - place no complexity at all outside the rootlevel CMakeLists. Declaration only. Place all custom logic solely in rootlevel CMakeLists only.
Any definite source on what a cmake2ism is or is not would be highly appreciated.
As a general observation, I see a lot of statements along the lines of "I state that XYZ is preferable over UVW", it would be nice to have to have background information (pros and cons anyone? what do the cmake authors/docs have to say about this?) on those statements so that everyone can form their own opinion instead of having to choose whom to trust about what's "standard" cmake.
I believe Stephen Kelly is negotiating a book deal on this. It'll be some time before that book lands though. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/ _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost
I'll ask again: do you have any reference to those statements? The official cmake docs seem to disagree about this specific case.
The cmake docs are updated on a git pull request basis by volunteers as and when someone feels they need to be updated. They are, in general, woefully out of date. And as I mentioned here before, Stephen never finished the cmake3 documentation effort he began due to changes in personal circumstance (returning home to Ireland).
Why should the root cmake be responsible for configuring a specific sub module. This is asking for cluttering everything into one single entity in a (almost) totally unrelated place. Why not keep it local to where it should be used.
I'll repeat myself yet again, but to be honest it'll be for the last time as I really don't care about this enough to spend more time repeating myself. I have other stuff to be doing than repeating myself (specifically, Outcome v2). In the cmake I mocked up we are separating concerns: "how to build this library" from "how to configure this library". It is like the intended difference between the SConstruct and SConscript files in scons: one file says what needs to be built and how they relate declaratively, the other file sets flags, optimisation, settings according to the target system etc imperatively. In exactly the same way as with scons, the non-root CMakeLists are the declarative structure describing a build graph. The rootmost CMakeLists are the imperative commands transforming some subset of that build graph into a build according to local conditions. That's the way you ought to be ideally speaking writing your cmake3. I don't know what else I can explain here. As Peter mentioned, this is 101 elementary build system theory here. It's nothing cmake specific, though because of how variable scopes usually begin with CMakeLists.txt, it does strongly encourage the imperative logic to go into the rootmost layer rather than somewhere more logically placed. But it is what it is.
One concrete example: I have networking layer which supports different transports, say tcp, libfabric, MPI and infiniband verbs. The user of my library doesn't really care about which one is used, so ideally, I would give him a compiled version of my library with the internal settings I believe are best for him. In that case, my network layer module would be configured with one of those options. Naturally, I would think, those options are defined within this very sub project. The root project really doesn't care, nor do I want that this implementation detail leaks into the users CMakeLists.txt.
Would you consider that bad practice to begin with? How would you solve this?
It depends on if you ever expect unknown third parties to want to do custom builds of your codebase. If you don't, your approach is fine, and it's easier. If you do, then you need to separate the declarative stuff from the imperative stuff. Then unknown third parties can reuse your declarative stuff, and skip or ignore your imperative stuff as they are doing a custom build.
There are tons of other use cases very similar to that. Having everything clobbering up in the root Cmakelists.txt doesn't sound appealing to me.
You obviously don't place ALL your imperative logic in the very rootmost CMakeLists. There are natural root points where build config markedly changes when you go up a directory level. That's where you locate the imperative logic. If higher level cmake wants to ignore your imperative logic, it skips over the directory with the CMakeLists with the imperative logic and adds the subdirectories of the inner directories directly, thus bringing in only the declarative parts only. So you "reach in" two directory levels to skip the imperative logic. Does this make more sense now? Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On Tue, 2017-06-20 at 22:56 +0100, Niall Douglas via Boost wrote:
I'll ask again: do you have any reference to those statements? The official cmake docs seem to disagree about this specific case.
The cmake docs are updated on a git pull request basis by volunteers as and when someone feels they need to be updated. They are, in general, woefully out of date. And as I mentioned here before, Stephen never finished the cmake3 documentation effort he began due to changes in personal circumstance (returning home to Ireland).
Why should the root cmake be responsible for configuring a specific sub module. This is asking for cluttering everything into one single entity in a (almost) totally unrelated place. Why not keep it local to where it should be used.
I'll repeat myself yet again, but to be honest it'll be for the last time as I really don't care about this enough to spend more time repeating myself. I have other stuff to be doing than repeating myself (specifically, Outcome v2).
We are not asking to repeat yourself, we are asking for references. As the references I have for best practices and modern cmake from both Daniel Pfeifer and Stephen Kelley do not align with what you are saying: https://www.slideshare.net/DanielPfeifer1/cmake-48475415 http://www.steveire.com/WhatWhyHowCMake.pdf https://github.com/boostcon/cppnow_presentations_2017/blob/master/05-19-2017... riday/effective_cmake__daniel_pfeifer__cppnow_05-19-2017.pdf Nor does the boost cmake repo they put together align with what you are saying either: https://github.com/steveire/BoostCMake https://github.com/boost-cmake/boost-cmake
In the cmake I mocked up we are separating concerns: "how to build this library" from "how to configure this library". It is like the intended difference between the SConstruct and SConscript files in scons: one file says what needs to be built and how they relate declaratively, the other file sets flags, optimisation, settings according to the target system etc imperatively.
In exactly the same way as with scons, the non-root CMakeLists are the declarative structure describing a build graph. The rootmost CMakeLists are the imperative commands transforming some subset of that build graph into a build according to local conditions.
Yes, the build scripts should be more declarative, but projects should be standalone as well, and not require a root cmake. Of course, my approach to the boost-cmake is fairly declaritive, there is only one conditional in the top-level cmake, whereas your cmake code is much less delcarative with a lot of conditionals all over.
paul wrote:
* Latest commit cd85034 on Aug 29 2013 * cmake_minimum_required(VERSION 2.8.11 FATAL_ERROR) I'd think that even Stephen Kelly can't produce idiomatic CMake 3.5 while using CMake 2.8. Although who knows.
On Jun 20, 2017, at 6:33 PM, Peter Dimov via Boost <boost@lists.boost.org> wrote:
paul wrote:
* Latest commit cd85034 on Aug 29 2013 * cmake_minimum_required(VERSION 2.8.11 FATAL_ERROR)
I'd think that even Stephen Kelly can't produce idiomatic CMake 3.5 while using CMake 2.8. Although who knows.
This is a lie though because it's using cmake 3 features. It won't build with cmake 2.8 at all.
Paul Fultz II wrote:
On Jun 20, 2017, at 6:33 PM, Peter Dimov via Boost <boost@lists.boost.org> wrote:
paul wrote:
* Latest commit cd85034 on Aug 29 2013 * cmake_minimum_required(VERSION 2.8.11 FATAL_ERROR)
I'd think that even Stephen Kelly can't produce idiomatic CMake 3.5 while using CMake 2.8. Although who knows.
This is a lie though because it's using cmake 3 features. It won't build with cmake 2.8 at all.
Yes, I looked at the code and it's very similar to what is being suggested today: https://github.com/steveire/BoostCMake/blob/master/listsfiles/libs/system/CM... Header-only libraries don't seem to enumerate dependencies though: https://github.com/steveire/BoostCMake/blob/master/listsfiles/libs/smart_ptr...
On Wed, 2017-06-21 at 05:03 +0300, Peter Dimov via Boost wrote:
Paul Fultz II wrote:
On Jun 20, 2017, at 6:33 PM, Peter Dimov via Boost <boost@lists.boost.org> wrote:
paul wrote:
* Latest commit cd85034 on Aug 29 2013 * cmake_minimum_required(VERSION 2.8.11 FATAL_ERROR)
I'd think that even Stephen Kelly can't produce idiomatic CMake 3.5 while using CMake 2.8. Although who knows.
This is a lie though because it's using cmake 3 features. It won't build with cmake 2.8 at all.
Yes, I looked at the code and it's very similar to what is being suggested today:
https://github.com/steveire/BoostCMake/blob/master/listsfiles/libs/system/CM akeLists.txt
Header-only libraries don't seem to enumerate dependencies though:
https://github.com/steveire/BoostCMake/blob/master/listsfiles/libs/smart_ptr /CMakeLists.txt
Yes, but I think its evolved since then. At least that is what is seems from watching Effective Cmake talk at C++Now this year. Utlimately, some authors will want to make their cmake standalone, which will require enumerating the dependencies, but we can't have some cmake scripts that can be used standalone and others that require being invoked with a root cmake. We need the build scripts to be consistent, and we need to support both scenarios.
I'll repeat myself yet again, but to be honest it'll be for the last time as I really don't care about this enough to spend more time repeating myself. I have other stuff to be doing than repeating myself (specifically, Outcome v2).
We are not asking to repeat yourself, we are asking for references. As the references I have for best practices and modern cmake from both Daniel Pfeifer and Stephen Kelley do not align with what you are saying:
This really is the last time I'll repeat myself. After this I won't comment further as you are not listening and you keep repeating the same points after I have already explained why you are mistaken. Those references of yours regard general purpose cmake in a wide general purpose use case. In Boost's situation, we have a highly homogeneous build and configuration, plus lots of end users who do custom builds. So the use case is different. So we employ a more suitable design. Those references of yours are also slightly or very out of date. I do know Stephen personally and I know in detail where he is coming from, and where he was going with cmake3 before he paused his efforts. His own public writings and previous Boost cmake efforts are not up to date with where cmake is now at. But as I've said here previously, don't take my word for it. If Boost decides to commit to some approach, Stephen should be hired to consult on any new design. I think you'll find he greenlights my design over say your design or indeed his own previous designs because mine is closer to latest cmake best practice, and it doesn't do so much extra unnecessary stuff right now, plus it's more more future proof. But that's speculation on my part.
Yes, the build scripts should be more declarative, but projects should be standalone as well, and not require a root cmake. Of course, my approach to the boost-cmake is fairly declaritive, there is only one conditional in the top-level cmake, whereas your cmake code is much less delcarative with a lot of conditionals all over.
Nobody at any stage has said they wouldn't work standalone. Not all declared targets within would work, but some would. It's up to imperative cmake to choose which declarative cmake to use based on whatever logic it chooses. As I've already explained several times now. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 06/20/2017 11:56 PM, Niall Douglas via Boost wrote:
I'll ask again: do you have any reference to those statements? The official cmake docs seem to disagree about this specific case.
The cmake docs are updated on a git pull request basis by volunteers as and when someone feels they need to be updated. They are, in general, woefully out of date. And as I mentioned here before, Stephen never finished the cmake3 documentation effort he began due to changes in personal circumstance (returning home to Ireland).
To be honest, and forgive my skepticism, I find this to be not a good sign of something that is to suppose to dominate the industry: - There is no real documentation available on the "best practices" you advertise, they are told based on anecdotes - Kitware is indeed a company who offers support and consulting and claims to maintain CMake. Not saying that you (or Stephen Kelly) don't know what you are talking about...
Why should the root cmake be responsible for configuring a specific sub module. This is asking for cluttering everything into one single entity in a (almost) totally unrelated place. Why not keep it local to where it should be used.
<snip>
In the cmake I mocked up we are separating concerns: "how to build this library" from "how to configure this library". It is like the intended difference between the SConstruct and SConscript files in scons: one file says what needs to be built and how they relate declaratively, the other file sets flags, optimisation, settings according to the target system etc imperatively.
I can see that and certainly see the benefit of it. I think there is a great chance to further drive the modularization with that approach. I guess the consensus would be (each of those file local to the module being built): - CMakeLists.txt: Describe targets declaratively - Dependencies.cmake: Describe dependencies (PUBLIC, PRIVATE, INTERFACE) - Install.cmake: Define the install logic - Setup.cmake: Define Options etc. and eventually call add_subdirectory to have the CMakeLists.txt being processed This would seperate concerns, the root CMakeLists.txt would then include the corresponding Setup.cmake and Install.cmake files. This would be how the authors envision that the module should be built. Third parties can either follow that approach or derive their own logic. The advantage of the advise to have everything clobbered in the root CMakeLists.txt would be: - it is scalable (Library authors don't need to ask for permission to have some options etc. added inside the root) - the information would be local to the package being built, that is every user can directly observe which different build options are available - Each package can be used standalone or within the super project. The imperative logic is split, and can be reused on a by need basis without copy&pasting from one large, central CMakeLists.txt Does this sound like a suitable approach?
In exactly the same way as with scons, the non-root CMakeLists are the declarative structure describing a build graph. The rootmost CMakeLists are the imperative commands transforming some subset of that build graph into a build according to local conditions.
That's the way you ought to be ideally speaking writing your cmake3. I don't know what else I can explain here. As Peter mentioned, this is 101 elementary build system theory here. It's nothing cmake specific, though because of how variable scopes usually begin with CMakeLists.txt, it does strongly encourage the imperative logic to go into the rootmost layer rather than somewhere more logically placed. But it is what it is.
Interesting that you bring this up ... If you are so much in favor of a declarative build system, we already have multiple. For example plain old make or Boost.Build. With that being said, wouldn't it be more viable to improve Boost.Build with: - Better Documentation - More examples - Tight integration with CMake: * Have Boost.Build generate XXXConfig.cmake and XXXTargets.cmake files * Have a CMake module that drives b2 This would make everyone happy, wouldn't it? People who prefer declarative builds can stick to Boost.Build. Boost.Build would stop being "asocial" to CMake based projects.
One concrete example: I have networking layer which supports different transports, say tcp, libfabric, MPI and infiniband verbs. The user of my library doesn't really care about which one is used, so ideally, I would give him a compiled version of my library with the internal settings I believe are best for him. In that case, my network layer module would be configured with one of those options. Naturally, I would think, those options are defined within this very sub project. The root project really doesn't care, nor do I want that this implementation detail leaks into the users CMakeLists.txt.
Would you consider that bad practice to begin with? How would you solve this?
It depends on if you ever expect unknown third parties to want to do custom builds of your codebase. If you don't, your approach is fine, and it's easier. If you do, then you need to separate the declarative stuff from the imperative stuff. Then unknown third parties can reuse your declarative stuff, and skip or ignore your imperative stuff as they are doing a custom build.
There are tons of other use cases very similar to that. Having everything clobbering up in the root Cmakelists.txt doesn't sound appealing to me.
You obviously don't place ALL your imperative logic in the very rootmost CMakeLists. There are natural root points where build config markedly changes when you go up a directory level. That's where you locate the imperative logic. If higher level cmake wants to ignore your imperative logic, it skips over the directory with the CMakeLists with the imperative logic and adds the subdirectories of the inner directories directly, thus bringing in only the declarative parts only. So you "reach in" two directory levels to skip the imperative logic.
Does this make more sense now?
Thanks for the explanations. It does indeed make more sense now. I hope I could summarize it correctly and provide a nice consensus.
Niall
On 21 June 2017 at 10:01, Thomas Heller via Boost <boost@lists.boost.org> wrote:
On 06/20/2017 11:56 PM, Niall Douglas via Boost wrote:
I'll ask again: do you have any reference to those statements? The official cmake docs seem to disagree about this specific case.
The cmake docs are updated on a git pull request basis by volunteers as and when someone feels they need to be updated. They are, in general, woefully out of date. And as I mentioned here before, Stephen never finished the cmake3 documentation effort he began due to changes in personal circumstance (returning home to Ireland).
To be honest, and forgive my skepticism, I find this to be not a good sign of something that is to suppose to dominate the industry: - There is no real documentation available on the "best practices" you advertise, they are told based on anecdotes
I sympathise. I've been CMake user for almost a decade and the lacking of up to date documented best practices, idiomatic recipes has been quite frustrating since day one. CMake mailing list is actually the only source where one ask for/can find those [1], but then there is no way to verify that received solutions actualk present the canonical CMake way Especially, none of such answers make it to the CMake docs. [1] eg. https://cmake.org/pipermail/cmake/2010-March/035596.html Best regards, -- Mateusz Loskot, http://mateusz.loskot.net
Thomas Heller wrote:
People who prefer declarative builds can stick to Boost.Build.
That's missing the point. Declarative build descriptions are preferable in any build system, and any serious use of one leads inevitably to this same conclusion. Witness, for instance, how Meson intentionally makes its language not Python, even though it's very much Python-looking, and how Bazel also has its own language. This is because if the language is Python, people are encouraged to program in it, instead of declaring targets. Target declarations compose, logic that manipulates global environment doesn't. Now it's true that the declarative approach is sometimes less convenient. Imperatively, you just check whether zlib is installed, #define or not HAVE_ZLIB and build different things based on that. Whereas the alternative is to have a separate zlib-support library target, which injects a .cpp file into the main program via usage-requirements, which registers the zlib support with the main library. But, the upside is that if everyone writes their build descriptions properly, you just link to the appropriate targets in the root build file and everything "just works".
On 06/21/2017 12:58 PM, Peter Dimov via Boost wrote:
Thomas Heller wrote:
People who prefer declarative builds can stick to Boost.Build.
That's missing the point. Declarative build descriptions are preferable in any build system, and any serious use of one leads inevitably to this same conclusion. Witness, for instance, how Meson intentionally makes its language not Python, even though it's very much Python-looking, and how Bazel also has its own language. This is because if the language is Python, people are encouraged to program in it, instead of declaring targets.
Target declarations compose, logic that manipulates global environment doesn't.
I totally get that a declarative build system is superior. The quote was taken out of context (or I didn't express my intent clearly). The thing is that we already have a declarative build system (Boost.Build). People want to interface with CMake though (which is only very slowly catching up with the declarative stuff), and I assume that most CMake scripts out in the wild are *not* in the declarative paradigm, and it takes to get accustomed to it. Yet, there are a lot of people complaining that Boost is "asocial" because it doesn't interface well with the predominant build plan generator out there. Since CMake is moving more and more towards being declarative, the gap seems to close and we could build a bridge. Or just advertise Boost.Build more prominently so it isn't seen as the odd one in the block, but the goto solution.
Now it's true that the declarative approach is sometimes less convenient. Imperatively, you just check whether zlib is installed, #define or not HAVE_ZLIB and build different things based on that. Whereas the alternative is to have a separate zlib-support library target, which injects a .cpp file into the main program via usage-requirements, which registers the zlib support with the main library. But, the upside is that if everyone writes their build descriptions properly, you just link to the appropriate targets in the root build file and everything "just works".
FWIW, this is totally doable within CMake. Consider this: # I guess this if statement has to persist, if someone knows a way # around it, please fix. Could be placed in another Setup.cmake or so to # seperate the build logic from this logic. if (PUMPKIN_WITH_ZLIB) # This sets ZLIB_FOUND to TRUE if found and exports and gives you the # ZLIB::ZLIB target. find_package(ZLIB) else() # Workaround to have the target available even when zlib was not # requested... add_library(ZLIB::ZLIB INTERFACE IMPORTED) endif() # Sets up the zlib dependent target... add_library(pumpkin_zlib EXCLUDE_FROM_ALL ${PUMPKIN_ZLIB_SOURCES}) target_compile_definitions(pumpkin_zlib PUMPKIN_HAVE_ZLIB) target_link_libraries(pumpkin_zlib ZLIB::ZLIB) # setup the main target. add_library(pumpkin ${PUMPKIN_SOURCES}) target_link_libraries(pumpkin PRIVATE $<$<BOOL:${PUMPKIN_WITH_ZLIB}>:pumpkin_zlib>) And yes, the syntax is horrible ;) The generator expressions are the way to achieve proper declarative build logic, I presume. This whole option and find_package thingy is boilerplate which could be put in a common place to give the impression on having no if statements in the module CMakeLists.txt :P
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Thomas Heller wrote:
FWIW, this is totally doable within CMake. Consider this:
# I guess this if statement has to persist, if someone knows a way # around it, please fix. Could be placed in another Setup.cmake or so to # seperate the build logic from this logic. if (PUMPKIN_WITH_ZLIB) # This sets ZLIB_FOUND to TRUE if found and exports and gives you the # ZLIB::ZLIB target. find_package(ZLIB) else() # Workaround to have the target available even when zlib was not # requested... add_library(ZLIB::ZLIB INTERFACE IMPORTED) endif()
# Sets up the zlib dependent target... add_library(pumpkin_zlib EXCLUDE_FROM_ALL ${PUMPKIN_ZLIB_SOURCES}) target_compile_definitions(pumpkin_zlib PUMPKIN_HAVE_ZLIB) target_link_libraries(pumpkin_zlib ZLIB::ZLIB)
# setup the main target. add_library(pumpkin ${PUMPKIN_SOURCES}) target_link_libraries(pumpkin PRIVATE $<$<BOOL:${PUMPKIN_WITH_ZLIB}>:pumpkin_zlib>)
What I had in mind was more like this: add_library(pumpkin ${PUMPKIN_SOURCES}) add_library(pumpkin_zlib ${PUMPKIN_ZLIB_SOURCES}) target_link_libraries(pumpkin_zlib pumpkin) target_link_libraries(pumpkin_zlib ZLIB::ZLIB) with the find_package logic residing in the rootmost CMakeLists, where the project links or does not link to pumpkin_zlib, as appropriate.
On 06/21/2017 02:11 PM, Peter Dimov via Boost wrote:
Thomas Heller wrote:
FWIW, this is totally doable within CMake. Consider this:
# I guess this if statement has to persist, if someone knows a way # around it, please fix. Could be placed in another Setup.cmake or so to # seperate the build logic from this logic. if (PUMPKIN_WITH_ZLIB) # This sets ZLIB_FOUND to TRUE if found and exports and gives you the # ZLIB::ZLIB target. find_package(ZLIB) else() # Workaround to have the target available even when zlib was not # requested... add_library(ZLIB::ZLIB INTERFACE IMPORTED) endif()
# Sets up the zlib dependent target... add_library(pumpkin_zlib EXCLUDE_FROM_ALL ${PUMPKIN_ZLIB_SOURCES}) target_compile_definitions(pumpkin_zlib PUMPKIN_HAVE_ZLIB) target_link_libraries(pumpkin_zlib ZLIB::ZLIB)
# setup the main target. add_library(pumpkin ${PUMPKIN_SOURCES}) target_link_libraries(pumpkin PRIVATE $<$<BOOL:${PUMPKIN_WITH_ZLIB}>:pumpkin_zlib>)
What I had in mind was more like this:
add_library(pumpkin ${PUMPKIN_SOURCES})
add_library(pumpkin_zlib ${PUMPKIN_ZLIB_SOURCES}) target_link_libraries(pumpkin_zlib pumpkin) target_link_libraries(pumpkin_zlib ZLIB::ZLIB)
with the find_package logic residing in the rootmost CMakeLists, where the project links or does not link to pumpkin_zlib, as appropriate.
What would be the rootmost CMakeLists.txt? The one in boostorg in our case? The one that our users would write? I am still having difficulties to see the benefits of having it all in the rootmost file (since I really have problems grasping what is actually means). For reference, I just had a quick glance at the iostreams (which has a dependency on zlib) Jamfile.v2. Two observations from my side: 1) It looks very imperative 2) The zlib dependency is not handled at the Jamroot but in the component directly. That is, it looks very similar to what I proposed earlier. What do I miss?
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Thomas Heller wrote:
For reference, I just had a quick glance at the iostreams (which has a dependency on zlib) Jamfile.v2. Two observations from my side: 1) It looks very imperative 2) The zlib dependency is not handled at the Jamroot but in the component directly.
Yes, which is why your idea that Boost.Build is for declarative people and CMake is for imperative people is wrong. :-)
On Wed, 2017-06-21 at 15:11 +0300, Peter Dimov via Boost wrote:
Thomas Heller wrote:
FWIW, this is totally doable within CMake. Consider this:
# I guess this if statement has to persist, if someone knows a way # around it, please fix. Could be placed in another Setup.cmake or so to # seperate the build logic from this logic. if (PUMPKIN_WITH_ZLIB) # This sets ZLIB_FOUND to TRUE if found and exports and gives you the # ZLIB::ZLIB target. find_package(ZLIB) else() # Workaround to have the target available even when zlib was not # requested... add_library(ZLIB::ZLIB INTERFACE IMPORTED) endif()
# Sets up the zlib dependent target... add_library(pumpkin_zlib EXCLUDE_FROM_ALL ${PUMPKIN_ZLIB_SOURCES}) target_compile_definitions(pumpkin_zlib PUMPKIN_HAVE_ZLIB) target_link_libraries(pumpkin_zlib ZLIB::ZLIB)
# setup the main target. add_library(pumpkin ${PUMPKIN_SOURCES}) target_link_libraries(pumpkin PRIVATE $<$<BOOL:${PUMPKIN_WITH_ZLIB}>:pumpkin_zlib>)
What I had in mind was more like this:
add_library(pumpkin ${PUMPKIN_SOURCES})
add_library(pumpkin_zlib ${PUMPKIN_ZLIB_SOURCES}) target_link_libraries(pumpkin_zlib pumpkin) target_link_libraries(pumpkin_zlib ZLIB::ZLIB)
with the find_package logic residing in the rootmost CMakeLists, where the project links or does not link to pumpkin_zlib, as appropriate.
The find_package shouldn't be in the root directory. It should be in the same project directory as that is what that project needs, and each project should be standalone. This won't prevent the ability to use `add_subdirectory`, but the root cmake is aware of what dependencies it is adding as a subdirectory and what dependencies its relying on being prebuilt, so this is where the adjustments to `find_package` go.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boo st
paul wrote:
On Wed, 2017-06-21 at 15:11 +0300, Peter Dimov via Boost wrote:
What I had in mind was more like this:
add_library(pumpkin ${PUMPKIN_SOURCES})
add_library(pumpkin_zlib ${PUMPKIN_ZLIB_SOURCES}) target_link_libraries(pumpkin_zlib pumpkin) target_link_libraries(pumpkin_zlib ZLIB::ZLIB)
with the find_package logic residing in the rootmost CMakeLists, where the project links or does not link to pumpkin_zlib, as appropriate.
The find_package shouldn't be in the root directory. It should be in the same project directory as that is what that project needs, and each project should be standalone.
In other words, what I wrote, with find_package(ZLIB) prepended? How are version conflicts resolved? If library A says find_package(Boost 1.62.0) and later library B says find_package(Boost 1.64.0), can I find_package(Boost 1.64.0) at root level first to keep them happy?
On Wed, 2017-06-21 at 19:01 +0300, Peter Dimov via Boost wrote:
paul wrote:
On Wed, 2017-06-21 at 15:11 +0300, Peter Dimov via Boost wrote:
What I had in mind was more like this:
add_library(pumpkin ${PUMPKIN_SOURCES})
add_library(pumpkin_zlib ${PUMPKIN_ZLIB_SOURCES}) target_link_libraries(pumpkin_zlib pumpkin) target_link_libraries(pumpkin_zlib ZLIB::ZLIB)
with the find_package logic residing in the rootmost CMakeLists, where the project links or does not link to pumpkin_zlib, as appropriate.
The find_package shouldn't be in the root directory. It should be in the same project directory as that is what that project needs, and each project should be standalone.
In other words, what I wrote, with find_package(ZLIB) prepended?
Yes
How are version conflicts resolved? If library A says find_package(Boost 1.62.0) and later library B says find_package(Boost 1.64.0), can I find_package(Boost 1.64.0) at root level first to keep them happy?
Yes, although, the root doesn't need to say `find_package` at all since its getting boost with `add_subdirectory`, right?
paul wrote:
How are version conflicts resolved? If library A says find_package(Boost 1.62.0) and later library B says find_package(Boost 1.64.0), can I find_package(Boost 1.64.0) at root level first to keep them happy?
Yes, although, the root doesn't need to say `find_package` at all since its getting boost with `add_subdirectory`, right?
No, my question here is about a zlib-like dependency, but since nobody specifies a version for it, I used Boost as a placeholder of a versioned preinstalled dependency.
The thing is that we already have a declarative build system (Boost.Build). People want to interface with CMake though (which is only very slowly catching up with the declarative stuff), and I assume that most CMake scripts out in the wild are *not* in the declarative paradigm, and it takes to get accustomed to it. Yet, there are a lot of people complaining that Boost is "asocial" because it doesn't interface well with the predominant build plan generator out there. Since CMake is moving more and more towards being declarative, the gap seems to close and we could build a bridge. Or just advertise Boost.Build more prominently so it isn't seen as the odd one in the block, but the goto solution.
I still like the automated Jamfile.v2 to cmake generator idea. Like the svn to git transition was done, except you never actually retire the old system. The generated cmake is for packaging and end users only. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On Wed, 2017-06-21 at 13:54 +0200, Thomas Heller via Boost wrote:
On 06/21/2017 12:58 PM, Peter Dimov via Boost wrote:
Thomas Heller wrote:
People who prefer declarative builds can stick to Boost.Build.
That's missing the point. Declarative build descriptions are preferable in any build system, and any serious use of one leads inevitably to this same conclusion. Witness, for instance, how Meson intentionally makes its language not Python, even though it's very much Python-looking, and how Bazel also has its own language. This is because if the language is Python, people are encouraged to program in it, instead of declaring targets.
Target declarations compose, logic that manipulates global environment doesn't.
I totally get that a declarative build system is superior.
This is what I was originally trying to do with BCM: bcm_boost_package(core VERSION 1.61.0 DEPENDS assert config ) This is quite simple and declarative, but then boost is not creating another build-like system that people are not familiar with. So I think we should try to use as much vanilla cmake as possible, but I do think using some custom function can help enumerating the dependencies in other contexts.
The quote was taken out of context (or I didn't express my intent clearly). The thing is that we already have a declarative build system (Boost.Build). People want to interface with CMake though (which is only very slowly catching up with the declarative stuff), and I assume that most CMake scripts out in the wild are *not* in the declarative paradigm, and it takes to get accustomed to it. Yet, there are a lot of people complaining that Boost is "asocial" because it doesn't interface well with the predominant build plan generator out there. Since CMake is moving more and more towards being declarative, the gap seems to close and we could build a bridge. Or just advertise Boost.Build more prominently so it isn't seen as the odd one in the block, but the goto solution.
Now it's true that the declarative approach is sometimes less convenient. Imperatively, you just check whether zlib is installed, #define or not HAVE_ZLIB and build different things based on that. Whereas the alternative is to have a separate zlib-support library target, which injects a .cpp file into the main program via usage-requirements, which registers the zlib support with the main library. But, the upside is that if everyone writes their build descriptions properly, you just link to the appropriate targets in the root build file and everything "just works".
FWIW, this is totally doable within CMake. Consider this:
# I guess this if statement has to persist, if someone knows a way # around it, please fix. Could be placed in another Setup.cmake or so to # seperate the build logic from this logic. if (PUMPKIN_WITH_ZLIB) # This sets ZLIB_FOUND to TRUE if found and exports and gives you the # ZLIB::ZLIB target. find_package(ZLIB) else() # Workaround to have the target available even when zlib was not # requested... add_library(ZLIB::ZLIB INTERFACE IMPORTED) endif()
# Sets up the zlib dependent target... add_library(pumpkin_zlib EXCLUDE_FROM_ALL ${PUMPKIN_ZLIB_SOURCES}) target_compile_definitions(pumpkin_zlib PUMPKIN_HAVE_ZLIB) target_link_libraries(pumpkin_zlib ZLIB::ZLIB)
# setup the main target. add_library(pumpkin ${PUMPKIN_SOURCES}) target_link_libraries(pumpkin PRIVATE $<$<BOOL:${PUMPKIN_WITH_ZLIB}>:pumpkin_zlib>)
And yes, the syntax is horrible ;)
The syntax is horrible. It really should just be: if(PUMPKIN_WITH_ZLIB) target_link_libraries(pumpkin pumpkin_zlib) endif() More people will understand the above than the generator expressions.
The generator expressions are the way to achieve proper declarative build logic, I presume.
The generator expressions are there to handle logic that can't be decided during configuration.
On 21/06/2017 09:01, Thomas Heller via Boost wrote:
On 06/20/2017 11:56 PM, Niall Douglas via Boost wrote:
The cmake docs are updated on a git pull request basis by volunteers as and when someone feels they need to be updated. They are, in general, woefully out of date. And as I mentioned here before, Stephen never finished the cmake3 documentation effort he began due to changes in personal circumstance (returning home to Ireland).
To be honest, and forgive my skepticism, I find this to be not a good sign of something that is to suppose to dominate the industry: - There is no real documentation available on the "best practices" you advertise, they are told based on anecdotes - Kitware is indeed a company who offers support and consulting and claims to maintain CMake.
Not saying that you (or Stephen Kelly) don't know what you are talking about...
Kitware makes no money from cmake. For them it's a cost of sales. They support it only in so far as cmake provides build for their money making products. cmake was always something they just open sourced to enable the clients of their paid products buy in to their custom build process without worrying about getting locked in. Most of the docs and features come from open source contribution as submitted patchsets. They vary enormously in quality, Kitware only provides minimum review before merging. Like with most open source build systems, there is a huge open bug count, many serious bugs unfixed in eight or more years. Nobody has ever claimed cmake is great, or even good. Nobody I know who uses it likes it or thinks it well designed. It's only major claim is that it isn't flawed with a showstopper failing like say scons or make, and it lets average devs use their preferred local IDE. But it has won the great build systems competition. It is the industry standard, nothing else is remotely close in market share. And that brings lots of ecosystem benefits which less popular build tools cannot achieve. There is very good reason that VS2017 can now directly load and work with cmake projects. Every other major IDE can, so Visual Studio had to keep up. That speaks volumes as to where the industry is going.
In the cmake I mocked up we are separating concerns: "how to build this library" from "how to configure this library". It is like the intended difference between the SConstruct and SConscript files in scons: one file says what needs to be built and how they relate declaratively, the other file sets flags, optimisation, settings according to the target system etc imperatively.
I can see that and certainly see the benefit of it. I think there is a great chance to further drive the modularization with that approach. I guess the consensus would be (each of those file local to the module being built): - CMakeLists.txt: Describe targets declaratively - Dependencies.cmake: Describe dependencies (PUBLIC, PRIVATE, INTERFACE) - Install.cmake: Define the install logic - Setup.cmake: Define Options etc. and eventually call add_subdirectory to have the CMakeLists.txt being processed
One very nice approach which I'd personally prefer, but felt it too risque to propose here, is that CMakeLists.txt is always auto generated and library devs cannot customise it. They can customise .cmake files in a cmake directory very similarly to what you just listed, but you are only permitted to use a certain whitelist of cmake commands in certain files, so for example the cmake/targets.cmake file solely allows declarative targets and no logic. Other files very like the above do stuff exactly like you suggest like install, setup etc. This forces library devs to write clean, scalable, reusable cmake. But I know boost-dev doesn't like their code being enforced via automated rules, and I think it would count as "cmake innovation", so I didn't propose that here. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On Wed, 21 Jun 2017 at 04:01 Thomas Heller via Boost <boost@lists.boost.org> wrote:
Interesting that you bring this up ... If you are so much in favor of a declarative build system, we already have multiple. For example plain old make or Boost.Build. With that being said, wouldn't it be more viable to improve Boost.Build with: - Better Documentation - More examples - Tight integration with CMake: * Have Boost.Build generate XXXConfig.cmake and XXXTargets.cmake files * Have a CMake module that drives b2 This would make everyone happy, wouldn't it? People who prefer declarative builds can stick to Boost.Build. Boost.Build would stop being "asocial" to CMake based projects.
For what it's worth, I would vote for this approach. I think it would solve all or most of the end user problems for those that want plug and play with cmake. -- chris
Hi, I've developed package manager based on cmake with declarative format and custom cmake insertions (for imperative purposes). Declarative format is generated into CMakeLists.txt. And now I think that declarative approach does not scale well. With increasing complexity of conditions of build system instructions/settings/configurations, declarative format becomes unmaintainable. Also it involves repetition of common constructions. So you need to be careful with it. Of course in simple cases when a library follows default rules, declarative config for it can be just empty. Some examples with comments: 1. https://cppan.org/projects/pvt.cppan.demo.boost pvt.cppan.demo.boost project page. Most of the libraries has their dependencies set correctly based on #includes. (Still boost deps are the plate of spaghetti with header only deps.) 2. https://cppan.org/pvt.cppan.demo.boost.filesystem/version/1.64.0/specificati... pvt.cppan.demo.boost.filesystem-1.64.0 specification. 3. https://pastebin.com/h4DB9tNj Full declarative config of boost with all libraries. Again, boost deps are not granular and look ugly. 4. https://pastebin.com/XPwW7ZHx LLVM project specification. Exactly on this level I think declarative approach got it drawbacks. There are a lot of similar targets that differ only with their name, so they can be wrapped into functions or macros as it's done in LLVM's cmake configs. ps. I'm not proposing to use cppan in boost. It's only FYI, maybe you find some useful ideas. https://cppan.org/ - package manager page itself. -- Egor Pugin
On 6/21/17 6:47 AM, Chris Glover via Boost wrote:
On Wed, 21 Jun 2017 at 04:01 Thomas Heller via Boost <boost@lists.boost.org> wrote:
Interesting that you bring this up ... If you are so much in favor of a declarative build system, we already have multiple. For example plain old make or Boost.Build. With that being said, wouldn't it be more viable to improve Boost.Build with: - Better Documentation - More examples
Boost build has been hampered by a number of design decisions which have never been revisited. a) It is "too smart". It tries to infer the toolset by looking around the file system. If one has more than one compiler set there is really know way to know which one is going to pick. So then one is forced to go into the workings of boost build to figure out how to over ride this behavior. b) It is driven by "side effects" that is the environment. It searches the local file system starting from the current directory and up the tree for a "special" Jamroot file. So if you move your project to another directory, there is no guarantee that it will work the same. This was seen by the original designers as a feature - it's smart enough to know what you really want. c) It has non-obvious dependencies on other files which one generally doesn't know about: user-config.jam (in the users home directory), project-config.jam (hmmm I'm not sure where this supposed to be), etc. All these are fruit of the idea that - we, the developers of bjam can figure out what you need so the build will be really simple. To fix bjam this idea has to be set aside. All of the linkages to other stuff in the file system should be replaced by either data in the jamfile or if not in the jamfile better yet runtime switches so the command syntax would look like b3 toolset="asfasfa" variant="12321" etc. Which would respond with: build failed, errors found: toolset "asdfasdf" not found variant must be either "debug" or "release" no jamroot specified in either jamfile or command line ... So now the user is sent to look for the specific piece of information he has to add. users who couldn't put stuff in the Jamfile who are tired of putting the stuff on the command line would address this as we always to - a oneline local script - Of course this would be coupled with the manual which would describe this stuff. Actually, the text for this stuff is already in boost build docs. It's just that no one knows what part he needs to read to get it right. So compiling this document and code reorganization to support the above would have to go hand in hand. Bjam is much, much better than is generally appreciated. Its' just that the designers were/are too clever and presume we are also. Robert Ramey
On 6/21/2017 11:23 AM, Robert Ramey via Boost wrote:
On 6/21/17 6:47 AM, Chris Glover via Boost wrote:
On Wed, 21 Jun 2017 at 04:01 Thomas Heller via Boost <boost@lists.boost.org> wrote:
Interesting that you bring this up ... If you are so much in favor of a declarative build system, we already have multiple. For example plain old make or Boost.Build. With that being said, wouldn't it be more viable to improve Boost.Build with: - Better Documentation - More examples
Boost build has been hampered by a number of design decisions which have never been revisited.
a) It is "too smart". It tries to infer the toolset by looking around the file system. If one has more than one compiler set there is really know way to know which one is going to pick. So then one is forced to go into the workings of boost build to figure out how to over ride this behavior.
The solution, of course, is always to pass toolset=xxx to the b2 command line. In my own use of b2 I always do this. snipped...
Robert Ramey
On 6/21/17 10:01 AM, Edward Diener via Boost wrote:
Boost build has been hampered by a number of design decisions which have never been revisited.
a) It is "too smart". It tries to infer the toolset by looking around the file system. If one has more than one compiler set there is really know way to know which one is going to pick. So then one is forced to go into the workings of boost build to figure out how to over ride this behavior.
The solution, of course, is always to pass toolset=xxx to the b2 command line. In my own use of b2 I always do this.
Right and so do I. I also pass the other switches. This re-enforces my point that bjam could my made much better and easier to use with huge changes by altering it's user interface philosophy. If I were required to pass all the switches either on the command line or in the local jamfile itself, then I would only get helpful error messages rather than inexplicable behavior. I believe that evolving the bjam user interface to his phisophy wouldn't be a huge project. Robert Ramey
snipped...
Robert Ramey
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
AmDG On 06/21/2017 09:23 AM, Robert Ramey via Boost wrote:
All these are fruit of the idea that - we, the developers of bjam can figure out what you need so the build will be really simple. To fix bjam this idea has to be set aside. All of the linkages to other stuff in the file system should be replaced by either data in the jamfile or if not in the jamfile better yet runtime switches so the command syntax would look like
b3 toolset="asfasfa" variant="12321" etc.
Which would respond with: build failed, errors found: toolset "asdfasdf" not found
ERROR: rule "asdfasdf.init" unknown in module "toolset". (Okay, I admit that's quite cryptic)
variant must be either "debug" or "release"
error: "12321" is not a known value of feature <variant> error: legal values: "debug" "release" "profile" "debug-python"
no jamroot specified in either jamfile or command line ...
error: no Jamfile in current directory found, and no target references specified. ------- error: Could not find parent for project at '.' error: Did not find Jamfile.jam or Jamroot.jam in any parent directory. Also: 'b3' is not recognized as an internal or external command, operable program or batch file. In Christ, Steven Watanabe
On 21.06.2017 18:01, Peter Dimov via Boost wrote:
Steven Watanabe wrote:
Also: 'b3' is not recognized as an internal or external command, operable program or batch file.
No wonder Boost is dying, we haven't even released b3 yet.
Well, a while ago I started an effort to write a new Python frontend to Boost.Build, naming it 'b3' (wrongly believing that the '2' in 'b2' represented a version). This is now morphed into 'faber' (https://github.com/stefanseefeld/faber, https://stefanseefeld.github.io/faber/doc/html/index.html), which I'm intending to present here soon, as a Python frontend to Boost.Build. Not sure whether that will prevent Boost from dying, though. ;-) Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 6/21/17 2:57 PM, Steven Watanabe via Boost wrote:
AmDG
On 06/21/2017 09:23 AM, Robert Ramey via Boost wrote:
All these are fruit of the idea that - we, the developers of bjam can figure out what you need so the build will be really simple. To fix bjam this idea has to be set aside. All of the linkages to other stuff in the file system should be replaced by either data in the jamfile or if not in the jamfile better yet runtime switches so the command syntax would look like
b3 toolset="asfasfa" variant="12321" etc.
Which would respond with: build failed, errors found: toolset "asdfasdf" not found
ERROR: rule "asdfasdf.init" unknown in module "toolset". (Okay, I admit that's quite cryptic)
LOL
variant must be either "debug" or "release"
error: "12321" is not a known value of feature <variant> error: legal values: "debug" "release" "profile" "debug-python"
OK
no jamroot specified in either jamfile or command line ...
error: no Jamfile in current directory found, and no target references specified. ------- error: Could not find parent for project at '.'
that's pretty cryptic
error: Did not find Jamfile.jam or Jamroot.jam in any parent directory.
Also: 'b3' is not recognized as an internal or external command, operable program or batch file.
Of course, I was refering to the next bjam version Note that my post made the complaint that if you leave the options unspecified, the system just fills them from .... = who knows were. It would be better to not do this and just print the error message: need to specify root with -? need to specify toolset need to specify variant need to specify linkage etc. Of course if the user then specifies the optiones erroneously as I did above, the error messages are sort of OK and he knows what to look into. The current sitation is that he gets some sort of message and WTF? Worse, he changes the environment slightly - such as user login which points to a differnt user-config.jam or some environmental variable he doesn't remember, or removes some file he doesn't remember he gets different behavior and has no clue why. These are effectively hidden global variables affecting the build system. It can take days to track down the source of errors like this. Maybe as an experiment one might consider adding yet another switch - -no_defaults and see how that works out. Robert Ramey
In Christ, Steven Watanabe
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey via Boost Sent: 21 June 2017 16:23 To: Chris Glover via Boost Cc: Robert Ramey Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
On 6/21/17 6:47 AM, Chris Glover via Boost wrote:
On Wed, 21 Jun 2017 at 04:01 Thomas Heller via Boost <boost@lists.boost.org> wrote:
Interesting that you bring this up ... If you are so much in favor of a declarative build system, we already have multiple. For example plain old make or Boost.Build. With that being said, wouldn't it be more viable to improve Boost.Build with: - Better Documentation - More examples
Boost build has been hampered by a number of design decisions which have never been revisited.
<snip>
Of course this would be coupled with the manual which would describe this stuff. Actually, the text for this stuff is already in boost build docs. It's just that no one knows what part he needs to read to get it right. So compiling this document and code reorganization to support the above would have to go hand in hand.
Bjam is much, much better than is generally appreciated. Its' just that the designers were/are too clever and presume we are also.
I've just returned from holiday to plough through all this discussion :-( I wish I hadn't! It feels like there are two groups, one with a spanner who think everything is a nut, and another group with loads of hammers, and think everything is a nail. The hammers are made of plastic, and the spanners are adjustable, but come without a manual. I have been pleading and begging for much more *effective* documentation with far more examples for bjam and the build system for 15 years, and still believe that this is root cause of the current mess. (The daft syntax, incomprehensible error messages, and failure to deal with windows file names with spaces also factors). I use two IDE, VS and CodeBlocks and using Boost header-only is entirely painless. Quite why there is so much opposition, I find difficult to understand. Libraries are much, much more troublesome, and I very much like auto-linking. Sadly, there are a few essential libraries like system, chrono, filesystem and test that work nicer with libraries, so I can see how people need more than Boost header-only. So personally, I am now fairly happy using bjam/b2, after years of swearing and gnashing of teeth. Compared to the creators, I'm oligoneuronic (just like most users), but the real cause was nearly all a massive communication failure. The documentation just does not work. (And, if I am honest, I just wanted not to need to know so much, especially when it didn't Just Work, as promised. The torrent of plaintive emails on Stackoverflow and Boost say that I am not alone. I'd like to live in the blissful ignorance that Daniela's team enjoy.) If a cmake that called b2 to install would make cmake_2.5_rs happy, I think this might be a good first step, but only a feeble band-aid. I fear we are about to get into a similar mess with ill-documented cmake 3.5, especially with an army of 2.5 mindset users. Personally, I don't wish to know about cmake - my brain is full ;-) I'm encouraged by Niall producing a working cmake 3.5 example - it even includes some comments explaining what is happening ! But it's doing the whole of Boost that is the elephant. For example, there are 591 files in boost/math/test and hundreds of tests in each file. The jamfile is not just a simple list of targets to build and run, there are many options and variants, depending on many Boost and several external libraries. So there is a BIG mountain ahead... Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
On 6/22/2017 7:06 AM, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey via Boost Sent: 21 June 2017 16:23 To: Chris Glover via Boost Cc: Robert Ramey Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
On 6/21/17 6:47 AM, Chris Glover via Boost wrote:
On Wed, 21 Jun 2017 at 04:01 Thomas Heller via Boost <boost@lists.boost.org> wrote:
Interesting that you bring this up ... If you are so much in favor of a declarative build system, we already have multiple. For example plain old make or Boost.Build. With that being said, wouldn't it be more viable to improve Boost.Build with: - Better Documentation - More examples
Boost build has been hampered by a number of design decisions which have never been revisited.
<snip>
Of course this would be coupled with the manual which would describe this stuff. Actually, the text for this stuff is already in boost build docs. It's just that no one knows what part he needs to read to get it right. So compiling this document and code reorganization to support the above would have to go hand in hand.
Bjam is much, much better than is generally appreciated. Its' just that the designers were/are too clever and presume we are also.
I've just returned from holiday to plough through all this discussion :-( I wish I hadn't!
It feels like there are two groups, one with a spanner who think everything is a nut, and another group with loads of hammers, and think everything is a nail.
The hammers are made of plastic, and the spanners are adjustable, but come without a manual.
I have been pleading and begging for much more *effective* documentation with far more examples for bjam and the build system for 15 years, and still believe that this is root cause of the current mess.
(The daft syntax, incomprehensible error messages, and failure to deal with windows file names with spaces also factors).
I use two IDE, VS and CodeBlocks and using Boost header-only is entirely painless. Quite why there is so much opposition, I find difficult to understand. Libraries are much, much more troublesome, and I very much like auto-linking. Sadly, there are a few essential libraries like system, chrono, filesystem and test that work nicer with libraries, so I can see how people need more than Boost header-only.
So personally, I am now fairly happy using bjam/b2, after years of swearing and gnashing of teeth. Compared to the creators, I'm oligoneuronic
No definition in my "Webster's Third New International Dictionary", I don't have the OED, don't recall the word in literature, find only 'Oligoneuron' on the web about a genus of flowering plants, so a definition would be appreciated as my own neurons are not firing enough connections to understand it. Perhaps it means an oligarchy of neurons, whatever that is supposed to be.
(just like most users), but the real cause was nearly all a massive communication failure. The documentation just does not work.
I tend to agree but on the basis that the Boost Build doc probably has most everything but not organized in ways that the C++ programmer can easily understand.
(And, if I am honest, I just wanted not to need to know so much, especially when it didn't Just Work, as promised. The torrent of plaintive emails on Stackoverflow and Boost say that I am not alone. I'd like to live in the blissful ignorance that Daniela's team enjoy.)
If a cmake that called b2 to install would make cmake_2.5_rs happy, I think this might be a good first step, but only a feeble band-aid.
I fear we are about to get into a similar mess with ill-documented cmake 3.5, especially with an army of 2.5 mindset users. Personally, I don't wish to know about cmake - my brain is full ;-)
rotfl
I'm encouraged by Niall producing a working cmake 3.5 example - it even includes some comments explaining what is happening !
But it's doing the whole of Boost that is the elephant. For example, there are 591 files in boost/math/test and hundreds of tests in each file. The jamfile is not just a simple list of targets to build and run, there are many options and variants, depending on many Boost and several external libraries. So there is a BIG mountain ahead...
+1 What I am really afraid of is not that Boost end-users do not like CMake, because obviously most programmers appear to love it, but that Boost will just be substituting one build system under its own control, which few really understand, for another build system controlled elsewhere, which more evidently understand but whose usage even more people disagree about. However if we can provide CMake for end-users from our bjam files, without tortuous work, I am all for it as long as I personally don't have to understand it. I find reading the CMake docs, such as they are, much more incomprehensible than the Boost Build docs.
Paul
--- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Edward Diener via Boost Sent: 22 June 2017 17:05 To: boost@lists.boost.org Cc: Edward Diener Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
On 6/22/2017 7:06 AM, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey via Boost Sent: 21 June 2017 16:23 To: Chris Glover via Boost Cc: Robert Ramey Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
<snip>
So personally, I am now fairly happy using bjam/b2, after years of swearing and gnashing of teeth. Compared to the creators, I'm oligoneuronic
No definition in my "Webster's Third New International Dictionary", I don't have the OED, don't recall the word in literature, find only 'Oligoneuron' on the web about a genus of flowering plants, so a definition would be appreciated as my own neurons are not firing enough connections to understand it. Perhaps it means an oligarchy of neurons, whatever that is supposed to be.
OK - I confess - made that up ;-) oligo - few neuronic - neurons (with the benefit of dimly remembered Latin - I was extruded forcibly though O level by my Mother who had a Classics degree - oligarchy - rule by a few people, and chemistry - oligomers - polymers with a few mers) But it seems a useful term of (self-)abuse?
What I am really afraid of is not that Boost end-users do not like CMake, because obviously most programmers appear to love it, but that Boost will just be substituting one build system under its own control, which few really understand, for another build system controlled elsewhere, which more evidently understand but whose usage even more people disagree about.
+1
However if we can provide CMake for end-users from our bjam files, without tortuous work, I am all for it as long as I personally don't have to understand it. I find reading the CMake docs, such as they are, much more incomprehensible than the Boost Build docs.
What should I be reading? https://cmake.org/cmake/help/v3.9/ dumps me in at the deep end and leaves me at "Huh?" should I invest in Mastering CMake Paperback - January 16, 2015 by Ken Martin (Author), Bill Hoffman (Author) $50 Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
Paul A. Bristow wrote:
OK - I confess - made that up ;-)
oligo - few
neuronic - neurons
(with the benefit of dimly remembered Latin - I was extruded forcibly though O level by my Mother who had a Classics degree - oligarchy - rule by a few people, and chemistry - oligomers - polymers with a few mers)
cf. oligophrenia
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Peter Dimov via Boost Sent: 22 June 2017 17:42 To: boost@lists.boost.org Cc: Peter Dimov Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
Paul A. Bristow wrote:
OK - I confess - made that up ;-)
oligo - few
neuronic - neurons
(with the benefit of dimly remembered Latin - I was extruded forcibly though O level by my Mother who had a Classics degree - oligarchy - rule by a few people, and chemistry - oligomers - polymers with a few mers)
cf. oligophrenia
Clearly applicable to me - as I'm told that I can't tell me Greek from my Latin ;-) Paul
On 6/22/2017 12:35 PM, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Edward Diener via Boost Sent: 22 June 2017 17:05 To: boost@lists.boost.org Cc: Edward Diener Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
On 6/22/2017 7:06 AM, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey via Boost Sent: 21 June 2017 16:23 To: Chris Glover via Boost Cc: Robert Ramey Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
<snip>
So personally, I am now fairly happy using bjam/b2, after years of swearing and gnashing of teeth. Compared to the creators, I'm oligoneuronic
No definition in my "Webster's Third New International Dictionary", I don't have the OED, don't recall the word in literature, find only 'Oligoneuron' on the web about a genus of flowering plants, so a definition would be appreciated as my own neurons are not firing enough connections to understand it. Perhaps it means an oligarchy of neurons, whatever that is supposed to be.
OK - I confess - made that up ;-)
I suspected that <g>.
oligo - few
neuronic - neurons
(with the benefit of dimly remembered Latin - I was extruded forcibly though O level by my Mother who had a Classics degree - oligarchy - rule by a few people, and chemistry - oligomers - polymers with a few mers)
Well 'oligo' is Greek, not that I know Greek. I did take Latin also in an American prep school, but I do not remember very much any more. A prep school in the US is a private school, since in Britain I believe the terminology is different.
But it seems a useful term of (self-)abuse?
What I am really afraid of is not that Boost end-users do not like CMake, because obviously most programmers appear to love it, but that Boost will just be substituting one build system under its own control, which few really understand, for another build system controlled elsewhere, which more evidently understand but whose usage even more people disagree about.
+1
However if we can provide CMake for end-users from our bjam files, without tortuous work, I am all for it as long as I personally don't have to understand it. I find reading the CMake docs, such as they are, much more incomprehensible than the Boost Build docs.
What should I be reading?
https://cmake.org/cmake/help/v3.9/ dumps me in at the deep end and leaves me at "Huh?"
I have the exact same reaction.
should I invest in
Mastering CMake Paperback - January 16, 2015 by Ken Martin (Author), Bill Hoffman (Author) $50
Paul
--- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
On Thu, 2017-06-22 at 17:35 +0100, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Edward Diener via Boost Sent: 22 June 2017 17:05 To: boost@lists.boost.org Cc: Edward Diener Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
On 6/22/2017 7:06 AM, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey via Boost Sent: 21 June 2017 16:23 To: Chris Glover via Boost Cc: Robert Ramey Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
<snip>
So personally, I am now fairly happy using bjam/b2, after years of swearing and gnashing of teeth. Compared to the creators, I'm oligoneuronic
No definition in my "Webster's Third New International Dictionary", I don't have the OED, don't recall the word in literature, find only 'Oligoneuron' on the web about a genus of flowering plants, so a definition would be appreciated as my own neurons are not firing enough connections to understand it. Perhaps it means an oligarchy of neurons, whatever that is supposed to be.
OK - I confess - made that up ;-)
oligo - few
neuronic - neurons
(with the benefit of dimly remembered Latin - I was extruded forcibly though O level by my Mother who had a Classics degree - oligarchy - rule by a few people, and chemistry - oligomers - polymers with a few mers)
But it seems a useful term of (self-)abuse?
What I am really afraid of is not that Boost end-users do not like CMake, because obviously most programmers appear to love it, but that Boost will just be substituting one build system under its own control, which few really understand, for another build system controlled elsewhere, which more evidently understand but whose usage even more people disagree about.
+1
However if we can provide CMake for end-users from our bjam files, without tortuous work, I am all for it as long as I personally don't have to understand it. I find reading the CMake docs, such as they are, much more incomprehensible than the Boost Build docs.
What should I be reading?
https://cmake.org/cmake/help/v3.9/ dumps me in at the deep end and leaves me at "Huh?"
should I invest in
Mastering CMake Paperback - January 16, 2015 by Ken Martin (Author), Bill Hoffman (Author) $50
There is this: https://cmake.org/cmake-tutorial/
On 6/22/2017 1:18 PM, paul via Boost wrote:
On Thu, 2017-06-22 at 17:35 +0100, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Edward Diener via Boost Sent: 22 June 2017 17:05 To: boost@lists.boost.org Cc: Edward Diener Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
On 6/22/2017 7:06 AM, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey via Boost Sent: 21 June 2017 16:23 To: Chris Glover via Boost Cc: Robert Ramey Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
<snip>
So personally, I am now fairly happy using bjam/b2, after years of swearing and gnashing of teeth. Compared to the creators, I'm oligoneuronic
No definition in my "Webster's Third New International Dictionary", I don't have the OED, don't recall the word in literature, find only 'Oligoneuron' on the web about a genus of flowering plants, so a definition would be appreciated as my own neurons are not firing enough connections to understand it. Perhaps it means an oligarchy of neurons, whatever that is supposed to be.
OK - I confess - made that up ;-)
oligo - few
neuronic - neurons
(with the benefit of dimly remembered Latin - I was extruded forcibly though O level by my Mother who had a Classics degree - oligarchy - rule by a few people, and chemistry - oligomers - polymers with a few mers)
But it seems a useful term of (self-)abuse?
What I am really afraid of is not that Boost end-users do not like CMake, because obviously most programmers appear to love it, but that Boost will just be substituting one build system under its own control, which few really understand, for another build system controlled elsewhere, which more evidently understand but whose usage even more people disagree about.
+1
However if we can provide CMake for end-users from our bjam files, without tortuous work, I am all for it as long as I personally don't have to understand it. I find reading the CMake docs, such as they are, much more incomprehensible than the Boost Build docs.
What should I be reading?
https://cmake.org/cmake/help/v3.9/ dumps me in at the deep end and leaves me at "Huh?"
should I invest in
Mastering CMake Paperback - January 16, 2015 by Ken Martin (Author), Bill Hoffman (Author) $50
There is this:
I am one of those people who find tutorials a completely hopeless way for me to understand any piece of software. I will buy some book instead, but I really want CMake to actually provide documentation which gives me a thorough overview of how CMake works and explains the parts/concepts of CMake as part of this overview. The style of documentation which so many people employ and so many people seem to love, a tutorial and then a mass of largely undifferentiated detail, makes me nauseous.
On 22.06.2017 12:04, Edward Diener via Boost wrote:
What I am really afraid of is not that Boost end-users do not like CMake, because obviously most programmers appear to love it, but that Boost will just be substituting one build system under its own control, which few really understand, for another build system controlled elsewhere, which more evidently understand but whose usage even more people disagree about.
Exactly. The Fact that we are having this discussion is ample demonstration that a move to "idiomatic" CMake is not a good idea, at least not for the stated reason. And to add that: Being among those who complained about the existing build system, as I'm unable to fix related issues when people file Boost.Python issues that are about its build logic, I'd love to move to a system that *I* understand and can help fix. But CMake is not that, so I'm not supportive of the move.
However if we can provide CMake for end-users from our bjam files, without tortuous work, I am all for it as long as I personally don't have to understand it.
Really ? What about those end users who submit bug reports (to your library) originating from issues they encounter with cmake ? How will you support them ? Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 6/22/2017 1:04 PM, Stefan Seefeld via Boost wrote:
On 22.06.2017 12:04, Edward Diener via Boost wrote:
What I am really afraid of is not that Boost end-users do not like CMake, because obviously most programmers appear to love it, but that Boost will just be substituting one build system under its own control, which few really understand, for another build system controlled elsewhere, which more evidently understand but whose usage even more people disagree about.
Exactly. The Fact that we are having this discussion is ample demonstration that a move to "idiomatic" CMake is not a good idea, at least not for the stated reason.
And to add that: Being among those who complained about the existing build system, as I'm unable to fix related issues when people file Boost.Python issues that are about its build logic, I'd love to move to a system that *I* understand and can help fix. But CMake is not that, so I'm not supportive of the move.
However if we can provide CMake for end-users from our bjam files, without tortuous work, I am all for it as long as I personally don't have to understand it.
Really ? What about those end users who submit bug reports (to your library) originating from issues they encounter with cmake ? How will you support them ?
Punt ? <g> I am assuming that any conversion of my library's bjam code to CMake should "just work", ie. there is somne sort of conversion facility ( or easy recipe ) from bjam to CMake that I can just run each time I make a change to the bjam files. If that is not the case, and I am expected to manually change CMake each time, I probably will not like such CMake support for end-users.
Stefan
On 22.06.2017 13:52, Edward Diener via Boost wrote:
I am assuming that any conversion of my library's bjam code to CMake should "just work",
haha !
ie. there is somne sort of conversion facility ( or easy recipe ) from bjam to CMake that I can just run each time I make a change to the bjam files. If that is not the case, and I am expected to manually change CMake each time, I probably will not like such CMake support for end-users.
What I'm worrying about isn't so much whether any bjam->cmake conversion can be automated or not, but how I will be able to help end-uses who report error messages stemming from cmake. They will be as cryptic to me as the b2 error messages, and require me to run the very command the end user was using to attempt to reproduce and fix the issue. In other words, it would require me to debug the cmake-based build system, not the original one this was created from. Don't fool yourself: whatever build system you hand your users, you better be comfortable with yourself. Stefan -- ...ich hab' noch einen Koffer in Berlin...
On Thu, 2017-06-22 at 12:06 +0100, Paul A. Bristow via Boost wrote:
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Robert Ramey via Boost Sent: 21 June 2017 16:23 To: Chris Glover via Boost Cc: Robert Ramey Subject: Re: [boost] [cmake] Minimum viable cmakeification for Boost
On 6/21/17 6:47 AM, Chris Glover via Boost wrote:
On Wed, 21 Jun 2017 at 04:01 Thomas Heller via Boost <boost@lists.boost. org> wrote:
Interesting that you bring this up ... If you are so much in favor of a declarative build system, we already have multiple. For example plain old make or Boost.Build. With that being said, wouldn't it be more viable to improve Boost.Build with: - Better Documentation - More examples
Boost build has been hampered by a number of design decisions which have never been revisited.
<snip>
Of course this would be coupled with the manual which would describe this stuff. Actually, the text for this stuff is already in boost build docs. It's just that no one knows what part he needs to read to get it right. So compiling this document and code reorganization to support the above would have to go hand in hand.
Bjam is much, much better than is generally appreciated. Its' just that the designers were/are too clever and presume we are also.
I've just returned from holiday to plough through all this discussion :- ( I wish I hadn't!
It feels like there are two groups, one with a spanner who think everything is a nut, and another group with loads of hammers, and think everything is a nail.
The hammers are made of plastic, and the spanners are adjustable, but come without a manual.
I have been pleading and begging for much more *effective* documentation with far more examples for bjam and the build system for 15 years, and still believe that this is root cause of the current mess.
Yes, that is why moving to cmake which has a larger community that already provides lots of these materials. Plus, we can get support from this larger community to help improve boost build infrastructure.
(The daft syntax, incomprehensible error messages, and failure to deal with windows file names with spaces also factors).
And after users spend many hours debugging these issues, they usually abanonded it: https://github.com/boostorg/build/issues/106
I use two IDE, VS and CodeBlocks and using Boost header-only is entirely painless. Quite why there is so much opposition, I find difficult to understand. Libraries are much, much more troublesome, and I very much like auto-linking. Sadly, there are a few essential libraries like system, chrono, filesystem and test that work nicer with libraries, so I can see how people need more than Boost header-only.
So personally, I am now fairly happy using bjam/b2, after years of swearing and gnashing of teeth. Compared to the creators, I'm oligoneuronic (just like most users), but the real cause was nearly all a massive communication failure. The documentation just does not work.
(And, if I am honest, I just wanted not to need to know so much, especially when it didn't Just Work, as promised. The torrent of plaintive emails on Stackoverflow and Boost say that I am not alone. I'd like to live in the blissful ignorance that Daniela's team enjoy.)
If a cmake that called b2 to install would make cmake_2.5_rs happy, I think this might be a good first step, but only a feeble band-aid.
I fear we are about to get into a similar mess with ill-documented cmake 3.5
Daniel Pfeifer's "Effective CMake" does a wonderful job explaining cmake and how to stucture it: https://www.youtube.com/watch?v=bsXLMQ6WgIk I also put together a wiki that goes over the best practices and guidelines: https://github.com/boost-cmake/bcm/wiki/Cmake-best-practices-and-guidelines
, especially with an army of 2.5 mindset users.
What army of cmake 2.5 users? I don't see anyone advocating for cmake 2.8 style of cmake.
Personally, I don't wish to know about cmake - my brain is full ;-)
I'm encouraged by Niall producing a working cmake 3.5 example - it even includes some comments explaining what is happening !
There is also my example, which follows the "effective cmake" structure: https://github.com/pfultz2/boost-cmake-demo It also supports both the `add_subdirectory` and `find_package` workflow, something that Niall's example does not.
But it's doing the whole of Boost that is the elephant. For example, there are 591 files in boost/math/test and hundreds of tests in each file. The jamfile is not just a simple list of targets to build and run, there are many options and variants, depending on many Boost and several external libraries. So there is a BIG mountain ahead...
There are the tests. For some libraries they are simple to add, for others, they are more complicated. There really should be a look at the testing infrastrucutre as well.
participants (13)
-
Chris Glover -
Edward Diener -
Egor Pugin -
Mateusz Loskot -
Niall Douglas -
paul -
Paul A. Bristow -
Paul Fultz II
-
Peter Dimov -
Robert Ramey -
Stefan Seefeld -
Steven Watanabe -
Thomas Heller