Re: [Boost-build] Generating CMake package files using Boost.Build

Subject: Re: [Boost-build] Generating CMake package files using Boost.Build
From: Raffi Enficiaud (raffi.enficiaud_at_[hidden])
Date: 2016-05-11 19:46:58

Le 10/05/16 à 22:06, Stephen Kelly a écrit :
> Raffi Enficiaud wrote:
>
>> Hi Steve,
>>
>> (I am trying to escape from the main boost.devel ML about this cmake
>> topic.)
>
> This refers to:
>
>  http://thread.gmane.org/gmane.comp.lib.boost.devel/266666
>
> and the follow up here:
>
>  http://thread.gmane.org/gmane.comp.lib.boost.devel/266852
>
> Those seem to have run their course now, so time to respond to this.
>
>> I have to say that, even if I am using cmake for more or less 10 years
>> now, I do little know about this package producer/consumer feature. I
>> guess it is rather "new" (in my time scale).
>
> This is a contradiction to what you wrote here:
>
>  http://thread.gmane.org/gmane.comp.lib.boost.devel/266852/focus=266944
>
> Raffi Enficiaud wrote:
>> We have diverging opinions. I am using cmake for more than 10 years now,
>> I do not feel like I am missing some big part of it.
>
> The package config files system is rather a 'big' and 'relevant' part of
> cmake.

I am more interested in building than packaging, so I do not know well 
this part. Also for packaging, there are many different techniques, and 
the ones I used where binary packages (Windows NSIS installer and .deb).

This is not in contradiction with what I said: I do not *feel* like I am 
missing some big part.

Let's go straight to the point: you know better than me the feature you 
propose to integrate to boost. I have never claimed the opposite: you 
have a better "Cmake CV" than I have. I also think that, if you want to 
make your point, you should not blame people that do not understand to 
the full extent all the packaging challenges and functionalities. I hope 
that the discussion will go more to the direction of explanations of 
your ideas, the challenges, the benefits.

I have no clear status on the packaging you propose, I believe I am not 
alone. The "no clear status" part is stated by the special phrasing ("I 
do believe", "If I understand well", etc) I used.

>
>> I do believe that what you propose is a good idea. I have some concerns
>> though. Please correct me if I am wrong.
>>
>> A/ If I understand well, what you propose is for end-users:
>
> Correct.
>
>> they have
>> "something" that allow them to have the binaries on their system
>> somewhere, and they can then integrate Boost parts better to their own
>> build environment.
>
> Yes.

So you agree that we are talking about binaries (see later in the post).

>
>>    - in what extent would this be better than using the regular CMake
>> find(boost) package? This is already doing a good job.
>
> * The config files would be versioned (and likely CI tested) with the boost
> source.
This means that the packaging becomes the responsibility of the boost 
people. But you also claim that this can be automated in bjam, correct?

> * The config files would be easier to maintain as they are generated from
> canonical information. CMake just makes guesses.
Sorry I do not understand: you mean
* you mean the canonical information would be bjam?
* what do you mean by "CMake just makes guesses" ?

> * The FindBoost file needs to be updated with every boost release (see
> _Boost_KNOWN_VERSIONS and the new _DEPENDENCIES variables). That's a burden
> which shouldn't exist at all and wouldn't exist if boost provided config
> files.
There are easy workarounds though, like populating 
"Boost_ADDITIONAL_VERSIONS" for the next 10 years with a for loop.

> * Config package files are the prefered mechanism of the cmake community.
I cannot tell. From my observations, this is not a widely adopted/used 
way of distributing binaries (nor sources) and the only examples I know 
are Qt and OpenCV. But you have another opinion on that: you think that 
this scheme for communicating "build knowledge" from upstream to 
downstream is superior to other schemes. I think one particularly good 
thing is that there is a clear and clean interface between the two 
projects wrt. the build directives. OTOH, heretics like me just 
find_libraries and add_subdirectories.

>>    - how would you handle dependencies? the question is related on both
>> sides: being able to generate the dependencies from boost to cmake
>> package, AND being able to tell the consumer about those dependencies.
>> How does it work in cmake?
>
> The config files contain IMPORTED targets which depend on each other. See
>
>  http://www.cmake.org/cmake/help/v3.4/manual/cmake-buildsystem.7.html
>
> and
>
>  http://www.cmake.org/cmake/help/v3.4/manual/cmake-packages.7.html
>
> Users specify their concrete dependencies and the targets inform CMake about
> their dependencies. CMake computes the rest.
>
>> In your previous example, if we have the dependency "log<-system"
>
> Your arrow (<-) is in the wrong direction. log depends on system.

It depends: DAG terminology is that the arrow points to the sink (log in 
this case). See eg. 
https://books.google.de/books?id=mKkIGIea_BkC&lpg=PA960&ots=VX6DJKT0yx&dq=placement%20of%20the%20arrow%20in%20the%20drawing%20of%20a%20DAG&hl=fr&pg=PA960#v=onepage&q=placement%20of%20the%20arrow%20in%20the%20drawing%20of%20a%20DAG&f=false

Also: 
https://en.wikipedia.org/wiki/Directed_acyclic_graph#Causal_structures

>
> You mean:
>
>  log -> system

I mean log depends on system, system is the source, log is the sink, and 
one of the causal effect of system is log. But let's take your convention.

>
> as in
>
>  http://www.steveire.com/boost/deps-june-14/log.dot
>
> and
>
>  http://www.steveire.com/boost/deps-june-14/log.png
>
>> , can
>> we just have
>>
>> """
>> find_package(BoostLog REQUIRED)
>> """
>>
>> and BoosSystem is automatically sourced?
>
> Yes.
>
>> Would that be interesting or
>> irrelevant?
>
> Yes. That simplicity is an explicit goal.

How would that work if it is distributed in a modular fashion? How does 
it know where to look if eg. I downloaded 2 different modules? Sorry for 
asking apparently very naive questions.

>
>> B/ what would be the form of those packages? For Qt for instance, we
>> have one big monolithic installation
>
> Qt is modular. Qt ships modular tarballs. Each Qt library installs its own
> CMake config package. There are no dependency cycles between Qt libraries.

I do not know. As a end-user of Qt, I can see only a big package from here:
https://www.qt.io/download-open-source/#section-2

To me, it looks like one big monolithic package (that has many 
components ... and that installs everything under some folder). Also, as 
a end-user, components may have many dependency cycles, but I am not 
impacted and I do not need to handle this: I consume the binaries.

Suppose I have my own libA that is dependent on many qt components. The 
way I see all those qt components is like a single node in the DAG of 
dependencies, this node may have many dependency cycles, the consumer is 
not impacted as long as he does not need to propagate those cycles into 
his own build system (case of binary distribution or header only 
libraries). This is what I understand it.

>
> It is definitely not monolithic. Certainly on a naive reading of what you
> wrote, it is just wrong.
>
> Boost is not modular. Boost is monolithic. Boost does not ship modular
> tarballs (nor does boost want to do so). There are many dependency cycles
> between Boost libraries.
>
> However, CMake can handle those cycles. I added code to CMake 3.0
> specifically for the boost case.
>
>> , and the consumer chose the
>> package. Do you see it like that as well: one big tar containing the
>> binaries for a specific platform?
>
> That is up to the Boost community to decide what they want
>
>  http://thread.gmane.org/gmane.comp.lib.boost.devel/254635
>
> Either way, a CMake config package can be created per boost library.
>
>> C/ In case you are distributing the cmake package, how would you
>> propagate the toolchain to the consumer, is this handled by CMake?
>
> The library does not choose the toolchain in CMake. The user chooses the
> toolchain. That's so basic and fundamental to CMake that you must already
> know that, and so clearly I just don't understand your question. Please
> rephrase.

What I understand is that the user choose the toolchain for his own 
stuff/project. This toolchain might not be propagated to the dependent 
package, and it is not clear to me the case of binary package in case of 
incompatible toolchains between the user's project and the package.

Example
* Qt on OSX has a clang64 folder. What happens if I consume Qt on my 
project with another compiler? Should the user recompile Qt? Is cmake 
smart enough to know that Qt compiled with my toolchain is/is not ABI 
compatible with the one that is shipped?
* OpenCV on Windows is shipped with binaries for eg. Visual 2012. What 
happens if I need to link to the static libraries on Visual 2015 x64?
* I compile boost with """toolset=clang cxxflags="-std=c++11 
-stdlib=libc++" linkflags="-stdlib=libc++" """ (or cflags=-fPIC 
cxxflags=-fPIC with GCC 4.6) . How is this information consumed in my 
project?

> In general, I suggest you look into cmake config packages - try using them
> see what they contain, see what the Qt ones contain, maybe the KDE or
> Grantlee ones.

I have no good scenario in mind. Maybe something with a CUDA project... 
How would I propagate the NVIDIA settings?

>
> Thanks,
>
> Steve.
>
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
>