Re: [boost] date_time -> serialization (Was: spirtit -> serialization)

Subject: Re: [boost] date_time -> serialization (Was: spirtit -> serialization)
From: Julian Gonggrijp (j.gonggrijp_at_[hidden])
Date: 2014-06-15 17:45:37

I just caught up with this discussion, and based on what I read I 
think the future automated dependency handler should indeed operate on 
a per-header basis. This would mean that the configuration file of a 
module would list all headers in the module, and for each header in 
the module list all headers that it directly depends on. Of course 
still with support for conditional (e.g. tool-dependent) dependency 
annotations. *Optional* dependencies however could then be detected 
automatically, as I'll explain in my inline reply below.

The handler would still download or not download modules entirely, but 
it would base its decisions on detailed per-header information.

Andrey Semashev wrote:
> Paul A. Bristow wrote:
>> Andrey Semashev wrote:
>>> Paul A. Bristow wrote:
>>>> 
>>>> If someone is using Serialisation then isn't there a very high
>>>> probability that they are also using DateTime?
>>> 
>>> I'd say there isn't. I'd say even if a user uses both Serialization and
>> DateTime this
>>> doesn't mean he uses Serialization support in DateTime.
>> 
>> Ah  - I had misunderstood  DateTime did not imply Serialization support in
>> DateTime.
> 
> I'm not sure I understand you. DateTime and its support for
> Serialization live in the same git repository, so when you checkout
> DateTime you get everything. The problem is that currently there is no
> way to separate core DateTime functionality from Serialization
> support.

That there is currently no way does not mean that a tool couldn't 
separate it. See below.

> The proposal was to move these support headers into another
> subdirectory inside the DateTime git submodule. By the build system,
> this would be equivalent to a new submodule. (BTW, we should introduce
> a new term for this;

Perhaps "satellite module"? I agree with some of the other thread 
participants that we should not prefer this kind of construction, 
though.

> otherwise we will get confused all the time). As
> a result you will still checkout everything of DateTime when you
> checkout its git repo, but the dependency graph will have two nodes
> for it - DateTime and DateTime.Serialization. If you don't use
> DateTime.Serialization part, you will not have to checkout
> Serialization as well to get the usable Boost subset.

There is a more elegant way to get the same result, as I'll detail 
below.

> 
>> However, does my general point is that many users are already pulling in a large
>> part of Boost, which may mean that your efforts may not be as useful in practice
>> as in theory?
> 
> As long as people keep checking out the complete Boost tree and use
> monolithic Boost distribution, the effect of our work will be
> relatively small. But our goal is modular Boost, which includes
> modular distribution, as I understand it.

+1

> 
>>>> Sub-sub-modules sound Very Evil to me.
>>> 
>>> Why?
>> 
>> More complicated file layout.
> 
> Agree, this is inconvenient. But as long as there is no better
> solution, this is an acceptable evil.

So I think there is a better solution. Given a module with a typical 
header layout

boost/
        mymodule.hpp
        mymodule/
                core_header_1.hpp
                core_header_2.hpp
                support_other_module_1.hpp
                support_other_module_2.hpp

with e.g. the following per-header dependencies

mymodule.hpp
        mymodule/core_header_1.hpp
        mymodule/core_header_2.hpp
        hismodule/core_header_8.hpp
        hermodule/core_header_4.hpp
mymodule/core_header_1.hpp
        hismodule/core_header_7.hpp
        hismodule/core_header_8.hpp
        hermodule/core_header_5.hpp
mymodule/core_header_2.hpp
        mymodule/core_header_1.hpp
        hermodule/core_header_4.hpp
        hermodule/core_header_5.hpp
mymodule/support_other_module_1.hpp
        mymodule/core_header_1.hpp
        mymodule/core_header_2.hpp
        other_module_1.hpp
mymodule/support_other_module_2.hpp
        mymodule/core_header_1.hpp
        other_module_2/detail/hidden_gem.hpp

the automated handler as described above could safely infer that 
hismodule and hermodule are core dependencies while other_module_1 and 
other_module_2 are optional. In other words, without any changes to 
directory layout, per-header dependency information should be 
sufficient for the handler to distinguish core dependencies from 
optional dependencies. It would also solve the issue of transitivity: 
if yourmodule depends on mymodule, the question whether it also 
depends on other_module_1 is simply answered by tracking whether a 
(core) header of yourmodule has mymodule/support_other_module_1.hpp as 
a (direct or indirect) dependency.

The only (soft) requirement is that modules have a "catch all" core 
header like mymodule.hpp. Even if a library lacks such a header, it 
can still be emulated with an annotation in the configuration file.

HTH,
-Julian