Re: [boost] Git Modularization Ready for Review

Subject: Re: [boost] Git Modularization Ready for Review
From: Niall Douglas (ndouglas_at_[hidden])
Date: 2013-05-09 11:17:54

> > I'm far more confused as to why? Surely the conventional approach is to
> > convert a SVN repo to GIT.
> 
> Good luck with that, given the way SVN "branches" (really directories)
> have been used in Boost's history.  No automated tool is able to
> determine how to map those things properly onto Git branches.  You can
> see evidence of that in repositories.txt (look at the "build" repository
> declarations for example).

Me personally I'd just chuck away any unmappable historical information.
Chalk it up to a cost of transition. If they really need the history, they
can go fetch it from the read-only SVN repo.

> You act as though we haven't been thinking about how to do this well and
> working on it for years already.  We're quite well-aware how submodules
> work.  The use of submodules by most developers is supposed to be a
> temporary transitional measure, since modularization is designed to
> allow you to work with only the parts of Boost relevant to your project.

You have mistaken my confusion for criticism. I am not criticizing. I am
confused.

> > Also, how is Boost going to manage dependency breakage across multiple
> > submodules?
> 
> What is "dependency breakage?"

Generally speaking Git submodules are for loosely coupled relations because
of how Git implements submodules. Some of Boost's constituent libraries do
meet that definition (BPL, BGL, Math). Some do not, as they are too
essential because so many other libraries depend on them (anything likely to
enter the standard with TR2, the MPL, the PPL etc).

I fear that if modularization is taken to its logical extreme, you could see
submodules get out of step with other submodules. You may of course have
already anticipated this and have implemented something I haven't realized.
As I said, I am confused.

> > Are you planning to keep headers in one monolithic repo
> 
> No
> 
> > but keep anything resulting in a DLL/SO in a submodule? Or are you
> > planning to break out the headers into submodules too?
> 
> Yes; it's already done, modulo some expected adjustments due to the
> review I am requesting.

One solution is to require all findable headers to live inside the main
Boost repo [1], and only implementation to reside in submodules. That keeps
minds focused on dependency management.

[1]: By findable I mean that when Boost library users do #include
<boost/whatever> they get the main Boost repo version, not the submodule
version. I absolutely would expect an automated tool to pull headers from
submodules, check them for ABI breakage and push them into the main repo. My
point is that some sanity check ought to be there.

> > I can see it becoming quite easy for submodules or branches or forks
> > thereof to get out of step with one another.
> 
> That's a *good* thing.  We're trying to decouple.  Most libraries will
> be developed against the previous release of their dependencies.
> 
> > I can see the present submodule heavy approach introducing a lot of
> > anti-Git workflow because those submodules inside the boostorg
> > organization are treated as special compared to other submodules.
> 
> I don't have any idea what you mean here

Git generally has you clone the authoritative master repo, you create a
branch, make your changes. Once you're finished, you fetch latest master,
merge into your branch until it's up to date, then generate a patch or pull
request from your branch to authoritative master. Whoever gatekeeps that
repo does the merge.

Where I'm getting confused - again - is that it would seem that the
submodule repos inside the Boost org section on github are the ones to be
regression tested and used to assemble releases. So, instead of treating all
git repos as one-among-many as say Linux would do, we've got a special
"high" gold candidate set of git repos and then all other git repos. This
appears to add an extra layer of syncing and merging, because surely Boost
admin has the power to modify any of the high gc set of repos.

I'm not saying that this choice isn't for very good reasons. It's just that
from my experience a submodule heavy git repo with lots of tightly coupled
submodules becomes a royal PITA for end users because you get all sorts of
weird obtuse breakages for apparently no reason at the time. Maybe of course
it's just me, and I have some mental block others don't have.

And I absolutely agree that other projects such as KDE have adopted your
approach and it seems to work for them. However Boost is not KDE. In my
mental model of the world, Boost more resembles the Linux kernel in terms of
interdependencies and size. KDE has a vastly more stable intra-module ABI
and much more loosely coupled submodules. This interpretation may be a
mistake of mine.

Niall

---
Opinions expressed here are my own and do not necessarily represent those of
BlackBerry Inc.