Re: [boost] [modularization] Modularizing Boost (modularization)

Subject: Re: [boost] [modularization] Modularizing Boost (modularization)
From: Stephen Kelly (steveire_at_[hidden])
Date: 2013-11-03 14:40:09

On 11/03/2013 08:01 PM, Bjørn Roald wrote:
> Yes, it is a messy to deal with other than the simplest submodule
> foreach commands, in particular it get very messy when some dynamic
> input data is needed.
>
> Since submodule foreach is to tricky, lack of submodule awareness in
> some of the git commands is very annoying.  The natural support
> feature for submodule grep would be something like
>
>    git grep --recursive <pattern>
>
> But that is not supported.  There is some talk about it here:
> http://lists-archives.com/git/729379-submodule-aware-grep.html
> but that was 3 years ago, and as far as I can tell this has not been
> added in any way to git, so I think we need to figure ways to get
> around without it and similar support in other some other git commands.

Hi,

I applaud your efforts on working on a solution to the problem, and your
detailed analysis!

>
> I tried the git-grep-submodule wrapper command from one of the
> suggested patches in the above cited discussion, and I combined it
> with a simple alias in my ~/.gitconfig file.
>
> [alias]
>     rgrep = "!f() { git grep $*; git grep-submodule $*;}; f"

It is good that the result from each submodule is not paged individually
(git grep-submodule handles that). However, the above results in two
result sets paged individually. I tried to find a way to fix that but it
seems git_pager can't be used in an alias.

> A problem is that these commands are not standard, so they will create
> problems when referenced in mailing lists etc.  Maybe boost should
> have git related utilities like this shared in some tools repository.

Maybe.

> To strip off modules with no commits, and the count:
>
> commit_count_by_author bdawes@ | grep -v " 0$" | cut -d ' ' -f 1
>
> which is similar to your use case I think.

I think so. Your script also supports my note that lightweight ad hoc
scripts need to be replaced by medium-weight scripts in order to get a
result.

>> Git submodules make horizontal work (ie, the kind of work that I've been
>> doing) harder.
>
> I agree, we need to find ways to make it less so.
>
> The basic work flow become complicated when working horizontally
> across submodules with dependencies between the changes. If somebody
> know how other projects deal with this, it would be good to know.

I'm only familiar with how Qt and KDE dealt with it.

It both cases, the solution was to modularize the code first, instead of
fracturing (not modularizing!) into 100+ interdependent repos first. Qt
split into ~10 repos, and kdelibs is about to be split into about 49 repos.

I realize that is not possible for boost.

> * Merging to master too early in one module may block other merges
>   in that module, so that may not make sense.

I expect this will be the source of some common classes of problems.

> People with special administrative roles need to get involved when
> such commits need to be synchronized across multiple modules. I assume
> we cannot expect the individual authors to respond to pull requests in
> a timely way in these cases.  

This along with the other notes you listed are indeed items which make
horizontal work harder post-split.

> Maybe some other branching and pulling scheme could help. Any thoughts.

I expect having multiple branching/pulling schemes to be used depending
on the usecase would be the source of a new set of problems.

Thanks,

Steve.