Re: [boost] Respecting a projects toolchain decisions (was Re: [context] new version - support for Win64)

Subject: Re: [boost] Respecting a projects toolchain decisions (was Re: [context] new version - support for Win64)
From: Dean Michael Berris (mikhailberis_at_[hidden])
Date: 2010-12-27 09:49:57

On Mon, Dec 27, 2010 at 12:17 PM, Rene Rivera <grafikrobot_at_[hidden]> wrote:
> On 12/26/2010 8:31 AM, Dean Michael Berris wrote:
>>
>> The choice of whether the current system is sufficient is not made by
>> some committee or some handful of users that get to decide whether the
>> system is sufficient or otherwise.
>
> Well, yes, and no.. Ultimately the choice is made by the Boost moderators
> and the people ponying up the server and personnel resources. They may be
> some consensus building on the dev list but AFAIR how to serve the Boost
> sources is not a "community" choice.
>

Ah, right. I completely missed that part.

I guess this has to go somewhere so that poor souls like me under the
delusion that Boost is a community project might be pointed to.

>>
>> I think you haven't been looking at -- or are ignoring -- the problems
>> that Boost is already having when it comes to making the development
>> effort more scalable.
>
> I have mentioned in the past that the real problems Boost has, have nothing
> to do with the tools. But instead with the organization and process.
>

Okay, fine.

So I guess I should say that Boost's problem is that the leaders of
the project -- the people mentioned above -- make the choices that
potential contributors don't really want to abide by, and have a
process in place that's not conducive to a faster contribution and/or
more scalable innovation pace.

Since the tools suggest and support a centralized development model,
how do you suggest you put in place an organization and process that
isn't centralized? This is not a rhetorical question, I really want to
know.

>>
>> Well, it's not mythical -- it's there, and the Boost Libraries have
>> pretty much been broken up already. The CMake migration is taking a
>> while and the only reason for that is there aren't enough help going
>> into the CMake effort.
>
> The fact that "there aren't enough people" to make a cmake version possible
> should be an indication that it should be reconsidered. If it's not possible
> for *one* person, working part time, to create and maintain the build
> system, it's already failed.
>

Well, it's not entirely true.

There's one Boost port that uses CMake that is done by one person
part-time. That effort I think is on-going for Debian packaging (or
something to that effect). He's on the Ryppl list too IIRC.

What's not going swimmingly well is the part where Boost libraries get
broken up into individual Git repositories each with their own CMake
build systems for tests and what not, and then having a means of
globbing them together when you pull in the release. It's not entirely
impossible, it's just that this high-level kung fu with CMake to make
this happen "seamlessly" is, well, high-level kung fu -- similar to
the same kind of Kung Fu required to make the same happen with
Boost.Build/Boost.Jam.

Now, there's a fork in the road which the Ryppl folks are wanting to
ponder which direction to go.

One path takes Boost down to making it depend entirely on CMake for
the dependency and discovery process -- something that will require
quite some investment into writing CMake scripts. It's not entirely
rocket science, it's just a lot of work to do. And, given the obvious
resistance by some (or maybe everyone?) on the main Boost developers
mailing list into moving away from Boost.Build/Boost.Jam to CMake as
the build system for Boost, this path might not be the best path to
take if only for the politics of the matter.

The other path takes Ryppl down the full-blown-glue that takes in
these disparate Boost libraries which have been broken up into
multiple Git repositories, adds metadata then uses the individual
CMake files that are in these repositories. This path also deals with
the smart dependency management that is oh so fascinatingly close to
impossible to solve optimally. That's another can of worms that has
its own issues.

I've already said a lot already on this, but really this discussion is
better done in a different thread, which should really be a Ryppl
update that shouldn't really be done by me. :D

>>
>> Well, see, all these things you mention are really tangential to the
>> issue of whether you're using Subversion or Git.
>>
>> Trac can be (and I think, should be) abandoned for something that
>> reflects better the workflow that Boost would want to encourage and
>> that performs better on the machine that is available to it. If the
>> solution is hosted for Boost then I would say it would be better.
>> Migration is always going to be an issue, but it's a mechanical issue
>> in reality. People just have to decide to do it, and then do it.
>
> Well, that last past is your problem! It's not that people have to decide to
> do it.. It's that people have to demonstrate it's possible with actual use
> cases.

Eh?

Are you seriously saying that you haven't seen any other workflow
besides the one that is already in place that will work for Boost?

> For example the Cmake effort tried to make a build system equivalent
> to BBv2, and it did not entirely succeed in having the same features. The
> same applies to any system you might think of replacing.

Okay... So replacing Subversion with Git is going to be an issue
because... Git supports all the things that Subversion supports and is
a distributed version control system to boot? I don't see the logic in
that.

> As a present
> example.. I'm working on replacing the test reporting system of Boost. But
> you don't see me trying to convince anyone a priory to switch to it or to
> devote resources to it. When I'm done with it, I'll show it to the
> community. And if I'm lucky I'll convince enough people that it's worth
> switching, shouldn't be hard in this domain though ;-) And the moderators,
> testers, and others devoting their personal resources will decide to switch.
>

Cool.

Now though, imagine when you're done with replacing the test reporting
system and instead of the community getting in what they think it
should have and also be able to help out in the effort, you slave
through that on your own and in the end other people think what you've
done is insufficient. Because you've hidden this work from the
community, you're not giving the community a chance to help you out
even in a little way by looking at what you're trying to accomplish
and maybe seeing things differently from your vision of the solution.

Maybe it's not your style, but I think this is precisely the reason
why the Boost library development process isn't as community friendly
as the other open source projects are. Because there's this "I'll go
do it my way, and then show it to everyone when I'm done" attitude,
the opportunity for collaboration is lost except in the very end when
it's almost too late to make any changes.

At any rate, I still look forward to a cooler way of seeing the
regression test results. :D

>> The commit hooks can be ported (quite easily if I may say so myself):
>> http://www.kernel.org/pub/software/scm/git/docs/githooks.html if there
>> was really enough momentum towards getting Boost from Subversion to
>> Git. The regression test runners could very well just change the
>> commands they use in the script -- instead of checking out, you'd
>> clone, and instead of updating, you'd pull.
>
>> All these things you mention are artificially made to look "hard"
>> because it's all a matter of migration really. The "hard" part is
>> accepting that there are better solutions out there already.
>
> Awesome... Please show us a working Git+Trac (or equivalent flavor of
> software you are proposing) of Boost will all the history and trac tickets
> ported over, i.e. with a working migration plan, and I'll consider it.
>

That's too easy.

First, I can't port all the Boost tickets yet, but it's a matter of
scripting moving the tickets over from Trac to GitHub issues. If you
don't like how GitHub does issue tracking (which is really simple with
a tagging system akin to Gmail labels) then we can use a JIRA
installation -- it's free for Open Source projects to use, can be
hosted on pretty much any machine, and there are importers for
different issue tracking systems, like Trac.

I recently got my hands on a Redmine installation and that's really
darn cool looking. Migrating the tickets over would be a matter of
writing the scripts to make that happen. If people think it's worth
doing then I might spend an afternoon writing the Python/Ruby scripts
to make that migration happen. Maybe people with more Python/Ruby kung
fu can make that happen faster.

It might just be me though thinking that Boost might benefit from
greater community involvement and encouraging collaborative
development over the current system. If that's the case, then I pretty
much give up on that, maybe try much luck again at convincing people
in the list to maybe give Git and JIRA a chance next year. ;)

-- 
Dean Michael Berris
about.me/deanberris