Re: [Boost-build] Future development

From: David Abrahams (dave_at_[hidden])
Date: 2007-10-06 02:47:23

on Fri Oct 05 2007, Rene Rivera <grafikrobot-AT-gmail.com> wrote:

> David Abrahams wrote:
>> on Fri Oct 05 2007, "Ray Lambert" <codemonkey-AT-interthingy.net> wrote:
>>> David Abrahams wrote:
>
>>> as to not see how this comment is totally relevant to the
>>> discussion?  From my perspective, it's the primary basis for the
>>> decision and apparently there are others here who agree with my
>>> assessment.
>> 
>> I'm not sure they do.  Nobody else has actually been claiming that
>> make is obsolete.
>
> I won't claim it's obsolete. But I would claim it's archaic. 

No argument there.

> The way it defines the dependency graph and node actualization is
> not designed to take into account the complexity of building modern
> modular software.

I don't understand.  What it does with targets at the low level is
IIUC very similar to what Jam does at the low level.  Jam is not
designed to take into account the complexity of building modern
modular software, so we built a large interpreted structure
(Boost.Build) on top of it.  How is that different?

> What I understand Ray to be arguing is that it's a less than ideal
> choice to base a make system on top of another one as archaic as make
> when there are better choices 

What might be a better choice to serve as a base make system, and by
what criteria are we to decide "better"-ness?  Are you suggesting that
the quality or capabilities of Cmake are somehow limited by the design
of underlying make tools?  If so, could you be specific?

> even if the better choices are less populous.
>
>> Rene apparently objects to the relationship between
>> Cmake and the underlying tools purely on testability grounds, and
>> though I'm not convinced, I'm also not discounting his objection.
>
> You might want to put yourself in the mind of a new Boost user adn
> walk through the process of what it would take them to start using
> Boost.  Essentially, mentally rewrite the getting stared guide and
> at every point think of how you want to present the Boost C++
> Libraries package to users. 

That's actually a good exercise for us to do, and it would certainly
help with the evaluation.

> In each of those think of what it would take the Boost developers to
> provide that presentation.  And at each point think of what users
> would do if they run into problems, and hence how would Boost
> developers respond. I know this is not something that Boost has
> traditionally done or thought about, but it's got to be hashed out
> before deciding to switch. 

I agree.

> As it's the only way you'll know if you are truly taking a step
> forward.

Yep, good idea.  Doug, can we work on this together?

>> I haven't heard any other objections to using an implementation of
>> the valuable Boost.Build abstractions on top of Cmake.
>
> And I'm also raising the support objection, both before and
> immediately above. 

I'm sorry, I was bundling that in with testability.  It's a closely
related issue, but obviously a different one, and you're right to
point that out.

> I've also mentioned some specific features that are requirements for
> Boost, but those could be considered within the testing umbrella.

Sorry, what specific features?

>>> You clearly have an agenda here to merge cmake with BB; 
>> 
>> No, I don't have any such agenda.  We already have, essentially, a
>> reimplementation of major BB concepts atop Cmake.  I am trying to
>> decide whether to support the replacement of BB with that system for
>> Boost's building and testing needs.  My only agenda at this point is
>> to try to convince myself of whether it's a good idea to do that.  I
>> have to admit I'm starting to lean toward it, but I haven't decided.
>
> OK, but why are you leaning toward it? The only reasons you've given so 
> far are that Cmake is not developed/supported by Boost.Build people, 

It has a large and responsive developer community, with a large user
community to support, and peoples' salaries depend on that support.

> and that it supports some "high" level set of features. 

Yeah, a higher level of abstraction than raw bjam.

> The first I still think is a mirage as compared to the people
> supporting BB.

You think support for Cmake is a mirage?  In what sense?

In my experience the support for BB is, in practice, and despite the
best efforts of a few dedicated individuals, insufficient.  Someone
who uses Boost.Build was just telling me today that with Cygwin
support appearing to drop, an express dislike for supporting VC++ on
the part of BB's main maintainer, Darwin builds not working for him,
and failures to target XP64 successfully, the future for his company's
main development platforms looks really bleak.  His stories of lost
days trying to keep BB builds working, of explaining to his coworkers
how to use it, and of support requests met with silence make quite an
impression.

I don't know how support would work out for Doug's implementation, but
a few interested Boost members giving quality support for 8,000 lines
of code looks pretty easy by comparison to the job of supporting all
of BBv2 and bjam combined.

> And the second seems to be balanced by the set of
> basic features Cmake doesn't support. Did I miss something?

I don't know.  What basic features doesn't Cmake support?  It seems
like most of what you asked about had the green checkmark in the Cmake
column.

>> My point was that the parts of make used by Cmake are
>> unsophisticated and by now "just work," and furthermore they work
>> quickly.  They are the parts most beaten on by make's huge user
>> community.  Unlike bjam, we will not have to troubleshoot the
>> native make tools, and there are lots of people outside Boost who
>> care if those tools break in important ways.
>
> I think that's a big assumption on your part. I'm fairly certain we
> will end up troubleshooting the native make tools Cmake is based
> on. And also troubleshooting Cmake itself. For the basic reason that
> we will push the functionality provided by them in the same way we
> push the functionality provided by BB+bjam, and analogous to how we
> push the functionality of C++ compilers. 

Do you think we'd end up troubleshooting Python if we reimplement
Boost.Build in terms of it?  What about the shells and tools that
execute our build commands?  At what point in the long chain of
dependencies between the user's build command and the spinning
platters on the disk that stores your compiled program can we stop
worrying?  Maybe I just don't understand what you mean by
"troubleshooting."

AFAICT the work that Doug did to replicate the important features of
Boost.Build in Cmake put zero pressure on the underlying make tools
and little or no pressure on Cmake itself.  If there were features he
needed Kitware responded -- I know he was talking about wanting a few,
but he also said that he could do without them, and he may have.

> And for every error in the build system we will have to determine
> where the problem lies, just as we do today, except we will be doing
> it for an addition N set of make systems.

No, I don't intend to do that.  If Cmake plus any given build system
doesn't work as advertised, it's a Cmake bug.  Cmake is supposed to be
a portable tool, thus it's Cmake's job to handle whatever
nonuniformity the underlying tools throw at it.

>>> My concern is for Boost.Build
>> 
>> I understand.  Well, I doubt if anything said here would convince
>> Volodya to discard the bjam codebase and adopt something like Doug's
>> 8000-line replacement, and as far as I'm concerned anyone who wants to
>> stick with that system can do so.
>
> Even if you could convince Volodya, you could not convince me to
> abandon BB+bjam. 

Nor would I want to.  It's not relevant to me.

> I personally use it outside of Boost. 

I knew that.

> And it's the reason I got involved in Boost.

Really?  I didn't know that.  I always assumed it was because of the
C++ side of things.

>> However, if Boost adopts a Cmake-based system, BB as we know it
>> will lose its biggest customer and its direct relevance to Boost.
>
> I suspect that's not the case. Boost may be the most public user of BB, 
> but I get the impression that Sandia is a bigger BB customer. 

That may be true.  I could ask them what the result would be if we
switched... but none of that really matters to me in this context.
All I care about in this thread is what would work best for Boost.  If
we choose to drop a bjam-based system I'll have plenty of time to
mourn the abandonment of "my baby" later.

> But perhaps we are using different definitions of "bigger". But as a
> comparison some "real" number seem appropriate:
>
> * BB milestone 11 was downloaded about 37K times. Which would be an
> indication of possible outside users. (Note, I can't find download
> counts for Cmake, so perhaps Kitware can look those up and provide
> them.)

Bill?  That info would be most useful.

> * Freshmeat stats for Boost.Build:
> » Rating:  8.44/10.00 (Rank N/A)
> » Vitality: 0.51% (Rank 304)
> » Popularity: 0.73% (Rank 7781)
>     Record hits: 7,955
>     URL hits: 4,225
>     Subscribers: 15

Interesting, but I don't really know how to interpret those numbers.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com