Re: [Boost-build] Son of b2 - suggestions

Subject: Re: [Boost-build] Son of b2 - suggestions
From: Robert Ramey (ramey_at_[hidden])
Date: 2016-11-15 16:18:03

On 11/15/16 12:18 AM, Vladimir Prus wrote:
>
> Hi Paul,
>
> On 29-Oct-16 2:57 PM, Paul A. Bristow wrote:
>> A few suggestions as you are getting closer to prototyping.
>
> Thanks for this list! I think the points you made are pretty good, and
> many are good points for all domains; I have even printed out this email
> for future reference.
>
> There are a few points (some implicit) on which I don't quite agree, and
> I wanted to spell it out.
>
> - As far as I'm concerned, we're not looking for a full-blown
> from-scratch rewrite. Getting where we are took some time and effort,
> and redoing everything is unlikely to work. There, I do agree with
> http://www.joelonsoftware.com/articles/fog0000000069.html

a very good reference.

I've worked with both CMake and Boost Build for many years so I'm more 
familiar than I want to be with these systems.  In my view, Boost Build 
has very good functionality.  Once one has things set up, I've found it 
to be very reliable.  Of course everyone knows that my complain has been 
that's it's very hard to get setup. I've never thought that the problem 
with it was the code itself, the language it's coded in (jam). I think 
the problem is the same as many, many software projects - a few guys 
write the code then the rope someone into documenting it.  At that point 
it becomes clear that conceptually its ambiguous.  Of course the 
original programmers don't see this since it always made sense to them.
(I know I constantly rant on this - sorry).

>
> - Both you and Robert bring 'too clever' point. While I understand
> this sentiment, being 'clever' is design goal of Boost.Build.

Ahhhh - clever in itself isn't really the problem.  To me that's an 
implementation issue.  The problem is lack of "conceptual integrity" and 
conceptual "layering".  This would permit one to understand it using 
simpler concepts which can be either composed and/or layered.
I know that this sounds ambiguous but I can tell you I recognize it when 
I see.  FWIW Boost Build is not the only offender here.  C++ itself - 
both the language and it's libraries suffer from this to significant 
extent which makes the language hard to teach, hard to learn and hard to 
explain in a systematic incremental way.  It's evolved into multiple 
layers - C code, Templates, Macros and now constexpr, namespaces.  When 
an idea crosses one of these domains into another, you get a mind 
mending idea like argument dependent lookup, two phase lookup, etc.

> There
> is pretty of tools where you have to write long-winded explicit code
> for everything, while the goal of Boost.Build is to make build
> description for individual modules as simple as possible,

A worthy goal - and I can see the attempt to do this.  I would even see 
that this part is mostly successful.  I can see layering when one thing 
requires another thing.

> and move
> complicated logic into build core or toolset definitions.

Ahhhh - side effects, global variables, lookup, overrides based on 
loading multiple user-config, site-config, etc.

And the concepts of features, properties, etc.  which are propagated and 
influence the build "automatically"

So the agony begins

That does
> not mean that the "clever-vs-explicit" knob is at optimal position,
> and we can move it.

lets' use "implicit" rather than "clever" here.

I think it's an error to see this as a one dimensional setting. Build in 
layering and composition from the start so if I learn one idea, I can 
easily apply it in combination with another idea.

> We can - and should - add more visibility into
> what's going on.

Right - of course that's a whole lot easier said than done. But that's 
why we make the big bucks.

> We've incrementally tweaked documentation into
> having too many disjoint introductions,

This a very common problem - it's everywhere.  Just about all software 
which requires a user manual suffers from this.

> In light of that, maybe the best way forward is to start from scratch
> with documentation.

I think this would be very, very, productive, illuminating and efficient.

> What I would like is to start over, writing an
> 'ideal' documentation for next version of Boost.Build, and fixing and
> expanding and porting implementation to match this ideal documentation.

I think that the result of this would be that one would start with a new 
"top level".  Once get one or two layers deep, you would likely find 
that you've already got most of the code written in the current system 
and you can copy in a lot of with few changes.  Again, I don't think the 
problem is the implementation, it's that it's ambiguous as to what one 
is trying to implement.  Worse, everyone thinks is obvious and no one 
realizes that everyone else is on a different page.

> and I fear we've outgrown docbook as well.

Another very interesting point.  A couple of issues here.

1) horrible syntax. Unreadable by humans.
2) one great idea - separating information content from presentation. 
(note the composition/layering of two simpler ideas).
3) It's a more or less universal format so one can pass around 
files/chunks of content without having to agree on presentation.
4) So I use XMLMind - I believe there is a cheap/free version.  It has 
served me very well. I have section in the Boost Incubator on XMLMind. 
It's out of date as regards to it's usage for documentation of Boost C++ 
libraries but that's not important here.  With XMLMind it's pretty easy 
to use the WYSIWYG editor to edit and view documents and it has good 
facilities for tables, figures, examples, illustrations, etc.  And it's 
pretty simple to use.  (It does take SOME effort to learn though).
5) documentation presentation - book, pdf, html, etc.  is handled by 
xslt - another major hack which is pretty much unintelligible. Another 
layer of agony.  But at least once one has it setup it works pretty well.

So I would love to see proposal for a new Boost Build Language built 
with XMLMind. I would hope that this would about 3 - 10 pages long.  It 
might have a BNF type grammar - but probably not.  It would spell out 
some rules which can be easily checked - especially lookup and 
overrides.  It would have a lot of thought invested in composition and 
it's cousin - dependency.

There is a great need in the world for a solution to this problem.  I'm 
curious to know what you come up with.

Robert Ramey