[boost] What would make tool authors happier..

Subject: [boost] What would make tool authors happier..
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2015-06-02 14:08:05

Warning.. Some of this may sound contentious at times.. But that's life ;-)

There are a number of disparate discussions going on at the moment that can
be summarized as:

* A tool author suggesting a change (to structure, or process, or something
else) that would make writing tools easier and less error prone.

* Lots of responses about how such a change would be disruptive to current
practices.

* Some responses about how such a change is beneficial to the future of
Boost, its growth and possible real modular future.

* Some more responses about how the tool authors should just account for
all the special cases of the current practices and still bring about the
vaunted modular future.

So I want to summarize some key changes, as I understand them, that I think
will make movement towards a better tooling and development future less
painful for the minuscule minority of the people who write the tools that
aim to bring that future into existence. First the key aspects of the
changes are:

  1. Normalized and regular directory structure of libraries, tools, and
root.
  2. Minimal non-library content and structure.
  3. Stricter rules and guidelines for library authors to follow regarding
the needs of the infrastructure tools.

Some of the specific changes some of us have mentioned that touch on the
above aspects are..

* Making the combined library headers go into boost-root/include. See
separate post from Peter Dimov about this.

This touches on #1 as it makes for a user and tool expected location. That
is, users are accustomed to looking for a top level include. And having
such a location would reduce the documentation and instruction needed to
point them away from their intuitive behavior. External tools also expect
to find such a directory for a library, which is what a monolithic Boost
looks like to tools. Hence it would make it easier to use, incorporate,
author, etc tools.

* What I've mentioned as "flattening" of libraries. Which touches on #1 and
#3 above. This has multiple parts/meanings so let me explain each one
separately.

First, it means is banning of git sub-modules outside of directly under
boost-root/libs and boost-root/tools. Dealing with git sub-modules is
difficult enough even when it's just a handful of sub-modules. But dealing
with the 130 sub-modules we have now is an insanity inducing experience
when there needs to be handling for special locations. Currently the only
culprit of this is "libs/numeric". And I understand that it was done this
way because that was the old structure converted to the modular git form.
But I think it's time to truly abandon the old structures as they are
standing in the way of better tools and user experience. As an example of
user experience, I once tried to set up my own subset of Boost as a git
repo that had all the libraries and tools needed for the documentation
toolchain. It was excruciating to manually discover what all those
libraries where particularly because of no initially knowing about this
aspect of our current sub-modules.

Second, it means formalizing and regulating the top level structure of
libraries. For the longest time we've had an accepted top level structure.
Unfortunately library authors have added to that top level structure. For
example to manage "sub-libraries" or "sub-parts" of their library... Which
is understandable. But it makes life more difficult for the tools that rely
on the structure assertions. For example currently the testing scripts rely
on people updating a single listing file at "boost-root/status/Jamfile.v2".
When in an ideal world the test tools would be able to automatically
discover that information. Practically it means that currently that Jamfile
lists 127 test dirs. But a cursory discovery of test dirs goes up to a
possible 197. As far as following the top-level library structure.. There
are currently 279 files and directories at the library top level that are
not in the accepted set (and I'm already excluding things like readmes and
changelogs, even though changelogs should be in docs).

But do note.. That I'm not suggesting that we immediately ban the old
structure. But that we start discussing what are the needs of library
authors and that we come up with a consistent and enforced structure that
can relied upon by users and tools.

Some changes that may be the first time I bring them up..

* Remove as much as possible from boost-root. This is mostly #2 above.

First, I would like to remove the "boost-root/more" directory. Stuff in it
are docs of various kinds and should be placed in "boost-root/doc", in the
website, in tool docs, or in library docs. While doing that I'm sure we
will likely also rewrite/clean-up those documents to better reflect the
present.

Second, I would like to move as many of the root build files to a
"boost-root/build" directory. Ideally they would all be moved but it may
not be possible, and practical, to do so. This would mirror the general
top-level structure and hence make it a bit more intuitive for new authors
and users to find.

Third, I would like to clean up the various CSS and HTML sources at the
root to either a smaller number (do we really need index.htm *and*
index.html?), or to the doc directory. I know.. This isn't exactly in the
vein of what makes it easier for tool developers. But it does make it
easier for users to initially navigate (because there's less noise for them
to look at).

Last, I would like to re/move the "boost-root/status" directory. Two
options I'm considering are moving to be "boost-root/test" to match the
usual name for testing scripts. Or removing it and replacing it logic in
the regression tools that is equivalent (i.e. move the functionality to the
separate regression git repo).

I believe that all those changes will help in moving us toward what I'm
starting to call a "comprehensive" Boost release. Such a release would not
follow the monolithic structure we currently have of needing the big
include directory. But instead would be a plain collection of the
individual libraries and tools that users can enable/install from the
comprehensive package they download. Or possibly use directly if they add
the individual libraries to their project search (and obviously build for
the ones that need it). Such a "comprehensive" release would make the
release process, and the tools driving it, a much simpler process and
almost certainly increase the frequency of comprehensive release (possibly
even to an almost weekly or daily occurrence).

Last note.. It's never too late to fix problems even if there's work needed
to adjust to the fixes. After all, programmers are accustomed to change. We
deal with it and move on. So if we can make changes that reduce our future
pain, we should do those changes.

-- 
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail