Re: [boost] Thoughts on Boost v2

Subject: Re: [boost] Thoughts on Boost v2
From: Tom Kent (lists_at_[hidden])
Date: 2014-05-22 10:23:09

On Thu, May 22, 2014 at 1:28 AM,  <pmenso57_at_[hidden]> wrote:
> ----- Original Message -----
>> From: "james" <james_at_[hidden]>
>
>> On 22/05/2014 02:20, pmenso57_at_[hidden] wrote:
>> > How about the burden of supplying a conforming compiler be put on MS
>> > rather than on everyone else?
>> How is that supposed to help *users* of Boost?
>> <snip>
>> To me, a butt-ugly bugfix is still better than no bugfix.
>> <snip>
>> (And take a look at the docs, and the coherency, and the focus on being useful for building real apps rather than some kind of research playground for advanced templating)
>
> Long term over short term.  Short term is to workaround every compiler bug under the sun--which is what Boost and many other libraries have done.  The result is a mess.  It provides little incentive for compilers to be fixed, and it causes every "portable" library to be written either in LCD C++ or be hacked up with alternate implementations for the ten different languages (i.e. compiler dialects) that it is targeting.
>
As I see it, this is the crux of the conflict here.

Users:
Want a boost that *is* supported on (nearly) the LCD C++. They have an
existing compiler (I would guess it is split roughly equally between
gcc and msvc, but they majority of those are probably gcc-4.6 and
msvc-10.0, not newer versions),  that is already being used on their
project and they might hope for a couple new features...like json or
thread pools or something (probably not deep c++ stuff like type
traits), but mostly want bug fixes for issues they find.

Developers (of the libraries):
Don't want to be held back by having to support LCD C++ because
1) The new (C++11/14/...) features make development of their library
faster/cleaner/less painful/possible at all.
2) They want to develop a library/feature that may someday go into the
language or standard libraries (a.k.a. the 'research' oriented
development)....which has historically been an important part of
boost.

So.....

Currently when we do releases of boost, we have a collection of
compilers that everything is tested against and by-and-large if you
are using one of those things will work for you. This goes all the way
back to msvc-8.0/gcc-4.4/clang-3.0. This is great for the users, they
get a ton of stuff and don't have to worry too much about it not
working. However, it is also overkill. I've never seen an organization
that uses even half of the libraries in boost. (I'd be interested in
hearing an example of one that does :-) ) This has actually been a
problem for my organization because with its huge size (several GB
compiled) it becomes unwieldy to manage across multiple projects.

*If* we want to move to a new organizational structure, the only way I
could see it working out is if it substantially more modular than what
we currently have. Here is an idea as to how this could be possible:

------

Each library would publish a list of what other libraries it depended
on and what compilers (and platforms?) it supported. This would go in
a file at the root of its git tree.

We would then provide a tool to the users (like 'b2 headers'/BCP on
steroids) that would take a list of libraries that the user wants and
the compilers that they want to use. It would then download (or fail
the dependency check) these libraries and their dependencies, and
build them (or download binaries). This would also work very well with
the various linux package managers as they would follow the same
dependency pattern with their own tools.

This would allow developers to introduce new libraries without having
to worry about supporting older compilers. Users of old compilers
would know, up-front, that this library isn't available for them. If
the library caught on and there was user demand, support for older
versions could be added later, if possible. The bar would be
substantially lower for getting a new library into boost, but I
wouldn't go as far as the original proposal went with only requiring
automated tests to pass. Libraries would still need to go through a
review process so that users can be assured that libraries in boost of
high quality.

We would still require having coordinated releases of all the
libraries for a specific version number, however the release would
simply be providing a file that lists the git tag for the version of
each library that makes up the release. The toughest part of this
would be making sure two libraries that both have a dependency on a
third library are dependent on the same version of that library.
Breaking changes in libraries that have dependencies would have to be
communicated to the dependent libraries well in advance of a release.

Overall, the biggest problem I see with this proposal is testing. By
being modular, individual test times would be greatly reduced. This
would hopefully enable CI like testing at each checkin, which isn't
currently possible when it takes 8+hrs to run the tests on some
platforms. However, as each library would be specifying exactly which
compilers it uses, we would need a way to make sure that those are all
getting hit. We currently don't have enough tester diversity to
completely accomplish this.

------

If this could all be made to work, it would let advanced users have
the advanced features that they want, while keeping the features that
the normal users use a lot fully supported on the platforms they need.
But it is just one idea...I'm sure there are lots of holes in it.

Tom Kent