Re: [boost] Let's stop distributing binaries

Subject: Re: [boost] Let's stop distributing binaries
From: Tom Kent (lists_at_[hidden])
Date: 2018-08-30 02:40:46

On Wed, Aug 29, 2018 at 9:53 AM Robert Ramey via Boost <
boost_at_[hidden]> wrote:

> The fact is that users of boost use compilers with differing levels of
> C++ standard conformance.  This creates a number of problems including
> (but probably not limited to)
>
> a) Using libraries built with different levels of C++ conformance may
> create ABI issues.
>
> b) it creates a discussion about "dropping C++03 support".  Actually it
> creates a discussion about that the above actually means.
>
> c) It's motivated a suggestion that all libraries move to header only.
> This is not  great idea - but I won't address that here.
>
> I believe all these issues - and more - would disappear if we just
> stopped distributing binaries.  Each user would be building his own
> boost version with (presumably) the same compiler.  Since C++03
> compatible libraries are C++?? compatible, each library author can
> maintain the level of backward compatibility that he wishes.
>
> But if we're not going to distribute binaries, then users might as well
> just clone the latest "certified - good - tagged" version in the master
> branch from the github repo.  That is, let's just stop or truncate the
> whole release circus.  Replace with an occasional
> "certification/versioning procedure" which would make available a
> "squashed" master suitable for efficient cloning.
>
> This would make everyone's life easier.
>
> Robert Ramey
>

I've never heard of any binaries for linux/unix/mac distributed by us, so I
guess this is reference to the Windows/Visual Studio binaries? As the
person who builds them, I suppose I should weigh in on this.

TL;DR - We need to provide an easier way to get windows/visual studio users
up and running with boost than building with b2. I love boost and want to
make it as easy as possible for others to use.

History of the binaries
-----------------------------
When I first got started with the boost project about a decade ago, I was a
decent C++ developer at a large company (Boeing). We use C++ to develop
simulators for our aircraft.

I had worked extensively with Visual Studio and our large code base. Our
primary product (our simulation) is hundreds of thousands of lines of code
and uses dozes of external dependencies. Most of the "external"
dependencies were internally (other groups within the company) produced
products. We would get dumps of .h and .lib files from other groups and add
the directories to our visual studio solution include/lib paths. We would
occasionally encounter ABI errors, but were very good at managing the
configuration of libraries coming in to keep these from affecting us. We
basically had a system similar to Conan...but fifteen years ago.

I got the task of adding Boost to one of our projects in our solution, so
we could use a couple very nice libraies: asio, date time, interprocess,
program options, thread, and test. Several of these are not header only, so
I knew I needed static libs for them. I fought for weeks trying to get
binaries. I wasn't familiar with bjam (and didn't know it was included with
the source!) so I tried to load the source into visual studio and compile
it....that didn't go well. Eventually I found the horrible "Getting started
with windows" page [1]. It tries to lead a user to the command `b2
--toolset=msvc-14.0 --build-type=complete stage`, but fails miserably to
identify the arguments that are actually needed and most users would never
figure out from the page that that is the command they need.

After many *dozen* hours fighting with b2, digging through terrible b2
documentation, and googling for people who had successfully built boost on
windows, I finally got my static libs out! It was a nightmare that still
motivates the work that I do to make binaries available.

After getting them built, I packaged them up for use by my team. After
doing this for a few releases, and seeing lots of other people with the
same horror stories on Stack Overflow ("build boost on windows" has 877
results [2]), I decided to make the binaries that I was now good at
building public. For years I had them on my website...but they were getting
big and my hosting provider was getting more annoying so I requested, and
the mailing list agreed, that I could release the binaries along side the
source release through the boost channels. Which is basically the state to
this day.

Current Usage
-------------------

The windows binaries are currently downloaded ~14,000 times per month from
BinTray and ~2000 times per week from sourceforge.

They are generally built with the default compiler options, which is ABI
compatible for most users with the correct version of visual studio. In
recent months, I have started seeing some issues from users who are
changing the /std flag (new to VS2015). Which I acknowledge was one of the
main issues mentioned above. There may also be more issues now that VS2017
is rolling new point releases very frequently. However, to remove the
binaries for this reason is throwing out the baby with the bath water.

Going Forward
--------------------

1) I plan to continue building the binaries and making them available the
day of release. If the community would prefer that I get a separate bintray
account and have them available there, I would be amenable to that, but I
don't see that as being a net gain for anyone. I'd like to keep them as
easily-discoverable as possible for new users.

2) If we can make a full cmake transition, where visual studio solutions
can easily be created to make the needed libraries, that would be a good
time to revisit this. I think there are still a lot of hurdles that need to
be cleared on the road to this though.

3) I'd like to (when I can find the time) start pushing the binaries for
each release into some of the new package management systems (conan, vcpkg,
etc). While keeping the existing ones available as well, as the windows
package management systems are only used by a tiny minority.

4) If boost widely drops C++03 support, I would likewise trim the toolsets
that we build binaries for. (Currently supported are VS2005/msvc-8.0
through VS2017/msvc-14.1)

5) Something needs to be done to seriously overhaul the getting started for
windows page. If I can find time I'll take a crack at that.

I have great respect for where this request comes from. I wish every
developer (especially the ten years ago me) could easily understand how to
build their own copy of boost. However, I think we as a community need to
spend a lot more time thinking about the end-users who want to use boost
but aren't already familiar with it.

Tom

[1] https://www.boost.org/doc/libs/1_68_0/more/getting_started/windows.html
[2] https://stackoverflow.com/search?q=build+boost+on+windows