Re: [boost] Let's stop distributing binaries

Subject: Re: [boost] Let's stop distributing binaries
From: Edward Diener (eldiener_at_[hidden])
Date: 2018-08-30 15:18:09

On 8/29/2018 10:40 PM, Tom Kent via Boost wrote:
> On Wed, Aug 29, 2018 at 9:53 AM Robert Ramey via Boost <
> boost_at_[hidden]> wrote:
> 
>> The fact is that users of boost use compilers with differing levels of
>> C++ standard conformance.  This creates a number of problems including
>> (but probably not limited to)
>>
>> a) Using libraries built with different levels of C++ conformance may
>> create ABI issues.
>>
>> b) it creates a discussion about "dropping C++03 support".  Actually it
>> creates a discussion about that the above actually means.
>>
>> c) It's motivated a suggestion that all libraries move to header only.
>> This is not  great idea - but I won't address that here.
>>
>> I believe all these issues - and more - would disappear if we just
>> stopped distributing binaries.  Each user would be building his own
>> boost version with (presumably) the same compiler.  Since C++03
>> compatible libraries are C++?? compatible, each library author can
>> maintain the level of backward compatibility that he wishes.
>>
>> But if we're not going to distribute binaries, then users might as well
>> just clone the latest "certified - good - tagged" version in the master
>> branch from the github repo.  That is, let's just stop or truncate the
>> whole release circus.  Replace with an occasional
>> "certification/versioning procedure" which would make available a
>> "squashed" master suitable for efficient cloning.
>>
>> This would make everyone's life easier.
>>
>> Robert Ramey
>>
> 
> I've never heard of any binaries for linux/unix/mac distributed by us, so I
> guess this is reference to the Windows/Visual Studio binaries? As the
> person who builds them, I suppose I should weigh in on this.
> 
> TL;DR - We need to provide an easier way to get windows/visual studio users
> up and running with boost than building with b2. I love boost and want to
> make it as easy as possible for others to use.
> 
> History of the binaries
> -----------------------------
> When I first got started with the boost project about a decade ago, I was a
> decent C++ developer at a large company (Boeing). We use C++ to develop
> simulators for our aircraft.
> 
> I had worked extensively with Visual Studio and our large code base. Our
> primary product (our simulation) is hundreds of thousands of lines of code
> and uses dozes of external dependencies. Most of the "external"
> dependencies were internally (other groups within the company) produced
> products. We would get dumps of .h and .lib files from other groups and add
> the directories to our visual studio solution include/lib paths. We would
> occasionally encounter ABI errors, but were very good at managing the
> configuration of libraries coming in to keep these from affecting us. We
> basically had a system similar to Conan...but fifteen years ago.
> 
> I got the task of adding Boost to one of our projects in our solution, so
> we could use a couple very nice libraies: asio, date time, interprocess,
> program options, thread, and test. Several of these are not header only, so
> I knew I needed static libs for them. I fought for weeks trying to get
> binaries. I wasn't familiar with bjam (and didn't know it was included with
> the source!) so I tried to load the source into visual studio and compile
> it....that didn't go well. Eventually I found the horrible "Getting started
> with windows" page [1]. It tries to lead a user to the command `b2
> --toolset=msvc-14.0 --build-type=complete stage`, but fails miserably to
> identify the arguments that are actually needed and most users would never
> figure out from the page that that is the command they need.
> 
> After many *dozen* hours fighting with b2, digging through terrible b2
> documentation, and googling for people who had successfully built boost on
> windows, I finally got my static libs out! It was a nightmare that still
> motivates the work that I do to make binaries available.
> 
> After getting them built, I packaged them up for use by my team. After
> doing this for a few releases, and seeing lots of other people with the
> same horror stories on Stack Overflow ("build boost on windows" has 877
> results [2]), I decided to make the binaries that I was now good at
> building public. For years I had them on my website...but they were getting
> big and my hosting provider was getting more annoying so I requested, and
> the mailing list agreed, that I could release the binaries along side the
> source release through the boost channels. Which is basically the state to
> this day.
> 
> 
> Current Usage
> -------------------
> 
> The windows binaries are currently downloaded ~14,000 times per month from
> BinTray and ~2000 times per week from sourceforge.
> 
> They are generally built with the default compiler options, which is ABI
> compatible for most users with the correct version of visual studio. In
> recent months, I have started seeing some issues from users who are
> changing the /std flag (new to VS2015). Which I acknowledge was one of the
> main issues mentioned above. There may also be more issues now that VS2017
> is rolling new point releases very frequently. However, to remove the
> binaries for this reason is throwing out the baby with the bath water.
> 
> 
> Going Forward
> --------------------
> 
> 1) I plan to continue building the binaries and making them available the
> day of release. If the community would prefer that I get a separate bintray
> account and have them available there, I would be amenable to that, but I
> don't see that as being a net gain for anyone. I'd like to keep them as
> easily-discoverable as possible for new users.
> 
> 2) If we can make a full cmake transition, where visual studio solutions
> can easily be created to make the needed libraries, that would be a good
> time to revisit this. I think there are still a lot of hurdles that need to
> be cleared on the road to this though.
> 
> 3) I'd like to (when I can find the time) start pushing the binaries for
> each release into some of the new package management systems (conan, vcpkg,
> etc). While keeping the existing ones available as well, as the windows
> package management systems are only used by a tiny minority.
> 
> 4) If boost widely drops C++03 support, I would likewise trim the toolsets
> that we build binaries for. (Currently supported are VS2005/msvc-8.0
> through VS2017/msvc-14.1)
> 
> 5) Something needs to be done to seriously overhaul the getting started for
> windows page. If I can find time I'll take a crack at that.
> 
> I have great respect for where this request comes from. I wish every
> developer (especially the ten years ago me) could easily understand how to
> build their own copy of boost. However, I think we as a community need to
> spend a lot more time thinking about the end-users who want to use boost
> but aren't already familiar with it.

You have reflected my thinking exactly. This whole business of 
"developers are so smart they will have no problem building Boost for 
whatever toolchains and for whatever libraries and for whatever C++ 
standard build levels they desire using b2" is such nonsense that I do 
not even know where to begin. I am not putting down Boost Build in any 
way but if Boost developers themselves have a hard time figuring out how 
to build individual libraries with particular toolchains at particular 
C++ standard levels and having the resultant output go where they want 
it to go, just imagine what this is like for the particular end-user who 
might want to build Boost as a whole for different toolchains at 
different levels while including/excluding some shared/static library 
building.

I am for Robert Ramey's suggestion that if we decide not to distribute 
Boost binaries in any way anymore we need to write a much better 
explanation of how to build Boost as a whole for a particular toolchain 
and a particular standard level and for the various combinations of 
shared/static libraries and for the various choices of what 
shared/static libraries can or cannot be built than what we currently 
have. In fact we need that now in the "Getting Started" page, no matter 
what direction we go, but I know if I press for that I will be accused 
of making more work those people knowledgeable about Boost Build who 
could write a much better explanation than what currently exists, so I 
will not do so. But thank you Tom, if only you have brought to attention 
that it is the end-users of Boost we should be serving and not just 
ourselves.

> 
> Tom
> 
> 
> [1] https://www.boost.org/doc/libs/1_68_0/more/getting_started/windows.html
> [2] https://stackoverflow.com/search?q=build+boost+on+windows