Re: Boost tarball release that works with CMake

From: René Ferdinand Rivera Morell (grafikrobot_at_[hidden])
Date: 2022-11-16 03:22:15

On Tue, Nov 15, 2022 at 9:03 PM Mojca Miklavec via Boost
<boost_at_[hidden]> wrote:
>
> Dear developers,
>
> For a CMake-based project we need to fetch Boost sources as part of
> the build process (unless boost is already installed, in which case
> that version can be used).
>
> We are using CMake's FetchContent that recursively clones the
> repository from GitHub, but that process is extremely slow. On the
> Windows runner in GitHub actions it takes roughly 3 extra minutes to
> "git clone" boost alone (or 11 minutes for something like "choco
> install boost-msvc-14.3"). I tried to fetch the .7z file instead, but
> that one doesn't work because the CMake support gets stripped from the
> distributed sources when generating the tarballs.
>
> A single file would result in a much faster and more reliable
> workflow. The file can be cached on GitHub Actions, on a local
> computer one doesn't need to repeatedly clone the repository (which
> also slows down other parts of CMake) etc.
>
> I started the discussion here:
>     https://github.com/boostorg/release-tools/pull/35
> where it has been explained to me that:
> - either the CMake support needs to be heavily refactored (unlikely to
> happen soon)
> - or the project could generate one tarball using the layout that's
> consistent with git repository
>
> The second option might eventually be fulfilled automatically by
> GitHub, but that is probably not likely to happen any time soon
> either.
>
> I can imagine that creating one extra '.tar.xz' with a slightly
> different layout to support CMake builds might be straightforward to
> make.
>
> Is there any chance to fulfill my dream to be able to build a boost
> tarball with CMake out-of-the-box?

I can't answer the CMake question for you, but... I need to explain
one issue that will hopefully dissuade you using tar balls. Many
months ago we had the default Boost CI pull the regular tarballs to do
testing. Tha had an interesting effect of creating a fair bit of
downloads from the jFrog repository that holds those tarballs.
Everything was great. CI would download Boost really fast. And get all
of it without problems. Or so we thought. One day everything stopped
working with download errors. We contacted jFrog about it. They looked
and saw that we hit a data cap limit. They nicely raised the limit.
Everything was great again. Some months later the same thing happened.
With the same resolution. Then it happened again (I don't actually
remember how many times we repeated that cycle). At which point we
concluded that approach was not viable. As we were using more
bandwidth than the entirety of all the other jFrog downloads combined
(this may be an exaggeration). At which point we started changing the
various CI methods to selectively git clone Boost (there's this great
tool that gets just the right projects you need). And we haven't had
download problems since then.

In conclusion.. Please avoid downloading Boost tarballs from your CI processes.

Thank you, in advance, for your consideration of the download
bandwidth of our gracious release archive files provider, jFrog.

-- 
-- René Ferdinand Rivera Morell
-- Don't Assume Anything  -- No Supone Nada
-- Robot Dreams - http://robot-dreams.net