[iostreams] Interest in multithreaded lzma filter?

From: Mike (mike.dev_at_[hidden])
Date: 2019-05-24 08:39:55

Accidentally only sent this to James and not the ML:

> Gesendet: Donnerstag, 23. Mai 2019 um 15:19 Uhr
> Von: "James E. King III" <jking_at_[hidden]>
>
>
> On Thu, May 23, 2019 at 8:39 AM Mike via Boost <boost_at_[hidden]> wrote:
> >
> > > Gesendet: Mittwoch, 22. Mai 2019 um 23:40 Uhr
> > > Von: "Jeff Bonyun via Boost" <boost_at_[hidden]>
> > >
> > > > ...  How will we detect whether
> > > > liblzma is capable?  If it is not capable but specified, will we allow
> > > > the compression anyway?
> > >
> > > I've been investigating this. liblzma defines a version macro,
> > > LZMA_VERSION. It looks like liblzma obtained its multithreaded function
> > > in 5.1.1 (in 2011). So the idea would be to branch on this macro, and
> > > fall back to the currently-used single-threaded function for an older
> > > liblzma version. In that case, requesting any number of threads will
> > > carry out the operation, but will only use a single thread. This makes
> > > it tolerant of different versions, but does mark up the code with a few
> > > #if branches. I'm assuming boost goes for tolerance in this case.
> >
> > Does a 2019 Boost release really have to be compatible with a Pre 2011
> > release of liblzma? Is this combination even tested?
> > I.e. are we sure this would be the only incompatibility?
> >
> > How about just adding this functionality unconditionally and see if
> > someone complains?
> >
>
> So basically if someone has an older lzma they are limited to boost
> 1.70.0 or earlier.  That's perhaps reasonable as long as it is
> documented, but it hasn't been the typical way I have seen things
> added to existing code.

Well, my second question was if you are confident that 1.70 does actually
work with liblzma versions older than 5.1.1? If this isn't tested, it is not
supported and if it is not supported it imho makes little sense to complicate
the source code to achieve potential compatibility with (what I assume to be)
 a very, very small number of setups.

>
> Another way to handle it is to keep the old path in a macro like
> BOOST_IOSTREAMS_NO_MULTITHREADED_LZMA.  This is common through boost.
> Ideally, detecting the liblzma version from the header could then set
> BOOST_IOSTREAMS_NO_MULTITHREADED_LZMA and it would always do the right
> thing, but the simplest way to do it is to let the consumer of the
> library set this if they need it.
>

Personally I'm not a fan of configuration macros when they are not absolutely
necessary because it effectively means you introduce yet another variant of boost
that may or may not be tested and may or may not be compatible with the variant
another dependency in my project was tested with (The total configuration space
of boost is already far too large for my taste).

As Robert Schumacher said in his cppcon talk: It's not just about you and the
user, but also every library between you and the user. All of them now either have
to support both choices or have to make one choice and hope they don't
become incompatible with a different 3rd party library.
Yet another reason, why package & dependency management is so hard in c++.

However, as long as the interface is untouched by this macro I guess it would
be fine.

Best

Mike