Boost logo

Boost :

From: Jeff Bonyun (jeff_at_[hidden])
Date: 2019-05-22 21:40:37

> ... How will we detect whether
> liblzma is capable? If it is not capable but specified, will we allow
> the compression anyway?

I've been investigating this. liblzma defines a version macro,
LZMA_VERSION. It looks like liblzma obtained its multithreaded function
in 5.1.1 (in 2011). So the idea would be to branch on this macro, and
fall back to the currently-used single-threaded function for an older
liblzma version. In that case, requesting any number of threads will
carry out the operation, but will only use a single thread. This makes
it tolerant of different versions, but does mark up the code with a few
#if branches. I'm assuming boost goes for tolerance in this case.

> Is there also an option to have it use the number of cores the
> operating system is advertising?

I'm glad you asked! The xz utility, that liblzma is packaged with, does
this. But it isn't inherent to liblzma itself, which considers "0"
threads to be an error. Copying the approach of xz, I'm intending to use
lzma_cputhreads, which is exposed by liblzma. This was added in 5.2.0
(2014); prior to that it used an internal function that I don't think we
should/can use. This would necessitate yet another #if branch. Assuming
this is acceptable, using "0" threads would indicate that this all-cores
method is desired. This matches the xz utility's approach. For liblzma
versions before 5.2.0, a value of "0" would be translated into "1"
instead. This isn't ideal, as the user clearly wanted more than 1
thread, but I'm not sure we should guess how many the user wanted.

It would still fail in one situation: if your liblzma was built with
`--disable-threads`. In this case, the documentation says that
lzma_cputhreads is omitted. I would imagine it would result in a
link-time error when building Boost.iostreams. I do not have a solution
to this problem, because I don't know how to detect this before the link
error. The best I can offer is a `#define BOOST_*` to disable it
manually if the user runs into this problem.

An alternative is using Boost.Thread, which would solve the problem
gracefully. But I was going to avoid pulling in the extra library, and
not do this. If you think the Official Boost Way would be to use
Boost.Thread, I can change directions.

>> If there is appetite for it, I can make a pull request on github.
> That's the best way to get started. Be sure to include tests that exercise it.

I'll work on it! The bones are done. It's the tests, particular
formatting for boost, and testing-with-different-liblzma versions that
needs to be done.


Boost list run by bdawes at, gregod at, cpdaniel at, john at