Re: [iostreams] Interest in multithreaded lzma filter?

From: Jeff Bonyun (jeff_at_[hidden])
Date: 2019-05-26 04:08:34

On 5/24/19 3:39 AM, Mike via Boost wrote:
> Well, my second question was if you are confident that 1.70 does actually
> work with liblzma versions older than 5.1.1? If this isn't tested, it is not
> supported and if it is not supported it imho makes little sense to complicate
> the source code to achieve potential compatibility with (what I assume to be)
>   a very, very small number of setups.

Aha! I have an answer. Boost.iostreams 1.70 works with liblzma 
4.999.7beta or later. The function signatures are different before that. 
So worrying about multithreaded compatibility before that is irrelevant, 
but it still leaves the 4.999.7beta to 5.2.0 interval to handle.

Luckily, the auto-detection of a sufficiently new version of liblzma 
seems to be working fine. For those keeping track, anything before 5.2.0 
will be built without threads, because it wasn't supported in the 
default build of liblzma. It will work fine as a single threaded 
library. Anything 5.2.0 or after will enable threads, and will be able 
to auto-detect the number of cores too.

On 5/24/19 11:27 AM, Peter Dimov via Boost wrote:
> If we could avoid the need for a configuration macro altogether, that 
> would be best; how common is liblzma built with --disable-threads? Or, 
> can we detect that somehow? 

Unfortunately the --disable-threads (or --enable-threads=no) ./configure 
arg to liblzma doesn't change the header files of liblzma at all. That 
would have been the easiest for us, if they defined a macro to reflect 
the lack of threading, but they don't.

By building them side-by-side, I can see the differences are in:
 Â  - the .so files (--disable-threads means it doesn't have 
`stream_encode_mt` or `lzma_cputhreads`, both of which I use).
 Â  - the .la file, which isn't typically part of a packaged release, but 
is installed if built from source.
 Â  - the .pc file for pkgconfig, which might be installed by packaged 
releases (my Ubuntu 16.04 has it, my RHEL 6 doesn't).

So unless we wanted to try finding the .so file that will be used, and 
search its symbols, I would say that we should default to assuming that 
threads are enabled on any sufficiently-new version, and provide a 
define flag to turn that off if the user is having trouble building. 
(And to the future developers searching the web for reasons why their 
boost won't build, I'm sorry. Try 
`<define>BOOST_IOSTREAMS_LZMA_NO_MULTITHREADED`.)

I'm finishing my pull request. Should be available in the next day or so.

Jeff