Re: [cmake] Problem with dependencies for configure checks

From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2021-06-04 15:30:24

On 6/4/21 5:44 PM, Niall Douglas via Boost wrote:
> On 04/06/2021 13:04, Andrey Semashev via Boost wrote:
>> On 6/4/21 2:03 PM, Niall Douglas via Boost wrote:
>>> On 04/06/2021 10:12, Andrey Semashev via Boost wrote:
>>>
>>>> My stance on this is that Boost.Filesystem must be built on a system
>>>> with kernel headers matching or older than the systems that will run the
>>>> binary. This matches the same requirements for Windows. I don't support
>>>> configs with headers newer than runtime kernel.
>>>
>>> It's trivially easy to do a runtime check for statx kernel support and
>>> fall back on older syscalls if needed.
>>>
>>> This is LLFIO doing so, and it works on Linux kernel 2.6:
>>>
>>> https://github.com/ned14/llfio/blob/develop/include/llfio/v2.0/detail/impl/posix/stat.ipp#L205
>>>
>>
>> The code you pointed to is not what I would call trivial. There are
>> quite a few places where statx is used in Boost.Filesystem, in some
>> places exclusively (i.e. there is no fallback).
> 
> There are only a **very** few things which statx provides which older
> syscalls cannot. statx's big gain is it is a snapshot not subject to
> races, but that hardly affects Filesystem.
> 
>> I could add a runtime
>> check, but I definitely don't want that check to happen on every call.
>> This means the result must be stored globally in a thread-safe manner.
> 
> At work we've got llfio::stat_t::fill() being called in some places
> millions of times per second.
> 
> It is implemented by attempting statx first, and if it is not supported,
> then stat or other syscalls to emulate the same.
> 
> You cannot cache statx support/non-support because only some filing
> systems support statx, and others do not. So depending on the path used,
> it may work, but at other times, not.

Isn't statx supposed to complete successfully through the stat route in 
the kernel, if the filesystem does not support statx natively?

> Right now at work we're on kernels which don't support statx, so we are
> always getting back ENOSYS and then falling back to emulation. We have
> not found any performance issues.
> 
>> Then there are other syscalls, on Linux and other systems, should I also
>> add runtime checks for those? All that to support what I consider
>> improperly configured systems. It just doesn't look like a worthwhile
>> effort while I have other things to do, including in Boost.Filesystem too.
> 
> A very great deal of Boost users are either on older Linux kernels, or
> are running Boost inside Docker containers where various newer syscalls
> are disabled e.g. where I work. In my opinion, if you like your users,
> you need to implement syscall fallback for all recent Linux syscalls,
> same as you'd do on Windows. I agree that you can reasonably cut off all
> Linux kernels past EOL, so you only need to support syscall fallback for
> kernels since EOL.

My point is that if you're running a kernel that doesn't support statx 
then you should compile Boost.Filesystem with headers from that kernel. 
It will not have statx syscall and Boost.Filesystem will compile to use 
the old stat syscall.

<rant>
Docker people seem to think of Docker images as of VMs while they are 
not. Hence they consider it normal to build the image on one system and 
then run it on another, with possibly older kernel. Well, this is wrong. 
It doesn't work that way, and Boost.Filesystem is not the only piece of 
software that will break because of that.
</rant>