Re: [config] First steps towards C++20 modules

From: Daniela Engert (dani_at_[hidden])
Date: 2024-08-08 18:53:52

Am 08.08.2024 um 08:11 schrieb Boris Kolpackov:
> Daniela Engert<dani_at_[hidden]> writes:
>
>> Does Microsoft's STL implementation count as "substantial"?
> I assumed they use the export-using approach, but you are right,
> they include the standard library headers in the module purview.
> Though they don't wrap them in one extern "C++" block but rather
> modified each header to explicitly `export extern "C++"` each name
> (via the _EXPORT_STD macro). So it's not quite the same -- they
> have a lot more precision over what ends up with extern "C++"
> (as well as what gets exported).

Language-wise either form is the same [dcl.link]/2

You're right. I remember when Stephan added the language linkage 
specification to exported entities. I think, he somewhere called it a 
"hack" to work around the compiler quirks that MSVC was experiencing 
when it came to definition merging of standard library entities. Fixing 
the (unfortunately still) outstanding bugs requires a decent amount of 
developer love (mostly by Cameron), more love than available these days.

>> BTW: do you know of *any* substantial non-STL codebase that uses modules in
>> any meaningful way? Besides Microsoft Office, of course.
> My go-to test is the async_simple project that was converted to modules
> by Chuanqi Xu:
>
> https://github.com/alibaba/async_simple/tree/CXX20Modules/
>
> I added build2 support and successfully built it using the standard
> library modules with Clang/libc++. That work is here:
>
> https://github.com/boris-kolpackov/async_simple/tree/CXX20Modules-build2
>
I'm aware of this repo.

Giving it another shot, I tried it in multiple environments:

 Â * Windows MSYS2 (Clang 18.1.8 + libc++): configure ok, build fails 
quite early due to some platform-related issue
 Â * Ubuntu 23.04 (Clang 18.1.3 + libc++): configure ok, build fails, the 
Ubuntu upgrade has killed some Clang tools again, sigh
 Â * Windows (MSVC 17.10): configure fail
 Â * Windows MSYS2 (GCC 14.2 + libstdc++): configure ok, build fails due 
to Clang-isms

I have no inclination to clean up this mess. The 3rd-party dependency 
'Asio' is highly platform-malleable, I'm using it in all of my 
presentations since 2022 - as a module on *all* platforms above (and more).

TBH: I don't see said repo as a good example for something remotely 
module-ish. I wouldn't go to the stage and present it as cross-platform 
at all. We can do much better.

> Other than that, I am aware of a couple of build2 users that are
> using modules (including import std) in green-field projects, but
> those are not public.
Very unfortunate, we could learn a lot.
> We do in our company. We use a mixture of headers, pure modules (made from
> scratch), modules with headers wrapped by 'extern "C++"' like you mention,
> modules with headers exporting (parts) of their interface by exported
> using-declarations, and combinations of that, in the codebase driving a huge
> machine that is developed since 2021 and has precursors (i.e. builds from
> code) starting from 2007. 3rd-party libraries (usually modules wrapping
> headers) are part of that. This codebase is in the 1M LOC range.
> That's encouraging, thanks for sharing. Is this MSVC-only or do you
> also build with Clang?

Our customers demand Windows, and Windows only. I use Clang in some of 
the tooling, but not for the core build process. MSVC was the only 
compiler that wouldn't choke on our code at around 2021/2022 and we 
stick with it 'til today.

>
>>> Feels like there bound to be gotchas.
>> Right: [...]
> Another thing I am fuzzy about that I forgot to mention is the
> implementation unit. Feel like the most sensible way is to build
> it off the header (rather than the module). The module interface
> also produces an object file. Are there no issues combining these
> two object files in the same build (duplicate symbols, etc)?

It shouldn't, unless there are still unresolved issues. I had a really 
hard time with Clang 16 in the demo code from from my 2022/2023 
presentations. I could barely work around them and had to change the 
Asio module to fix the exception issue. Clang 17 is much better, but 
still needs workarounds. I didn't test Clang 18 so far because I'm 
occupied with other stuff and a different presentation. May be, I'll 
find the time to recheck again on all platforms and see if all the 
3rd-party libraries didn't evolve too much.

Regarding your specific question: the current Clang documentation still 
mentions a few implementation issues with object files that require 
workarounds. May be you're seeing that kind-of-bug in Asio where it is 
exposing TU-local entities: that's discouraged in general and outright 
ill-formed in modules (Clang hardly detects this!). This needs fixing! 
You can't just wrap existing Asio and call it a day, despite what some 
people may think.

Thanks
Dani

-- 
PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5