Hi. I would like to share a little more info. The speedup mentioned (1.44x) was achieved in Release config. I tested in Deubg config as well. Speed-up was 1.89x. So, the expected speed-up would be >3x in Release and >5x in Debug for the full Boost. Best, Hassan Sajjad On Sat, Apr 19, 2025, 22:19 Hassan Sajjad <hassan.sajjad069@gmail.com> wrote:

Hi.

I am seeking an endorsement of my software, HMake ( a library in a way (hconfigure) ), so that it can go through the review and possibly be included in Boost as a modern replacement for b2.

Following is a comprehensive list of benefits that HMake provides compared to b2 or other build-systems. Some of this might not apply to one or the other.

1) Using C++ itself. You have to learn a library interface, but not a new programming language. 2) Single file for the whole build specification. 3) Needs very little maintenance or build-file modifications. You can use globbing patterns for specifying source and module files. Specifying tests can also be made simpler, e.g., with a convention to prepend a character in the file-name like (r, c), etc. (run, compile). And a JSON file if you expect any run to produce a particular output. So, you reconfigure when you add/delete a new test or modify the JSON file instead of first editing the build file. hmake.cpp file size is up to 10x smaller compared to CMakeLists due to C++ expressibility. 4) Very performant and efficient. On par with Ninja in both metrics, if not better. Some optimizations are still under development. 5) Multiconfig. You can build multiple configurations with one command. Has b2-inspired features/properties. 5) Fast configuration. Configuration will always be less than 5s ( even with up to 10 full Boost configurations on Windows ). 6) Very Extensible. Can be easily extended to support features like code generation, etc. The build algorithm allows for dynamic nodes and edges in the same process. No other does that. 7) Has header-units support, which, contrary to C++ modules, offers build-speed improvements without any source-code changes.

The following are not yet implemented. But if you evaluate the above points, you can see that the following can be easily supported as well.

8) I wrote this paper https://htmlpreview.github.io/?https://github.com/HassanSajjad-302/iso-paper.... It is at most a 2-day work for a compile-dev to support this. This will allow 4x compilation speed improvement for Boost and even bigger gains for bigger projects. 9) HMake will support a feature called BigHu (described in previous email), which makes rebuilds incredibly fast as well. Also, I think header-units can be backported for older C++ versions as well, since they change semantics very little ( mostly around the preprocessor ). 10) HMake will also support https://lists.isocpp.org/sg15/2023/11/2146.php and https://lists.isocpp.org/sg15/2023/11/2106.php if these get implemented. 11) compile-commands.json support. HMake will generate these at configure time for the configurations being configured. 12) CMake integration support through CPS. Some features, like header-units might not work. 13) HMake will support multiple other programming languages, frameworks like iOS and Android app generation, different installers, etc.

HMake is still a novel build-system that is under heavy development. A project of Boost size using this in production will provide highly valuable insights that will tremendously improve the HMake itself as well. HMake is written in C++20, follows best practices, is extensively tested, uses TDD, and the core library is around 15k C++ lines.

I can also do a video call presentation of my software. This might be a quicker way to make yourself familiar with my software capabilities than reading documentation. Because I can do live examples and explain details accordingly.

Best, Hassan Sajjad

On Wed, Apr 16, 2025 at 5:21 PM Hassan Sajjad <hassan.sajjad069@gmail.com> wrote:

...

The interactions to reproduce the results. https://github.com/HassanSajjad-302/HMake/blob/main/Projects/Boost/hmake.cpp

On Wed, Apr 16, 2025, 16:34 Hassan Sajjad <hassan.sajjad069@gmail.com> wrote:

...

Hi.

I compiled 25 Boost libraries with C++20 header-units, including examples and Tests of a few. This resulted in a 1.44x speed-up if we exclude the scanning. Scanning was as slow as the header-unit build.

I have a solution for that. I am sending this paper (Making some final edits) https://htmlpreview.github.io/?https://github.com/HassanSajjad-302/iso-paper... , which allows for a build-system to use a scanning-less approach to build header-units.This method requires only minimal modifications to the compiler and enables memory-mapped file sharing, which significantly reduces filesystem overhead and further boosts speed.

My proposal for a scanning-less approach will allow HMake to have another amazing feature, monolithic Big header-units. BigHu will allow you to map all the headers from a directory to a single header-unit. This header-unit includes all these header-files and consumes them as header-files. Now, whenever the compiler requests a header-unit from this directory, the build-system instead sends this consolidated BigHu. This will not only improve the clean build speed, because of fewer process launches. But also massively boosts the rebuild speed as few header-units need to be read instead of lots of files. No source-code editing will be needed to use this feature. And you will have the option to either build a library as BigHu (if you don't mean to edit it) or as small header-units (if you are working on it).

HMake will soon support compile_commands.json, which I guess can work fine with header-units. Thus you can benefit from both, faster compilation and working intelliscence.

The API shown in hmake.cpp is a work in progress. HMake build-system benefits are numerous, even if we leave out the header-units support. This makes it a perfect replacement for B2. I believe we can complete the switch in the next 2 months. And if some compiler supports my paper, we get >3x faster compilation as well.

Best, Hassan Sajjad

Generally speaking your "benefits list" has quite bold statements

Are you proposing replacement of b2 for Boost or just yet another build system to the club?

These statements are based on concrete evidence. I am proposing to replace b2. And why is it good? I mean you need to have a really good point to make

build system that uses C++ files to define a build...

3x faster compilation in Release and >5x faster in Debug mode for Boost and better for bigger projects. No other build-system can do that. HMake will have a few other game-changing features in the future as well. It is not possible to evolve a DSL based build-system like that.

I mean today CMake is the most common industry standard build system

that works really well. Why would you want to use it over something like this?

CMake can not support C++20 header-units. One benefit that I forgot is that once you have compiled your project with header-units in HMake and you want to move to C++20 modules, you do not need to make a single edit in your configuration file (Not true for projects like Boost which want to support both header-files and modules. In this case some macro definitions will be needed). With this, #10 and #8, C++20 modules and header-units become an absolute bliss to use with HMake. See build systems run multiple tests (cmake, autotools and others) to make

sure that everything is as expected. How exactly do you think you'll be able to speed it up? Because most of the configuration time is actually performing checks like potential dependencies, test builds, etc.

By caching the results and not running tests every time. Also, configure.exe uses the same code that is used by build.exe. It is fully parallel as well. Now I don't want to discourage you but it is a really non-standard

approach to build.

Isn't it superior, though? And on more technical details: while it is more than ok to share ideas,

generally speaking seeking of an endorsement is something done to get ready for a review... Not sure it is even close to it.

How so? Best, Hassan Sajjad On Mon, May 5, 2025 at 5:49 PM Artyom Beilis via Boost < boost@lists.boost.org> wrote:

Hi,

...

Following is a comprehensive list of benefits that HMake provides

compared

...

to b2 or other build-systems.

Generally speaking your "benefits list" has quite bold statements

Are you proposing replacement of b2 for Boost or just yet another build system to the club?

...

1) Using C++ itself. You have to learn a library interface, but not a new programming language.

And why is it good? I mean you need to have a really good point to make build system that uses C++ files to define a build...

...

4) Very performant and efficient. On par with Ninja in both metrics, if not better. Some optimizations are still under development.

So how would it be better than just old good cmake + ninja?

I mean today CMake is the most common industry standard build system that works really well. Why would you want to use it over something like this?

Serious question that needs a deep answer.

...

5) Fast configuration. Configuration will always be less than 5s ( even with up to 10 full Boost configurations on Windows ).

See build systems run multiple tests (cmake, autotools and others) to make sure that everything is as expected. How exactly do you think you'll be able to speed it up? Because most of the configuration time is actually performing checks like potential dependencies, test builds, etc.

Now I don't want to discourage you but it is a really non-standard approach to build.

And on more technical details: while it is more than ok to share ideas, generally speaking seeking of an endorsement is something done to get ready for a review... Not sure it is even close to it.

Regards, Artyom

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

I have reservations about this. I am not in favor of rewriting the configuration selection for the various combinations of architecture, binary format, address model, and ABI. This would require significant effort and could introduce unnecessary complexity, given the work already invested in the current implementation.

HMake API has features / properties inspired by / mapped on b2 features. On Tue, May 6, 2025 at 12:18 PM oliver.kowalke--- via Boost < boost@lists.boost.org> wrote:

06.05.2025 00:59:04 Hassan Sajjad via Boost <boost@lists.boost.org>:

...

I am proposing to replace b2.

I have reservations about this. I am not in favor of rewriting the configuration selection for the various combinations of architecture, binary format, address model, and ABI. This would require significant effort and could introduce unnecessary complexity, given the work already invested in the current implementation.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

. I am proposing to replace b2.

While I can agree that b2 isn't particularly good build system by any stretch of imagination There was a long and yet incomplete effort of moving to CMake that most around there are familiar with. CMake works very well and is very well supported all around. Replacing one not-so-good homebrew system that was at least polished with yet another homebrew system is a horrible idea. Boost needs _less_ of "non-invented-here-syndrome": not _more_ of it Refrasing Stroustrup there are two types of build systems - everybody complains about and nobody uses, and cmake is one everybody is complaining about ;-)

I mean today CMake is the most common industry standard build system

...

that works really well. Why would you want to use it over something like this?

CMake can not support C++20 header-units. One benefit that I forgot is that once you have compiled your project with header-units in HMake and you want to move to C++20 modules,

It seems like a feature gap that can be fixed at CMake level - either by cmake developers or by external patch. Have you considered talking with CMake developers about it and/or submitting a patch? What do they think about this? I'm sure they are aware of the problem :-)

By caching the results and not running tests every time. Also, configure.exe uses the same code that is used by build.exe. It is fully parallel as well.

You don't reconfigure each and every build... cmake caches _a lot_ sorry but it seems to be a false statement. Best Artyom

On Tue, May 6, 2025 at 5:05 PM René Ferdinand Rivera Morell <grafikrobot@gmail.com> wrote:

On Tue, May 6, 2025 at 2:52 AM Artyom Beilis via Boost <boost@lists.boost.org> wrote:

...

...

. I am proposing to replace b2.

While I can agree that b2 isn't particularly good build system by any stretch of imagination

Specific build systems, and all software, has advantages and disadvantages. We would benefit from feedback as to improvements we can make to our tools.

See, I discussed it a lot back in 2011 on this list. As bottom line when I worked on Boost.Locale having large experience with both, autotools cmake and even basic makefiles I totally failed to perform all tasks I needed I don't recall exactly but basic stuff like finding 3rd party libraries or conditional builds was total nightmare and very hard to maintain. Compare for example cmake: https://github.com/boostorg/locale/blob/develop/CMakeLists.txt and jamfile https://github.com/boostorg/locale/blob/develop/build/Jamfile.v2 that is twice as large and some tests that are built in in cmake (like test header/library) are generated in separate build The concept of "conditional flags" per build makes anything beyond simple tasks very challenging I couldn't complete the build system on my own and Vladimir Prus had come to my rescue... Even after that I failed to create conditional tests (so I just put ifdef to make them do nothing) but I think Alexander Grund that took the maintenance from me and does absolutely amazing job had fixed it, Bottom line, in my opinion bjam is a very limited build system. Maybe it servers Boost ok, but I would never choose one outside the scope mostly due its design of doing everything in the same flow under all conditions rather than the much simpler and easy to use CMake approach. I don't tell cmake is perfect - ohhh boy it isn't but I find it much more manageable, easy to use and well documented. Maybe stuff had changed since 2011 but for me it was very unpleasant experience, and I worked with autotools that were surprisingly simpler to use (at least for me) So take my words with a grain of salt :-) Artyom

On Tue, May 6, 2025 at 10:39 PM Дмитрий Архипов via Boost <boost@lists.boost.org> wrote:

вт, 6 мая 2025 г. в 17:55, Artyom Beilis via Boost <boost@lists.boost.org>:

...

See, I discussed it a lot back in 2011 on this list.

As bottom line when I worked on Boost.Locale having large experience with both, autotools cmake and even basic makefiles I totally failed to perform all tasks I needed I don't recall exactly but basic stuff like finding 3rd party libraries or conditional builds was total nightmare and very hard to maintain.

Compare for example cmake: https://github.com/boostorg/locale/blob/develop/CMakeLists.txt and jamfile https://github.com/boostorg/locale/blob/develop/build/Jamfile.v2 that is twice as large and some tests that are built in in cmake (like test header/library) are generated in separate build

I looked at those two files. As far as I can see, the vast majority of that jamfile is configuring dependencies, particularly Iconv and ICU. The only reason there's nothing similar in the CMakeLists.txt is because there are Find modules for those libraries provided with CMake. Ironically enough, they were not provided with CMake in 2011 (FindICU is provided since version 3.7 which was released in 2016, FindIconv since version 3.11 released in 2018). Moreover, things like private/public properties were only added to CMake in 2013. Again, ironically as an effort to copy b2 usage requirements.

Actually in cppcms/booster.locale original cmake (where boost.locale came from) I used one liners to find headers and libraries only difference was for debug build to find iculibrary with "d" suffix, I didn't use predefined FindICU Now finding 3rd part library is the central part of the configuration/build system for real world projects. Looking as an example at my big FOSS C++ project dependencies CppCMS: icu, zlib, openssl/gnu-tls, iconv, pcre, python CppDB: mysql, odbc, postgresql, sqlite Dlprimitives: python, sqlite, OpenCL, protobuf, openblas/cblas, hdf5, boost-python, boost-numpy OpenLiveStacker: libtiff, cppcms, libcurl, indilib, indigo, libraw, libgphoto, opencv, cfitsio, libusb, asisdk, touptek sdk, zlib, android camera2 And finding each of them in CMake is virtually two lines (header, library) and I rarely use FindXXX with exception of some complex stuff (ironically boost python) So Boost is quite a unicorn when the vast majority of the libraries are header only or come with 0 dependencies and using icu in Regex in Locale was a major event. So from the point of view of somebody who comes from outside it is bare practicality and simplicity of using/configuring the dependencies.

...

Bottom line, in my opinion bjam is a very limited build system. Maybe it servers Boost ok, but I would never choose one outside the scope mostly due its design of doing everything in the same flow under all conditions rather than the much simpler and easy to use CMake approach.

I can agree that it could be more easy to think in exactly one configuration. Which makes writing b2 build scripts properly more challenging than CMake build scripts. But ironically again, this part of b2 complexity also gradually enters CMake due to multiconfig generators (just look at generator expressions). Nevertheless, I find it strange that you call b2 very limited compared to CMake, when it's actually more powerful, and it's exactly that power that incurs complexity.

I understand what you are talking about because for windows build there is a huge matrix of Under Windows {Release,Debug}x{DynamicLink,StaticLink}x{DynamicRuntime,StaticRuntime} And under Linux it is typically shared object and static object - BTW something that autotools did very well and drive me crazy at the beginning with CMake - autotools buid automatically shared object and static while in cmake it is more verbose Now typically under windows you have two Debug and Release runtimes because they are ABI incompatible (not going to why they did such a dreadful decision - but it is another topic) So there is a mode in cmake that tries to build them both - and it does not work well that is why I never support VS projects with dual builds in my cmakes and tell windows users grab ninja and do a separate build for each of the debub or release configurations. So yes for building crazy matrices bjam helps a lot I totally agree - but all this till you need to find appropriate dependency for each and every case and turn on/off a feature if you found it or not - and not it becomes a nightmare since you may found ICU for dynamic linking in release but not for static linking or not for debug. So I do understand practicality of Boost Build bjam for Boost itself (as long as you don't need any 3rd party) but I find it rather unpractical for outside of the Boost crib Artyom

On Tue, May 6, 2025 at 12:52 AM Artyom Beilis via Boost < boost@lists.boost.org> wrote:

...

. I am proposing to replace b2.

While I can agree that b2 isn't particularly good build system by any stretch of imagination

There was a long and yet incomplete effort of moving to CMake that most around there are familiar with. CMake works very well and is very well supported all around.

Replacing one not-so-good homebrew system that was at least polished with yet another homebrew system is a horrible idea.

Boost needs _less_ of "non-invented-here-syndrome": not _more_ of it

This actually seems like a fantastic time to shill this Python script I've been working on. It lives here: https://github.com/cmazakas/boost/blob/c2py/c2.py This is intended to be a full drop-in replacement for b2. Right now, it only works on Linux but that's because I want a quality implementation before I work on a cross-platform solution. I'm actually chatting with one of the CMake devs on slack about how to alter the generation of the Ninja files. I might just manually patch them myself in the interim though. You can invoke it similarly to b2 today: ./c2.py \ hash2 test \ --cxxstd 11,20 \ --variant debug,release \ --address-model 64 \ --toolset clang-19,gcc-14 \ --asan --ubsan \ --ctestflags="--output-on-failure -R 'xxh3'" \ --no-cmake \ -j20 What this tool does is, it generates a series of CMake build directories under `build_c2py`, configuring each project in parallel and then builds the tests in parallel using recursive Make invocations. It's not as fast as b2 is but because it uses CMake, it really, really opens up the door for us. I've been thinking about looking for beta testers. I use this script daily to drive my work with the Alliance. The script is woefully incomplete but I think it's a good first step. - Christian

On Tue, May 6, 2025 at 7:46 AM Peter Dimov via Boost <boost@lists.boost.org> wrote:

Christian Mazakas wrote:

...

You can invoke it similarly to b2 today:

./c2.py \ hash2 test \ --cxxstd 11,20 \ --variant debug,release \ --address-model 64 \ --toolset clang-19,gcc-14 \ --asan --ubsan \ --ctestflags="--output-on-failure -R 'xxh3'" \

--output-on-failure (and --no_tests=error) should be the default, really.

Great call! I can update the script to do that trivially.

...

--no-cmake \

What does --no-cmake do?

I should update the name but it makes the script skip the CMake configuration phase. Right now, the script uses a very simple and naive mangling scheme for creating the names of build directories. A goal of mine was to have them be quasi-human readable so that way a frustrated user could intuitively inspect a specific build configuration so like b2, I don't include any passed cxxflags in the mangled name. To make sure that an accurate configuration is always used, I delete the existing CMakeCache.txt if it was there, which means a full re-configuration by CMake. `--no-cmake` is a flag used to skip this part and instead `make -C build_c2py` is invoked directly.

...

-j20

Ha, in the ideal world I suppose we'd check for the available number of CPUs and default to the max available. This is the value passed down to the Make invocation. - Christian

It seems like a feature gap that can be fixed at CMake level - either by cmake developers or by external patch.

Have you considered talking with CMake developers about it and/or submitting a patch? What do they think about this?

This is a little funny. If CMake developers can not, how can I? I can only talk about how my build-system supports header-units. Also, I should not have said, "CMake can not support C++20 header-units". Anything is possible. Header-Units support is built on the following four features in HMake. 1) Next-Gen build-algorithm HMake build-algorithm support both dynamic nodes and dynamic edges. CMake supports modules through Ninja, which only supports dynamic edges. Example6 and Example7 in HMake Architecture section demonstrate this capability. 2) Vertical Integration HMake is both a build-system generator and a build-system itself. Ninja supports dynamic edges through a separate process. While building modules, it launches a new process that consumes all of the scanning output and outputs a file in Ninja consumable format. Ninja then consumes this file to learn about the dynamic edges. While HMake does all of this itself. Suppose Ninja supports dynamic nodes, and even then, to ensure a header unit is built before the consumer, it will need to launch a new process to establish the dependency. It will have to do this every time it discovers a new header-unit. And even then, it will have to determine what compile-command to use with that header-unit. Suppose the user wants to build ASIO with ASIO_STANDALONE. Only the ASIO header-units should be built with this macro. ASIO header-units can potentially be discovered in any of its dependent targets. HMake goes over the list of header-unit-includes to determine the directory from which this header-unit is coming. And then the mapping of the directory to target to determine the cpptarget to attach this header-unit to. i.e the target whose compile-command will be used to compile this header-unit. Ninja lacks the context to do all this. CMake will have to turn to a build-system itself to support header-units. 3) Game-Changing Feature, Node HMake has another feature called Node (explained in documentation). For a module file, to support incremental build, HMake caches its header-includes, module-deps, header-unit-deps, compile-command, scanning-command. Without the Node feature, this cache will explode. Node feature results in little build residue. I estimate that for full Boost, for full Configuration, it will be 1MB. This ensures ultra-fast loading, ultra-fast performance. 4) C++ While this is often contended, I think this is extremely important. E.g., now I plan to remove the scanning support and add scanning-less support. C++ allows for quick iteration. It allows one to focus on the problem instead of the DSL. All in all, as it stands, this is a fundamental change. Best, Hassan Sajjad On Tue, May 6, 2025 at 12:52 PM Artyom Beilis via Boost < boost@lists.boost.org> wrote:

...

. I am proposing to replace b2.

While I can agree that b2 isn't particularly good build system by any stretch of imagination

There was a long and yet incomplete effort of moving to CMake that most around there are familiar with. CMake works very well and is very well supported all around.

Replacing one not-so-good homebrew system that was at least polished with yet another homebrew system is a horrible idea.

Boost needs _less_ of "non-invented-here-syndrome": not _more_ of it

Refrasing Stroustrup there are two types of build systems - everybody complains about and nobody uses, and cmake is one everybody is complaining about ;-)

...

I mean today CMake is the most common industry standard build system

...

that works really well. Why would you want to use it over something like this?

CMake can not support C++20 header-units. One benefit that I forgot is that once you have compiled your project with header-units in HMake and you want to move to C++20 modules,

It seems like a feature gap that can be fixed at CMake level - either by cmake developers or by external patch.

Have you considered talking with CMake developers about it and/or submitting a patch? What do they think about this?

I'm sure they are aware of the problem :-)

...

By caching the results and not running tests every time. Also, configure.exe uses the same code that is used by build.exe. It is fully parallel as well.

You don't reconfigure each and every build... cmake caches _a lot_ sorry but it seems to be a false statement.

Best Artyom

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Mon, Jun 30, 2025 at 3:36 AM Daniele Lupo via Boost < boost@lists.boost.org> wrote:

On 05/05/2025 23:45, Hassan Sajjad via Boost wrote:

...

...

Generally speaking your "benefits list" has quite bold statements

Are you proposing replacement of b2 for Boost or just yet another build system to the club?

These statements are based on concrete evidence. I am proposing to replace b2.

If I remember well, this is not the first time that you want to propose it, and I was against it for different reasons:

- We are already trying to replace the build system with cmake. - CMake can have pros and cons, but actually it's one of biggest standard in C++ development. Changing a "boost only" build system for another system that will used only for boost (I don't see hmake widely used) it's pretty useless.

Fwiw, I've made some good progress on this front and I've come up with this tool here: https://github.com/cmazakas/c2 This is a pure CMake b2-like runner for tests. Because it supports arbitrary CMake targets, it can be extended even for something like docs. - Christian

On Tue, Jul 1, 2025 at 11:23 AM Matt Gruenke via Boost < boost@lists.boost.org> wrote:

On Monday, June 30, 2025 at 16:42, Christian Mazakas via Boost < boost@lists.boost.org> wrote:

...

Fwiw, I've made some good progress on this front and I've come up with this tool here: ...

This is a pure CMake b2-like runner for tests.

Regarding c2, I wonder if ctest been ruled out? It integrates nicely with CMake and has its own test runner. Maybe it doesn't do everything you need or want, but perhaps it would be easy enough to enhance?

c2 dispatches to ctest to invoke everything. There's actually a couple of caveats with how I did things that are probably at least worth mentioning. But in general, c2 is just a glorified script that does: cmake .. && cmake --build . && ctest . Because of how I have to change some of the output ninja files, I inadvertently broke the compile-fail tests Peter's authored in BoostTest.cmake. I was able to fix it, by having c2 pass a c2-specific config variable that the Boost CMake could inspect and then use to do the correct thing. So far, I haven't run into any actual show-stoppers and it seems to work well for Boost. What's most important about the tool is that it lets you define your build matrix using a simple JSON file. And you can also specify a toolchain file per build variant directly. - Christian

Hi Marcin,

Has this feature been implemented in any compiler? Upstreaming such a change to LLVM would demonstrate that it is mature enough to be used in production software and also provide a stamp of approval.

Not yet. This is the next step. I can not add no-scanning build support without this, without which there can not be a useful drop-in header-units replacement. It won't be easy to convince the community to switch to a new build system

that requires modification to the C/C++ compiler. These are often unlikely to be adopted unless there is a significant demand for that feature in the community.

The changes required are minimal. Check the README in this repo https://github.com/HassanSajjad-302/ipc2978api. I also added small modifications to the paper. I am confident that Clang will accept the changes. Here is a comment from Clang dev and Tooling Study Group chair https://www.reddit.com/r/cpp/comments/1jb8acg/comment/mhvvfva/?utm_source=sh... I would also like to add that this isn't related to the design of modules.

Despite lots of claims, I have never seen a proposed design that would actually be any easier to implement in reality. You can make things easier by not supporting headers, but then no existing code can use it. You can also do a lot of things by restricting how they can be used, but then most projects would have to change (often in major ways) to use them. The fundamental problem is that C++ sits on 50+ years of textual inclusion and build system legacy, and modules requires changing that. There's no easy fix that's going to have high perf with a build system designed almost 50 years ago. Things like a module build server are the closest, but nobody is actually working on that from what I can tell.

Best, Hassan Sajjad On Tue, May 6, 2025 at 7:07 PM Marcin Copik <mcopik@gmail.com> wrote:

Hi Hassan,

8) I wrote this paper

...

https://htmlpreview.github.io/?https://github.com/HassanSajjad-302/iso-paper... . It is at most a 2-day work for a compile-dev to support this. This will allow 4x compilation speed improvement for Boost and even bigger gains for bigger projects.

Has this feature been implemented in any compiler? Upstreaming such a change to LLVM would demonstrate that it is mature enough to be used in production software and also provide a stamp of approval.

It won't be easy to convince the community to switch to a new build system that requires modification to the C/C++ compiler. These are often unlikely to be adopted unless there is a significant demand for that feature in the community.

Best, Marcin

...

9) HMake will support a feature called BigHu (described in previous email), which makes rebuilds incredibly fast as well. Also, I think header-units can be backported for older C++ versions as well, since they change semantics very little ( mostly around the preprocessor ). 10) HMake will also support https://lists.isocpp.org/sg15/2023/11/2146.php and https://lists.isocpp.org/sg15/2023/11/2106.php if these get implemented. 11) compile-commands.json support. HMake will generate these at configure time for the configurations being configured. 12) CMake integration support through CPS. Some features, like header-units might not work. 13) HMake will support multiple other programming languages, frameworks like iOS and Android app generation, different installers, etc.

HMake is still a novel build-system that is under heavy development. A project of Boost size using this in production will provide highly valuable insights that will tremendously improve the HMake itself as well. HMake is written in C++20, follows best practices, is extensively tested, uses TDD, and the core library is around 15k C++ lines.

I can also do a video call presentation of my software. This might be a quicker way to make yourself familiar with my software capabilities than reading documentation. Because I can do live examples and explain details accordingly.

Best, Hassan Sajjad

On Wed, Apr 16, 2025 at 5:21 PM Hassan Sajjad <hassan.sajjad069@gmail.com

...

wrote:

...

The interactions to reproduce the results.

https://github.com/HassanSajjad-302/HMake/blob/main/Projects/Boost/hmake.cpp

...

On Wed, Apr 16, 2025, 16:34 Hassan Sajjad <hassan.sajjad069@gmail.com> wrote:

...

Hi.

I compiled 25 Boost libraries with C++20 header-units, including

examples

...

...

and Tests of a few. This resulted in a 1.44x speed-up if we exclude the scanning. Scanning was as slow as the header-unit build.

I have a solution for that. I am sending this paper (Making some final edits)

https://htmlpreview.github.io/?https://github.com/HassanSajjad-302/iso-paper...

...

, which allows for a build-system to use a scanning-less approach to build header-units.This method requires only minimal modifications to the compiler and enables memory-mapped file sharing, which significantly reduces filesystem overhead and further boosts speed.

My proposal for a scanning-less approach will allow HMake to have another amazing feature, monolithic Big header-units. BigHu will allow you to map all the headers from a directory to a single header-unit. This header-unit includes all these header-files and consumes them as header-files. Now, whenever the compiler requests a header-unit from this directory, the build-system instead sends this consolidated BigHu. This will not only improve the clean build speed, because of fewer process launches. But also massively boosts the rebuild speed as few header-units need to be read instead of lots of files. No source-code editing will be needed to use this feature. And you will have the option to either build a library as BigHu (if you don't mean to edit it) or as small header-units (if you are working on it).

HMake will soon support compile_commands.json, which I guess can work fine with header-units. Thus you can benefit from both, faster compilation and working intelliscence.

The API shown in hmake.cpp is a work in progress. HMake build-system benefits are numerous, even if we leave out the header-units support. This makes it a perfect replacement for B2. I believe we can complete the switch in the next 2 months. And if some compiler supports my paper, we get 3x faster compilation as well.

Best, Hassan Sajjad

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

CMake is very "unsafe" build system. CMake often break compatability and make many problem on stable server systems like Debian, RHEL, CentOS and etc, where CMake is too old alltime. I think CMake may be alternative build sytem, but not common. -- wbr, Leonid

wt., 1 lip 2025 o 14:21 cppleo--- via Boost <boost@lists.boost.org> napisał(a):

CMake is very "unsafe" build system. CMake often break compatability and make many problem on stable server systems like Debian, RHEL, CentOS and etc, where CMake is too old alltime.

I think CMake may be alternative build sytem, but not common.

CMake has a convenient system of binary releases that are easy to install without requiring root permissions. Updating CMake to the latest version is rather straightforward, even on HPC clusters that often run ancient software. Best, Marcin

-- wbr, Leonid _______________________________________________ Boost mailing list -- boost@lists.boost.org To unsubscribe send an email to boost-leave@lists.boost.org https://lists.boost.org/mailman3/lists/boost.lists.boost.org/ Archived at: https://lists.boost.org/archives/list/boost@lists.boost.org/message/Q33SW4HV...

I'm confused, where can people download B2 binaries? On Jul 1, 2025 11:52 AM, cppleo--- via Boost <boost@lists.boost.org> wrote: Latest Release (4.0.3): Binary distributions: Windows x64 Installer: Windows x64 ZIP Windows i386 Installer: Windows i386 ZIP Windows ARM64 Installer: Windows ARM64 ZIP macOS 10.13 or later macOS 10.10 or later Linux x86_64 Linux aarch64 SunOS sparc64 SunOS x86_64 Is that all? Where is FreeBSD binaries? Where is NetBSD binaries? Where is non-x86 and non-ARM binraies for Linux? What about legacy systems works on factories (Windows 2000, OS/2 (ArcaOS))? The FreeBSD is very popular server OS. Legacy systems will continue work. If you ignore them, they ignore you. (How do you think what is chipper: halt a rolling mill or keep use legacy system with Windows NT 4.0, Windows 2000, OS/2, Novell NetWare, etc?) Removing C++03 compatability from many libraries and breaking Boost.ASIO in 1.87 seriously damaged the Boost reputation. If Boost break building via b2 (non-CMake), Boost will damage reputation again. -- wbr, Leonid _______________________________________________ Boost mailing list -- boost@lists.boost.org To unsubscribe send an email to boost-leave@lists.boost.org https://lists.boost.org/mailman3/lists/boost.lists.boost.org/ Archived at: https://lists.boost.org/archives/list/boost@lists.boost.org/message/TQ3QIDYD...

On 02/07/2025 17:04, Christian Mazakas via Boost wrote:

On Tue, Jul 1, 2025 at 10:52 AM cppleo--- via Boost <boost@lists.boost.org> wrote:

...

Legacy systems will continue work. If you ignore them, they ignore you. (How do you think what is chipper: halt a rolling mill or keep use legacy system with Windows NT 4.0, Windows 2000, OS/2, Novell NetWare, etc?)

Removing C++03 compatability from many libraries and breaking Boost.ASIO in 1.87 seriously damaged the Boost reputation. If Boost break building via b2 (non-CMake), Boost will damage reputation again.

Also, I'd like to see some citations for this. At least for Math and Multiprecision, I don't think think we have ever had a bug report submitted asking us to reinstate C++03 (or even C++11) support.  On the other hand we pretty consistently get reports from folks using bleeding edge features and compilers, and (rightly) complaining when they don't work.  It's actually surprising how much stuff has been broken by new C++ standards: it's mostly things getting marked as deprecated (float_denorm_style is the next one for us to sort out), but there were changes to the preprocessor for C++23 which have hit us too.

On a practical note, it's simply not feasible to support every possible language standard, and certainly not possible for a larger library to rigorously CI them all. Best, John.

On 07/07/2025 14:45, Artyom Beilis via Boost wrote:

Boost breaking compatibility is the Achilles heel of Boost since I remember it (around 1.33) and it had never changed. It makes Boost highly problematic - almost irrelevant for any long term C++ project that needs to run over a large range of platforms that provide its own Boost. More than that, Boost is almost impossible to use in libraries or other interfaces because of breaking API/ABI virtually in any situation.

I've said this many times. This should not be an issue. Every library at some point breaks with the past and go ahead, for many reason, maintanance, readability etc... And for that reason we also have the X.Y.Z version numbering where - Z: number updated when there are minor bugfixes but no API changes - Y: number updated when there are API changes and updated of the featured but maintaining API compatibility - X: number updated when there are API changes that make the library not compatible with previous versions. It's true that there's a lot of legacy code that uses boost and cannot be upgraded to new compilers or standards, but it's also true that they can remain stuck with their own boost version, the one that works. In my opinion at some point it will be necessary to break compatibility with old standards and compilers, update boost, remove old legacy code that at this point will be useless and start Boost 2.0. Users of old versions can use the 1.Y.Z version of boost, they don't stop working because a new version is out, and it's possible to decide a maintanance period of time, i.e. one year, for which we can release 1.Y.ZZ versions that fixes bugs in this main branch of boost, if needed. It's not different from about many other companies do with their libraries, and this will allow a migration to a more efficient boost that must do not handle a lot of compilers and standards and architectures, and maintain the old version for an amount of time that we can decide.

...

If Boost break building via b2 (non-CMake), Boost will damage reputation again. [Sarcasm ON] I don't think anybody but Boost maintainers will notices [Sarcasm Off]

Maybe. A lot of people uses b2 for building boost, instead of using vcpkg, conan and so on. Daniele

It Bootstrapped C++11 but for long term projects Boost nowadays isn't very useful.

Since this was with respect to keeping things ABI stable. In my view, boost accepting ABI breakage is the very reason that it stays relevant. The standard library for some popular compilers are stuck in the past while there are known improvements which are not implemented due to ABI stability. This makes the standard library way less useful for long-term projects, since its known that they will never improve, or even fix issues in case it potentially changes the ABI. See for example std regex or std unordered map. Boost implementing improvements regardless of ABI is one of the great things with Boost.

On Monday, July 7, 2025 14:06, Jakob Lövhall wrote:

Since this was with respect to keeping things ABI stable. In my view, boost accepting ABI breakage is the very reason that it stays relevant.

Just out of curiosity: are you developing on Windows or Linux? For Linux devs, the issue is that if we link in some 3rd library supplied by the distro, and it happens to link in some Boost libs, then we must either take the distro version of Boost for our own code, rebuild the other distro-provided libraries we want to use that link in Boost, or just avoid the mess and forego using Boost in our code. I'd bet most of you have run into people or projects that make a concerted effort to avoid using Boost, and I'm sure that's one of the reasons. ABI compatibility is indeed a high bar, but I see value in at least trying to manage when the breaking changes happen. Not that I'd expect Boost to change now, but just wanted to share my experience with the matter. Matt

On Monday, July 7, 2025 3:40 PM, Jakob Lövhall wrote:

It is more like a fence, or wall, blocking improvement imho.

It doesn't block all improvements, nor would I envision a scenario where strict ABI compatibility is required over an extended range of releases. I merely suggested that ABI-breaking changes might be managed in some way, rather than being completely ad hoc. One way they could be managed is with versioned namespaces. Another way would be to adhere to proper semantic versioning and then space out the releases with ABI and API -breakage a little more.

Sure sometimes libraries are built with a earlier version that one would like. It sucks to be on that earlier version when its known that there is some dope feature in a newer version.

I think we're mostly saying the same thing, here. If ABI-incompatible versions of Boost were fewer and farther in between, then you could usually upgrade to a newer version of Boost and utilize some newer features, even when linking against some 3rd libraries that also use Boost. They wouldn't hold you back so strictly, like they do today.

at some point in the future, dependencies could update and you get improvements.

Distros try to avoid incompatible library dependency changes, [i]precisely because[/i] they can ripple through library dependency chains like a bunch of dominos. The exceptions are when you're on a rolling-release distro or you perform a wholesale upgrade of a stable distro.

With ABI stability you could upgrade right away (if the issue is specifically dynamic linkage to targets with non hidden symbols), but pay the cost of not having the same improvements.

Right. You'd be limited in the nature and extent of improvements you could attain, without jumping to a new major version. I do wonder how often ABI-incompatible changes are really necessary. There's also an uncounted benefit, which is the potential for more libraries (esp. open source) to use Boost, if some of the current downsides of doing so were mitigated. Unfortunately, I think that ship has pretty much sailed. Matt -----Original Message----- From: Jakob Lövhall <lovhall@protonmail.com> Sent: Monday, July 7, 2025 3:40 PM To: Boost developers' mailing list <boost@lists.boost.org> Cc: Matt Gruenke <mgruenke@tycoint.com> Subject: Re: [boost] Re: API Stability (was: A proposal for a modern build-system_

Just out of curiosity: are you developing on Windows or Linux?

Linux

ABI compatibility is indeed a high bar, It is more like a fence, or wall, blocking improvement imho.

Sure sometimes libraries are built with a earlier version that one would like. It sucks to be on that earlier version when its known that there is some dope feature in a newer version. But, then sure, be stuck on that earlier version. Then you get all the functionality from that version, but on the other hand, at some point in the future, dependencies could update and you get improvements. With ABI stability you could upgrade right away (if the issue is specifically dynamic linkage to targets with non hidden symbols), but pay the cost of not having the same improvements. Jakob

On Mon, Jul 7, 2025 at 11:51 AM Matt Gruenke via Boost < boost@lists.boost.org> wrote:

On Monday, July 7, 2025 14:06, Jakob Lövhall wrote:

...

Since this was with respect to keeping things ABI stable. In my view, boost accepting ABI breakage is the very reason that it stays relevant.

Just out of curiosity: are you developing on Windows or Linux? For Linux devs, the issue is that if we link in some 3rd library supplied by the distro, and it happens to link in some Boost libs, then we must either take the distro version of Boost for our own code, rebuild the other distro-provided libraries we want to use that link in Boost, or just avoid the mess and forego using Boost in our code.

I'd bet most of you have run into people or projects that make a concerted effort to avoid using Boost, and I'm sure that's one of the reasons.

ABI compatibility is indeed a high bar, but I see value in at least trying to manage when the breaking changes happen. Not that I'd expect Boost to change now, but just wanted to share my experience with the matter.

Isn't this overcome by just statically linking in the Boost code you need specifically and then making sure none of the symbols are publicly visible? - Christian

This isn't my area of expertise, but a workaround for incidental shared library symbol collisions appears to exist in the form of ELF symbol versioning. I think this could enable multiple versions of boost to be linked into the same executable, without the symbols from one overriding any symbols from the other. If my understanding is correct, this could provide the benefits you sought from static linking, without foregoing the maintenance advantages of dynamic linking. Not that it would alleviate most of the issues stemming from ABI instability, but I think it would address some cases.

This is easier to do at the source code level by placing exported symbols in a detail namespace whose name is mangled with the Boost version: this is what regex does, and I suspect other libraries too.  It has the side effect of reducing support requests from people who accidentally try and link to the wrong library version, as well as allowing to different versions to co-exist. John.

In article <CAOsONKz8s8OvWQE09xceK6w9J=zhW=sR2Hbqu0=AAeZDWOYAqw@mail.gmail.com>, "Artyom Beilis via Boost" <boost@lists.boost.org> writes:

It is very hard to use Boost in any large scale project without something like that.

You have all the sources and a permissive license. You can hire an engineer to modify the last version of boost libraries you find acceptable as an ABI compatible baseline to incorporate whatever improvements you desire under the constraints you require. In other words, there is no barrier that prevents you from doing what you want to do. Considering the cost of creating boost libraries from scratch that were provided to anyone at no cost with no restrictive licening, paying someone to make improvements on top of such a foundation is an amazing bargain. So I guess I don't understand the problem. -- Richard -- "The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline> The Terminals Wiki <http://terminals-wiki.org> The Computer Graphics Museum <http://ComputerGraphicsMuseum.org> Legalize Adulthood! (my blog) <http://LegalizeAdulthood.wordpress.com>

...

And the rest of the updates are backward API/ABI compatible.

ABI compatibility is a delicate matter. Yes, I agree that for some long-term project it can be helpful, but it can also limit the development of libraries a lot. We all know about std::regex and its limits due to the fact that it needs to maintain ABI compatibility.

There are lots of ways to handle ABI compatibility (Qt's d-pointer is classic) But we are not even in an API area. API is broken virtually every release. It is a huge problem for a long running project especially when you have no idea what version of Boost you are going to be linked with. I would accept at least API stability - but it does not exist. Artyom

Artyom Beilis wrote:

But we are not even in an API area. API is broken virtually every release.

I'm not sure that's quite correct, although it would obviously depend on how you measure "brokenness". "API" is "broken" even within the same release if you upgrade your compiler or you -std level. At the same time, more than half of all Boost libraries hardly change at all, so they can't be breaking API. But in addition to that, some libraries do break user code, but not because the API changed; rather, a PR or a fix introduces a bug or an incompatibility.

I’m happy to be wrong but don’t suspect that creating a new build system from scratch (without massive commercial backing) is likely to be a path of minimal resistance.

I stay motivated because HMake has a lot of benefits compared to the competitors. Commercial backing will indeed be seriously helpful. On Tue, May 6, 2025 at 6:43 PM Jonathan Coe via Boost <boost@lists.boost.org> wrote:

Hi,

Sharing my thoughts so that the HMake author does not invest time unduly.

Even with exceptional and detailed documentation, this feels like a high-risk change that’s very unlikely to take place without evidence of significant adoption elsewhere. Build systems are hard and most folk seem to prefer familiar foibles over innovation.

If the build system space is interesting, it might be worth spending time improving existing tools (tooling around CMake is challenging) rather than starting from scratch.

I’m happy to be wrong but don’t suspect that creating a new build system from scratch (without massive commercial backing) is likely to be a path of minimal resistance.

J

...

On 6 May 2025, at 13:26, Дмитрий Архипов via Boost < boost@lists.boost.org> wrote:

ср, 16 апр. 2025 г. в 15:22, Hassan Sajjad via Boost < boost@lists.boost.org>:

...

Hi.

I compiled 25 Boost libraries with C++20 header-units, including

examples

...

and Tests of a few. This resulted in a 1.44x speed-up if we exclude the scanning. Scanning was as slow as the header-unit build.

If this is a request for endorsement, then the problem is that the tool/library in question is not really documented. First, compare your README with CMake's tutorial

https://cmake.org/cmake/help/latest/guide/tutorial/A%20Basic%20Starting%20Po...

...

or B2's tutorial https://www.bfgroup.xyz/b2/manual/main/index.html#b2.tutorial. Both explain what file to create, what to put in there, what command to invoke from the shell. Your simple C++ example (https://github.com/HassanSajjad-302/HMake?tab=readme-ov-file#c-examples ) starts in the middle of the file, and is definitely more hand-wavy. After reading the paragraphs following the code snippet I am not sure what commands I need to invoke.

So, a supposed benefit of HMake is that you don't need to learn a new syntax. But learning a new library's API can be a significant task on its own.

void updateBTarget(class Builder &builder, unsigned short round) override

What is a "round" here? I am pretty well-versed in build system lingo, and I have no idea. In the following paragraph you explain that it is related to how C++ modules are built (in HMake and in CMake). There are no "rounds" in C++ modules spec or in CMake's document on how to use it with modules. So, this concept is particular to HMake. This highlights the fact that you need a thorough documentation of the API. WIthout it your tool/library is unusable in a serious scenario.

Let's now compare a minimal C++ project in CMake, B2 and HMake.

// HMake #include "Configure.hpp" void configurationSpecification(Configuration &config) { config.getCppExeDSC("app").getSourceTarget().sourceFiles("main.cpp"); } void buildSpecification() { getConfiguration(); CALL_CONFIGURATION_SPECIFICATION } MAIN_FUNCTION

# CMake cmake_minimum_required(VERSION 3.12) project(Tutorial) add_executable(app main.cpp)

#B2 exe app : main.cpp ;

I don't see how HMake is an improvement. It's neither shorter nor more readable. For example, waht is "DSC"? This also highlights a common annoyance with using a regular programming language for build scripts: build scripts connect sources with targets, their names are strings, so you have to use a lot of quoting.

Finally, let's consider getConfiguration. From the README I gather that the build script has to contain a description of every configuration that the project supports. That is, users can't experiment with a new configuration without modifying the build script. This is completely inadequate in my opinion. It is particularly inadequate for open source software projects. From the same description I gather that hmake also has to create a dedicated directory for each configuration. Note that the current main build system Boost uses (B2) does not work like that. It creates configurations on the fly from the user's build request and the targets' requirements. There's also no "active configuration" in B2, it can build multiple configurations simultaneously.

My main point is still the lack of documentation. Without documentation there could be no discussion of endorsement, as we simply cannot evaluate what to endorse.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

ср, 7 мая 2025 г. в 01:25, Hassan Sajjad <hassan.sajjad069@gmail.com>:

Specifying new Configurations is very easy. Like boost/hmake.cpp had the following two configurations defined to benchmark the debug build with header-units.

getConfiguration("conventional").assign(TreatModuleAsSource::YES, ConfigType::DEBUG); getConfiguration("hu").assign(TreatModuleAsSource::NO, TranslateInclude::YES, ConfigType::DEBUG);

Creating build-dirs at build-time will be very slow. HMake creates these in parallel at configure-time.

The issue here is not that it's hard to create new configurations. The issue is that the project's build scripts aren't an appropriate place for this. I shouldn't need to change files that are checked into version control system in order to just build the project for my system.

This is a recurrent complaint, so I concede. But I don't know where to start. I will be happy if you provide more ideas.

Meson (https://mesonbuild.com) is a relatively new build system. It has fairly well-organised and pretty documentation. You can start by copying its basic structure. You also definitely need a reference documentation for your library.