Re: A proposal for a modern build-system

From: Hassan Sajjad (hassan.sajjad069_at_[hidden])
Date: 2025-05-07 02:57:01

Hi Marcin,

> Has this feature been implemented in any compiler? Upstreaming such a
> change to LLVM would demonstrate that it is mature enough to be used in
> production software and also provide a stamp of approval.
>

Not yet. This is the next step. I can not add no-scanning build support
without this, without which there can not be a useful drop-in header-units
replacement.

It won't be easy to convince the community to switch to a new build system
> that requires modification to the C/C++ compiler. These are often unlikely
> to be adopted unless there is a significant demand for that feature in the
> community.
>

The changes required are minimal. Check the README in this repo
https://github.com/HassanSajjad-302/ipc2978api. I also added small
modifications to the paper. I am confident that Clang will accept the
changes. Here is a comment from Clang dev and Tooling Study Group chair
https://www.reddit.com/r/cpp/comments/1jb8acg/comment/mhvvfva/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

I would also like to add that this isn't related to the design of modules.
> Despite lots of claims, I have never seen a proposed design that would
> actually be any easier to implement in reality. You can make things easier
> by not supporting headers, but then no existing code can use it. You can
> also do a lot of things by restricting how they can be used, but then most
> projects would have to change (often in major ways) to use them. The
> fundamental problem is that C++ sits on 50+ years of textual inclusion and
> build system legacy, and modules requires changing that. There's no easy
> fix that's going to have high perf with a build system designed almost 50
> years ago. Things like a module build server are the closest, but nobody is
> actually working on that from what I can tell.

Best,
Hassan Sajjad

On Tue, May 6, 2025 at 7:07 PM Marcin Copik <mcopik_at_[hidden]> wrote:

>  Hi Hassan,
>
> 8) I wrote this paper
>>
>> https://htmlpreview.github.io/?https://github.com/HassanSajjad-302/iso-papers/blob/main/generated/my-paper.html
>> .
>> It is at most a 2-day work for a compile-dev to support this. This will
>> allow 4x compilation speed improvement for Boost and even bigger gains for
>> bigger projects.
>>
>
> Has this feature been implemented in any compiler? Upstreaming such a
> change to LLVM would demonstrate that it is mature enough to be used in
> production software and also provide a stamp of approval.
>
> It won't be easy to convince the community to switch to a new build system
> that requires modification to the C/C++ compiler. These are often unlikely
> to be adopted unless there is a significant demand for that feature in the
> community.
>
> Best,
> Marcin
>
>
>> 9) HMake will support a feature called BigHu (described in previous
>> email),
>> which makes rebuilds incredibly fast as well. Also, I think header-units
>> can be backported for older C++ versions as well, since they change
>> semantics very little ( mostly around the preprocessor ).
>> 10) HMake will also support
>> https://lists.isocpp.org/sg15/2023/11/2146.php
>> and https://lists.isocpp.org/sg15/2023/11/2106.php if these get
>> implemented.
>> 11) compile-commands.json support. HMake will generate these at configure
>> time for the configurations being configured.
>> 12) CMake integration support through CPS. Some features, like
>> header-units
>> might not work.
>> 13) HMake will support multiple other programming languages, frameworks
>> like iOS and Android app generation, different installers, etc.
>>
>> HMake is still a novel build-system that is under heavy development. A
>> project of Boost size using this in production will provide highly
>> valuable
>> insights that will tremendously improve the HMake itself as well. HMake is
>> written in C++20, follows best practices, is extensively tested, uses TDD,
>> and the core library is around 15k C++ lines.
>>
>> I can also do a video call presentation of my software. This might be a
>> quicker way to make yourself familiar with my software capabilities than
>> reading documentation. Because I can do live examples and explain details
>> accordingly.
>>
>> Best,
>> Hassan Sajjad
>>
>> On Wed, Apr 16, 2025 at 5:21 PM Hassan Sajjad <hassan.sajjad069_at_[hidden]
>> >
>> wrote:
>>
>> > The interactions to reproduce the results.
>> >
>> https://github.com/HassanSajjad-302/HMake/blob/main/Projects/Boost/hmake.cpp
>> >
>> > On Wed, Apr 16, 2025, 16:34 Hassan Sajjad <hassan.sajjad069_at_[hidden]>
>> > wrote:
>> >
>> >> Hi.
>> >>
>> >> I compiled 25 Boost libraries with C++20 header-units, including
>> examples
>> >> and Tests of a few.
>> >> This resulted in a 1.44x speed-up if we exclude the scanning. Scanning
>> >> was as slow as the header-unit build.
>> >>
>> >> I have a solution for that. I am sending this paper (Making some final
>> >> edits)
>> >>
>> https://htmlpreview.github.io/?https://github.com/HassanSajjad-302/iso-papers/blob/main/generated/my-paper.html
>> >> ,
>> >> which allows for a build-system to use a scanning-less approach to
>> build
>> >> header-units.This method requires only minimal modifications to the
>> >> compiler and enables memory-mapped file sharing, which significantly
>> >> reduces filesystem overhead and further boosts speed.
>> >>
>> >> My proposal for a scanning-less approach will allow HMake to have
>> another
>> >> amazing feature, monolithic Big header-units. BigHu will allow you to
>> map
>> >> all the headers from a directory to a single header-unit. This
>> header-unit
>> >> includes all these header-files and consumes them as header-files. Now,
>> >> whenever the compiler requests a header-unit from this directory, the
>> >> build-system instead sends this consolidated BigHu. This will not only
>> >> improve the clean build speed, because of fewer process launches. But
>> also
>> >> massively boosts the rebuild speed as few header-units need to be read
>> >> instead of lots of files. No source-code editing will be needed to use
>> this
>> >> feature. And you will have the option to either build a library as
>> BigHu
>> >> (if you don't mean to edit it) or as small header-units (if you are
>> working
>> >> on it).
>> >>
>> >> HMake will soon support compile_commands.json, which I guess can work
>> >> fine with header-units. Thus you can benefit from both, faster
>> compilation
>> >> and working intelliscence.
>> >>
>> >> The API shown in hmake.cpp is a work in progress. HMake build-system
>> >> benefits are numerous, even if we leave out the header-units support.
>> This
>> >> makes it a perfect replacement for B2. I believe we can complete the
>> switch
>> >> in the next 2 months. And if some compiler supports my paper, we get
>> >3x
>> >> faster compilation as well.
>> >>
>> >> Best,
>> >> Hassan Sajjad
>> >>
>> >>
>> >>
>>
>> _______________________________________________
>> Unsubscribe & other changes:
>> http://lists.boost.org/mailman/listinfo.cgi/boost
>>
>