On Wed, Oct 5, 2016 at 10:19 AM, Dominique Devienne <ddevienne@gmail.com> wrote:

[...] A build system IMHO should have two stages:

1) A high level first stage, driven from a high level, declarative, human friendly syntax,

and input files, the whole lot processed by a smart engine to derive all the minutia of the build itself.

2) A lower level second stage, driven solely by a representation of the "minutia" in some format,

which deals with all the forking, parallelizing, progress reporting, etc...

Another way to look at the above is with an analogy to compiling C++,

which clear separation between the front-end and the back-end.

The first stage (B.B) would be the "compiler".

The second stage (e.g. ninja) would be the "assembler" and "executor".

The "assembly" (or .o/.obj if you prefer) would be the model persisted in SQLite.

And like a compiler, you could generate it w/ or w/o debugging information.

Even SQLite works that way, compiling SQL into VDBE "programs",

which are then "interpreted", sometimes with parallelism even (for sorting, relatively recent addition),

something that's again quite similar to what Ninja does IMHO.

Somehow build tools typically don't take formal routes, w/o any grammars/EBNF, ASTs,

intermediate representations, clear phases and separation of concerns, even though they typically

come from C/C++ developers (especially the Boost ones) very much aware of all this. --DD

 

Now for the build "minutia" format, I'd strongly advocate for SQLite. Because it's small, fast, smart, fantastic IMHO.

It would force to model properly the build state in relational form, and allow any tool to extract information about

the build with just a few queries. No JSON parsing; incrementally updatable; compact since binary.

PS: Interesting (and timely) to read that VS "15" is using SQLite as a cache for its project information.

This is actually not far from what I'm advocating here. FWIW.

https://blogs.msdn.microsoft.com/vcblog/2016/10/05/faster-c-solution-load-with-vs-15/