[Boost-build] [future] Implementation language(s)..

Subject: [Boost-build] [future] Implementation language(s)..
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2016-10-18 09:59:36

Before we too far ahead of ourselves we have one particular decision to
make.. What do we write the next Boost Build in? Starting with the rough
modules.. It shows some possibilities:
Project Descriptions DSL (i,e, Jam), Python, Lua
Extensions C++, Python, Lua, Haskel
Build System C++, Python, Lua, Haskel
Target Graph C++
Scheduler C++
Executor C++
And, of course, more than one answer is possible. I only listed the ones
that seemed obvious to me. And the pros/cons are specific to each module.
So going from top-to-bottom..

== Project Descriptions ==

* DSL/Jam
Pros: It's known to devs and users; It's easy to script *small* tasks; We
can fix & adjust warts as we see fit; It fits well the task of *declaring*
build targets.
Cons: It has all the warts we know about; Essentially no tool (ie
IDE/editor) support.

* Python
Pros: Full language; Many built-in utilities; It's known to many; There are
many tools to support it; Built-in many OSs, easy to install in others.
Cons: Not a good fit for declaring targets; Large temptation to put lots of
logic at the project declaration layer; Too big to embed (needs external
install and linking); Heavyweight run-time, and hence slower startups.

* Lua
Pros: Full language; It's known to many; Easy to embed (ie no external
install needed); Good tool support (syntax coloring in most editors);
Lightweight run-time, hence fast startups.
Cons: Not well known in this group; Unknown if it would fit declaring
targets.

== Extensions (ie toolsets, external lib config, etc.) ==

I'm not going to include DSL/Jam in the choices here, because I think we
all agree that this is one of the key pain points in the current design.

* C++
Pros: We all know it well; Excellent tool support; Performance; If build
system is in C++ would provide direct access to it; Likely easier to
integrate with project descriptions layer (if needed).
Cons: Rarely built-in OS tools (but required for at least C++ projects
anyway); No common existing package distribution service.

* Python
Pros: Full language; Many built-in utilities; It's known to many; There are
many tools to support it; Built-in many OSs, easy to install in others;
Existing package distribution (pip); We can likely leverage some of the b2
Python port code; Packages support both native C/C++ and Python.
Cons: Too big to embed (needs external install and linking); Heavyweight
run-time, and hence slower startups.

* Lua
Pros: Full language; It's known to many; Easy to embed (ie no external
install needed); Good tool support (syntax coloring in most editors);
Lightweight run-time, hence fast startups;
Cons: Not well known in this group; Not a lot of built-in libraries; No
existing packaging system.

== Build System ==

* C++
Pros: We all know it well; Excellent tool support; Performance; Likely
easier to integrate with project descriptions layer (if needed).
Cons: Rarely built-in OS tools (but required for at least C++ projects
anyway); No existing package distribution service.

* Python
Pros: Full language; Many built-in utilities; It's known to many; There are
many tools to support it; Built-in many OSs, easy to install in others;
Existing package distribution (pip); We can likely leverage some of the b2
Python port code; Packages support both native C/C++ and Python.
Cons: Too big to embed (needs external install and linking); Heavyweight
run-time, and hence slower startups.

== Runtime ==

Collecting all the rest of the modules into one runtime.

* C++
Pros: We all know it well; Excellent tool support; Performance; Likely
easier to integrate with project descriptions layer (if needed).
Cons: Rarely built-in OS tools (but required for at least C++ projects
anyway); No existing package distribution service.

* Python
Pros: Full language; Many built-in utilities; It's known to many; There are
many tools to support it; Built-in many OSs, easy to install in others;
Existing package distribution (pip); We can likely leverage some of the b2
Python port code; Packages support both native C/C++ and Python.
Cons: Too big to embed (needs external install and linking); Heavyweight
run-time, and hence slower startups.

First, if there are other languages you think we should consider please
speak up. Note, I've not included Haskel here, even though I listed in the
wiki. As I think, after some research, that it's not worth seriously
considering.

Second, what I recommend.. Given the above what I'm thinking we should do
it is:

Project Description: DSL/Jam. It's served well for this task before and we
know its characteristics.
Extensions: Python. It's flexibility and packaging system would be ideal.
Build System: C++.
Runtime: C++.

This essentially means that most of the system would be in C++. In
particular I would suggest that we write the "Build System" and lower
modules/components as a shared library that can be used from Python, or any
other language that has an FFI. That way we make it easier to integrate
into IDEs, thinking of them as front-ends to the build system. Also one
major advantage is that we can publish the entire thing, ie with the CLI
front-end, as a Python pip installable package.

-- 
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
-- 
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail