Re: [boost] [process] The Formal review preiod is nearing the end

Subject: Re: [boost] [process] The Formal review preiod is nearing the end
From: Boris Schäling (boris_at_[hidden])
Date: 2016-11-06 14:59:22

Let me start by saying that I was the maintainer of [Boost.]Process from 
2008 to 2015. [Boost.]Process was created in 2006, and I have seen each and 
every draft of a C++ process management library in Boost since then. I'm 
familiar with all the challenges faced throughout the years and the 
shortcomings of the various solutions found. I participated in each and 
every discussion on a C++ process management library on the Boost mailing 
lists since 2008 and have heard the same arguments all over. :) I gave a 
talk on [Boost.]Process at C++Now in 2011, gave another one at a C++ meetup 
in Sydney this year and will give two more talks later this month at Meeting 
C++ in Berlin and a C++ meetup in Amsterdam.

[Boost.]Process is a difficult library to work on, and I want to thank 
Klemens that he dared to pick up the library and enter another review. I 
have exchanged countless of emails with Klemens this year to make sure he 
understands how we got to where we are. While Klemens made some changes to 
the version I handed over to him and made his own steps forward, I helped 
him avoiding making steps backwards.

The last [Boost.]Process version I worked on is known as version 0.5. The 
documentation can be found at <http://www.highscore.de/boost/process0.5/> 
(source code is at <https://github.com/BorisSchaeling/boost-process>).

If you evaluate Klemens' version at 
<http://klemens-morgenstern.github.io/process/index.html> and have some 
extra time on your hand, it would be extremely useful if you could compare 
Klemens' version with [Boost.]Process 0.5. We went through a lot of 
incremental improvements in the past, and it would be good to know whether 
Klemens' version is yet another step in the right direction.

There were not only incremental improvements though but also tangible 
results: We always wanted to support asynchronous I/O in [Boost.]Process. 
This included waiting asynchronously for a child process to exit. So we 
worked on a new I/O service object to support asynchronous waiting on 
Windows. The I/O service object was added to Boost.Asio in 2012 (known as 
boost::asio::windows::object_handle).

A huge step forward was the introduction of executors and initializers - an 
idea from Jeff Flinn in 2011. All [Boost.]Process versions since then are 
based on executors and initializers. His idea changed the design of 
[Boost.]Process quite a lot as it provides a generic solution to the problem 
of how to make [Boost.]Process extensible. It is important to understand 
Jeff's idea and the problem it solves as any other C++ process management 
library has to solve the very same problem to be at least as good in that 
aspect.

It turns out that when creating a child process an infinite number of 
configuration options can be set. The Windows and Linux functions being 
called when a child process is created accept only a limited number of 
arguments. If however, for example, the root directory of a child process on 
Linux should be changed, the function chroot() has to be called. The 
configuration option "root directory" is set through an additional function 
call. It's easy to create a C++ process management library which just 
supports a few cross-platform configuration options. It's much more 
difficult to create a C++ process management library which allows a library 
user to customize the process of creating child processes any way he likes. 
Jeff's idea was to use objects as named parameters. If [Boost.]Process 
provides a function which accepts any objects (or actually any types), 
library users can create their own named parameters and pass them to that 
function. They could for example create a named parameter which changes the 
root directory for a child process.

Jeff's idea isn't necessarily the only solution to how to provide an 
extensible interface. And of course one could argue that this is a 
negligible problem a process management library doesn't need to solve. For 
[Boost.]Process extensibility has always been one of the major goals though. 
We want to avoid that developers start using [Boost.]Process only to find 
out later that they can't do what they need to do because the library isn't 
extensible enough. Ideally any alternative process management library states 
if and how it solves this problem.

Here are some of the other not so obvious goals of [Boost.]Process:

* It should be entirely clear to a library user how resources are managed 
(which resources could leak when and how, and how the library helps a user 
to avoid leaking resources; and I'm not talking about resources you could 
manage with a smart pointer or alike).

* It should be possible to do asynchronous I/O (this includes reading, 
writing and waiting for child processes to exit).

* Similar concepts found across operating systems should be represented by 
one concept (for example, standard streams), different concepts should be 
represented by different concepts (for example, signals on Linux and object 
handles on Windows - these are the OS services used for asynchronous 
waiting).

I'm not going to discuss these goals and the current solutions in detail 
(this email is already too long; I will talk about resource management in 
[Boost.]Process at Meeting C++ in case anyone wants to know more).

If you look at the history of this library, it's surprising how far we got. 
As of today [Boost.]Process is still not where I believe it should be. 
However I believe there is a greater chance that [Boost.]Process evolves 
faster within the Boost community. After all there is considerable interest 
in a process management library in Boost. That said I vote for including the 
library in Boost because I hope there will be more momentum and a more 
continuous interest from the Boost community. This again will hopefully lead 
to better versions of [Boost.]Process faster than it did in the past 10 
years. Whether this is a good enough reason to include the library in Boost 
Antony should decide. :)

Boris