Re: [boost] Boost.Process 0.5: Another update/potential candidate for an official library

Subject: Re: [boost] Boost.Process 0.5: Another update/potential candidate for an official library
From: Yakov Galka (ybungalobill_at_[hidden])
Date: 2012-11-13 17:42:08

On Wed, Nov 7, 2012 at 10:47 PM, Boris Schaeling <boris_at_[hidden]> wrote:

> I've uploaded a new version of Boost.Process 0.5 to <
> http://www.highscore.de/boost/process0.5/>:
>

I do not see the documentation mentioning thread safety, although the
implementation *is not* thread safe.

The following is original research done by me when implementing my own
process library. I could not find any references for this issue, and in
fact for most applications this is not a problem. Yet, I think that my duty
is to share this, and as a library author, you should give it a thought.

Consider the following function (taken directly from the FAQ):

void f(const std::string &path)
{
    boost::process::pipe p = create_pipe();
    { // 1
        file_descriptor_sink sink(p.sink, close_handle);
        execute(
            run_exe(path),
            bind_stdout(sink)
        ); // 2
    } // 3
    file_descriptor_source source(p.source, close_handle);
    stream<file_descriptor_source> is(source);
    std::string s;
    while (is >> s)
        std::cout << s << std::endl;
}

Let there be two threads A and B, one calling f("a.exe") and the other
f("b.exe"). Assume a.exe being a long living program and b.exe a short
living one. (A more realistic scenario is that we launch a.exe, do short
communication, and then detaching from it, that is closing the pipe on the
parent side and letting the child run indefinitely. For example it may be
an external app launched by the user from a GUI thread. b.exe may be some
other utility used internally by our program launched by a back-ground
thread.)

Now, consider the following execution order: A1, B1, A2, B2, A3, B3. The
result is that the write-end of the pipe of b.exe is inherited by a.exe too
(i.e. 'leaked' into a.exe). b.exe exits, closing its write-end of the pipe.
However, a.exe still holds (indefinitely) the write-end of b's pipe!
Result: thread B will hang until the unrelated process a.exe exits (and
perhaps all the children that a.exe created in its own turn).

Morale: The idea that the inheritability of the handle is a global state is
fundamentally flowed. It was fine on UNIX when one process meant one
thread, but it just does not fit into the world of multi-threaded
applications + synchronous i/o. I guess Windows inherited the concept from
UNIX.

How can it be solved? Well, on Windows, I know only one way to ensure that
a handle gets inherited directly by one process only. Unfortunately it
relies on undocumented features (create suspended process, DuplicateHandle,
Read/WriteProcessMemory, resume process). Something similar should be
implementable on other platforms too.

Does it worth it? Do not know. Perhaps just declare binding child i/o to be
thread unsafe.

DISCLAIMER: All the said above was verified on Windows, although not with
this library, but the test code did essentially the same. I *am not* sure
about other platforms, but to my understanding POSIX suffers from exactly
the same problem. If someone can refute my claims, please enlighten me.

Hope this helps,
-- 
Yakov